WO2024030004A1

WO2024030004A1 - Optical device, electronic device including same, display method, control method, and control device

Info

Publication number: WO2024030004A1
Application number: PCT/KR2023/011566
Authority: WO
Inventors: 권진호; 김민선
Original assignee: 엘지이노텍 주식회사
Priority date: 2022-08-05
Filing date: 2023-08-07
Publication date: 2024-02-08

Abstract

Disclosed in an embodiment is an optical device including: a plurality of lenses; a barrel in which the plurality of lenses are arranged; a first light guide disposed in the barrel; an opening formed in the side of the barrel; a light source device coupled to the opening; a through-hole formed in the side of the barrel; and a light-receiving portion arranged adjacent to the through-hole.

Description

Optical devices, electronic devices including the same, display methods, control methods and control devices

Embodiments relate to optical devices, electronic devices including them, display methods, control methods, and control devices.

Virtual Reality (VR) refers to a specific environment or situation, or the technology itself, that is similar to reality but is not real, created through artificial technology using computers.

Augmented Reality (AR) refers to a technology that synthesizes virtual objects or information in the real environment to make them look like objects that exist in the original environment.

Mixed Reality (MR) or Hybrid reality refers to combining the virtual world and the real world to create a new environment or new information. In particular, it is called mixed reality when it refers to real-time interaction between reality and virtual reality.

At this time, the created virtual environment or situation stimulates the user's five senses and provides spatial and temporal experiences similar to reality, allowing the user to freely move between reality and imagination. In addition, users can not only immerse themselves in this environment, but also interact with things implemented in this environment, such as manipulating or giving commands using real devices.

Recently, research on equipment (gear, devices) used in these technical fields has been actively conducted. However, the need to detect wearing of such equipment and to detect the user's biological signals is emerging.

In addition, such equipment requires adjustment based on the user's vision, and the need for such vision adjustment is emerging. Furthermore, there is a need for simple controllers for these devices. Additionally, there is an emerging need to provide additional functions for these devices.

Embodiments provide an optical device and an electronic device that accurately detect a user's biosignals when using an optical device used in AR (Augmented Reality), etc., and an electronic device including the same.

Additionally, an optical device with reduced noise for sensing (biological) signals and an electronic device including the same are provided.

Additionally, an optical device that can accurately detect whether a user is wearing it and an electronic device including the same are provided.

Additionally, an optical device with improved power efficiency and an electronic device including the same are provided.

Additionally, an electronic device and display method that provides images based on the user's vision are provided.

In addition, an electronic device and display method are provided that allow users to more comfortably perceive virtual images, etc. through images selected by the user.

Additionally, a control device and control method are provided to reduce malfunctions by activating input types in response to the user's status.

In addition, a control device and control method that can intuitively control an electronic device without the attention of others are provided.

Additionally, an optical device and an electronic device that can more easily suppress the fogging phenomenon are provided.

Additionally, an optical device with improved reliability through heat dissipation and an electronic device including the same are provided.

Additionally, an optical device with improved energy efficiency and an electronic device including the same are provided.

The problem to be solved in the embodiment is not limited to this, and it will also include means of solving the problem described below and purposes and effects that can be understood from the embodiment.

An optical device according to an embodiment includes a plurality of lenses; a barrel on which the plurality of lenses are disposed; a first light guide disposed on the barrel; an opening formed on a side of the barrel; a light source device coupled to the opening; a through hole formed on a side of the barrel; and a light receiving unit disposed adjacent to the through hole.

It may include a barrier portion disposed between the through hole and the light receiving portion.

and an optical signal generator that generates an optical signal including image information, wherein the light receiver may be disposed between the first light guide and the optical signal generator.

The plurality of lenses include a first lens and a second lens disposed between the first light guide and the optical signal generator, and the first lens may be smaller in size than the second lens.

The light receiving unit may be disposed between the first lens and the second lens.

The through hole may be disposed between the first lens and the second lens.

The light receiving unit may protrude beyond the outer surface of the barrel.

The barrier portion may protrude beyond the outer surface of the barrel.

The light source device includes a housing having an opening; a second light guide disposed in the housing; and a light source that emits light toward the second light guide.

The light receiving unit may be driven while light is emitted from the light source.

It may include a control unit that controls the light source device, the optical signal generator, and the light receiver.

The control unit may receive a first light receiving signal from the light receiving unit when there is no current applied to the light source device.

The control unit may determine whether to drive the light source device by comparing the first light reception signal and a first set value.

When the first light-receiving signal is smaller than the first set value, the control unit may drive the light source and receive a second light-receiving signal from the light receiving unit.

The control unit may compare the second light reception signal with a second set value and receive a third light reception signal from the light reception unit in response to the operations of the light source and the optical signal generator.

An electronic device according to an embodiment includes a frame; an input unit disposed in the frame and outputting an input signal; An optical device disposed in the frame to generate an image; a display unit connected to the optical device and outputting an image generated by the optical device; and a control unit that controls an image signal for the image, wherein the control unit provides image signals of test images with different focal lengths and determines the user's visual acuity based on the image selected by the input unit.

Inspection images with different focal lengths may be adjusted by the input unit.

There may be a plurality of inspection images having different focal lengths, and each of the plurality of inspection images may have a different focal length.

The selected image may be at least one of the plurality of inspection images.

After determining the user's visual acuity, the control unit may adjust and output the focal distance for the image based on the determined user's visual acuity.

When there are multiple images selected, the controller may determine the user's visual acuity based on the maximum focal length.

The inspection images having different focal lengths may include a plurality of objects having different focal lengths in one image.

The control unit may adjust the focal length of the inspection image by moving the lens in the optical device.

and an optical element disposed on the frame, wherein the optical element may include a variable focus lens.

The control unit may change the curvature of the optical element.

A display method according to an embodiment includes outputting inspection images with different focal lengths; determining the user's visual acuity based on an image selected from the inspection images having different focal lengths; and outputting a virtual image based on the determined visual acuity of the user.

After determining the visual acuity, the method may include storing the determined visual acuity of the user.

The determining step may include outputting a plurality of inspection images with different focal lengths.

The step of outputting the virtual image may include changing the focal distance for the virtual image based on the determined visual acuity of the user.

The determining step may include selecting at least one image from among the plurality of inspection images having different focal lengths; and determining the user's visual acuity based on the maximum focal distance when there are multiple selected images.

A control device according to an embodiment includes a sensing unit that outputs a sensing signal including motion information; and a controller; wherein the controller determines the user state from the exercise information and determines activation of the gesture operation and the touch operation in response to the user state.

When the intensity of the detection signal is greater than the first set value, the controller may compare whether the amplitude of the detection signal is greater than the second set value.

If the amplitude of the detection signal is greater than a second set value, the controller may determine the user state to be an exercise state.

If the amplitude of the detection signal is less than a second set value, the controller may compare whether the intensity of the detection signal is greater than a third set value.

If the intensity of the detection signal is greater than the third set value, the controller determines the user state to be a boarding state, and the third set value may be greater than the first set value.

If the intensity of the detection signal is less than the first set value, the controller compares whether the amplitude of the detection signal is greater than the fourth set value, and the fourth set value may be smaller than the second set value. .

The controller may determine the user state as a moving state if the amplitude of the detection signal is greater than the fourth set value.

The controller may determine the user state to be a stopped state if the amplitude of the detection signal is less than the fourth set value.

When the touch operation is activated, the controller can set the position of the user's initial input to any of the top, bottom, left, and right with respect to the central part.

A control method according to an embodiment includes receiving a detection signal; determining user status; determining activation of a gesture action or a touch action in response to the determined user state; and transmitting a command in response to the activated operation.

The step of determining the user state may include comparing whether the amplitude of the detection signal is greater than a second set value when the intensity of the detection signal is greater than a first set value.

If the amplitude of the detection signal is greater than a second set value, determining the user state as an exercise state may include.

If the amplitude of the detection signal is less than a second set value, comparing whether the intensity of the detection signal is greater than the third set value may include.

If the strength of the detection signal is greater than the third set value, determining the user state as a boarding state;

The third set value may be greater than the first set value.

If the intensity of the detection signal is less than the first set value, comparing whether the amplitude of the detection signal is greater than the fourth set value, wherein the fourth set value may be less than the second set value. there is.

If the amplitude of the detection signal is greater than the fourth set value, determining the user state as a moving state may include.

If the amplitude of the detection signal is less than the fourth set value, determining the user state as a stationary state may include.

An electronic device according to an embodiment includes a frame; An optical device disposed in the frame to generate an image; a display unit connected to the optical device and outputting an image generated by the optical device; and a heat dissipation unit connected to the optical device and disposed inside the frame.

The optical device includes a barrel on which a plurality of lenses are disposed; an opening formed on a side of the barrel; It may include a light source device coupled to the opening.

The light source device is

a housing having an opening; a light source emitting light toward the second light guide; and

It may include a substrate connected to the light source.

The heat dissipation unit may be in contact with the substrate.

The heat dissipation portion may extend from the outer surface of the housing to the frame.

The optical device may include an optical signal generator that generates an optical signal including image information, and the heat dissipation portion may extend adjacent to the optical signal generator.

It may include a heat supply unit disposed on the frame.

The heat supply unit and the heat dissipation unit may be spaced apart from each other.

The frame has an upper area; and a lower region disposed below the upper region, wherein one of the heat supply unit and the heat dissipation portion is disposed in the upper region, and the other one of the heat supply unit and the heat dissipation portion is disposed in the lower region. You can.

The heat dissipation unit may be disposed along an outer surface of the frame.

According to an embodiment, an optical device and an electronic device that accurately detect a user's biological signals are implemented when using an optical device used in AR (Augmented Reality), etc., and an electronic device including the same.

Additionally, an optical device with reduced noise for sensing (biological) signals and an electronic device including the same can be implemented.

Additionally, an optical device that can accurately detect whether a user is wearing it and an electronic device including the same can be implemented.

Additionally, an optical device with improved power efficiency and an electronic device including the same can be implemented.

Additionally, in electronic devices and display methods used in AR (Augmented Reality), etc., it is possible to implement electronic devices and display methods that provide images based on the user's vision.

Additionally, it is possible to implement an electronic device and display method that allows users to more comfortably perceive virtual images, etc. through images selected by the user.

Additionally, the embodiment implements a control device and control method that reduces malfunctions by activating input types in response to the user's status for electronic devices used in AR (Augmented Reality), etc.

Additionally, it is possible to implement a control device and control method that can intuitively control electronic devices without the attention of others.

In addition, optical devices and electronic devices that can more easily suppress the fogging phenomenon are implemented.

Additionally, an optical device with improved reliability through heat dissipation and an electronic device including the same can be implemented.

Additionally, an optical device with improved energy efficiency and an electronic device including the same can be implemented.

The various and beneficial advantages and effects of the present invention are not limited to the above-described content, and may be more easily understood through description of specific embodiments of the present invention.

1 is a conceptual diagram showing an embodiment of an AI device,

Figure 2 is a block diagram showing the configuration of an extended reality electronic device according to an embodiment of the present invention;

Figure 3 is a perspective view of an augmented reality electronic device according to the first embodiment of the present invention;

4 to 6 are conceptual diagrams for explaining various display methods applicable to the display unit according to an embodiment of the present invention;

7 is a conceptual diagram of an electronic device and an optical device according to an embodiment;

8 is a conceptual diagram of an optical device according to an embodiment;

9 is a perspective view of an optical device according to an embodiment;

Figure 10 is an exploded perspective view of an optical device according to an embodiment;

Figure 11 is a view taken along line AA' in Figure 9,

Figure 12 is an enlarged view of portion K1 in Figure 11,

Figure 13 is an enlarged view of portion K2 in Figure 11,

14 is another example of a light source device according to an embodiment,

15 is a flowchart of a method of operating an electronic device or optical device according to an embodiment;

16 is a timing diagram explaining the operations of the light source, optical signal generator, and light receiver;

17 is a flowchart of a display method according to an embodiment;

18 is an additional flow chart for a display method according to an embodiment;

19 and 20 are diagrams illustrating visual acuity determination in a display method according to an embodiment;

21 is a diagram illustrating image output in a display method according to an embodiment;

22 is a flowchart of a display method according to another embodiment;

23 is a diagram illustrating visual acuity determination in a display method according to another embodiment;

24 is a flowchart of a display method according to another embodiment;

Figure 25 is a perspective view of an augmented reality electronic device according to an embodiment according to a modified example;

Figure 26 is a block diagram of a control device according to an embodiment of the present invention;

27 is a conceptual diagram of a control device according to an embodiment of the present invention,

28 is a flowchart of a control method according to an embodiment of the present invention;

Figure 29 is a specific flowchart of a control method according to an embodiment of the present invention;

30 is a diagram illustrating the operation and control method of a control device according to an embodiment of the present invention;

31 is a diagram illustrating the operation and control between a control device and an electronic device according to an embodiment of the present invention;

Figure 32 is a diagram showing the interface of an electronic device according to an embodiment of the present invention;

33 is an example diagram illustrating the operation of an electronic device using a control device according to an embodiment of the present invention;

Figure 34 is a diagram explaining the operation of a control device according to an embodiment of the present invention;

35 is a side view of an electronic device according to an embodiment;

36 is a plan view of an electronic device according to an embodiment;

37 is a conceptual diagram of an optical device according to an embodiment;

38 is a perspective view of an optical device according to an embodiment;

Figure 39 is an exploded perspective view of an optical device according to an embodiment;

Figure 40 is a view taken along line AA' in Figure 38,

Figure 41 is an enlarged view of portion K1 in Figure 40,

Figure 42 is an enlarged view of portion K2 in Figure 40;

43 is a side view of an electronic device according to another embodiment;

44 is a plan view of an electronic device according to another embodiment;

45 is a side view of an electronic device according to another embodiment;

Figure 46 is a plan view of an electronic device according to another embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

However, the technical idea of the present invention is not limited to some of the described embodiments, but may be implemented in various different forms, and as long as it is within the scope of the technical idea of the present invention, one or more of the components may be optionally used between the embodiments. It can be used by combining and replacing.

In addition, terms (including technical and scientific terms) used in the embodiments of the present invention, unless explicitly specifically defined and described, are generally understood by those skilled in the art to which the present invention pertains. It can be interpreted as meaning, and the meaning of commonly used terms, such as terms defined in a dictionary, can be interpreted by considering the contextual meaning of the related technology.

Additionally, the terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention.

In this specification, the singular may also include the plural unless specifically stated in the phrase, and when described as “at least one (or more than one) of A and B and C”, it is combined with A, B, and C. It can contain one or more of all possible combinations.

Additionally, when describing the components of an embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used.

These terms are only used to distinguish the component from other components, and are not limited to the essence, sequence, or order of the component.

And, when a component is described as being 'connected', 'coupled' or 'connected' to another component, the component is not only directly connected, coupled or connected to that other component, but also It can also include cases where other components are 'connected', 'combined', or 'connected' due to another component between them.

Additionally, when described as being formed or disposed "above" or "below" each component, "above" or "below" refers not only to cases where two components are in direct contact with each other, but also to one This also includes cases where another component described above is formed or placed between two components. In addition, when expressed as "top (above) or bottom (bottom)", it may include not only the upward direction but also the downward direction based on one component.

1 is a conceptual diagram showing an embodiment of an AI device.

Referring to Figure 1, the AI system includes at least one of an AI server 16, a robot 11, an autonomous vehicle 12, an XR device 13, a smartphone 14, or a home appliance 15 connected to a cloud network. It is connected to (10). Here, a robot 11, an autonomous vehicle 12, an XR device 13, a smartphone 14, or a home appliance 15 to which AI technology is applied may be referred to as AI devices 11 to 15.

The cloud network 10 may constitute part of a cloud computing infrastructure or may refer to a network that exists within the cloud computing infrastructure. Here, the cloud network 10 may be configured using a 3G network, 4G, Long Term Evolution (LTE) network, or 5G network.

That is, each device 11 to 16 constituting the AI system can be connected to each other through the cloud network 10. In particular, the devices 11 to 16 may communicate with each other through a base station, but may also communicate with each other directly without going through the base station.

The AI server 16 may include a server that performs AI processing and a server that performs calculations on big data.

The AI server 16 is connected to at least one of the AI devices that make up the AI system, such as a robot 11, an autonomous vehicle 12, an XR device 13, a smartphone 14, or a home appliance 15, and a cloud network ( It is connected through 10) and can assist at least part of the AI processing of the connected AI devices 11 to 15.

At this time, the AI server 16 can train an artificial neural network according to a machine learning algorithm on behalf of the AI devices 11 to 15, and directly store or transmit the learning model to the AI devices 11 to 15.

The AI server 16 receives input data from the AI devices 11 to 15, infers a result value for the received input data using a learning model, and generates a response or control command based on the inferred result value. It can be transmitted to AI devices (11 to 15).

Alternatively, the AI devices 11 to 15 may infer a result value for input data using a direct learning model and generate a response or control command based on the inferred result value.

<AI+Robot>

The robot 11 uses AI technology and can be implemented as a guidance robot, a transport robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, etc.

The robot 11 may include a robot control module for controlling operations, and the robot control module may mean a software module or a chip implementing it as hardware.

The robot 11 uses sensor information obtained from various types of sensors to acquire status information of the robot 11, detect (recognize) the surrounding environment and objects, generate map data, or determine movement path and driving. It can determine a plan, determine a response to user interaction, or determine an action.

Here, the robot 11 may use sensor information obtained from at least one sensor among lidar, radar, and camera to determine the movement path and driving plan.

The robot 11 can perform the above operations using a learning model composed of at least one artificial neural network. For example, the robot 11 can recognize the surrounding environment and objects using a learning model, and can determine an operation using the recognized surrounding environment information or object information. Here, the learning model may be learned directly from the robot 11 or from an external device such as the AI server 16.

At this time, the robot 11 may perform an action by generating a result using a direct learning model, but it also transmits sensor information to an external device such as the AI server 16 and receives the result generated accordingly to perform the action. It can also be done.

The robot 11 determines the movement path and driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and controls the driving unit to follow the determined movement path and driving plan. The robot 11 can be driven accordingly.

The map data may include object identification information about various objects arranged in the space where the robot 11 moves. For example, map data may include object identification information for fixed objects such as walls and doors and movable objects such as flower pots and desks. Additionally, object identification information may include name, type, distance, location, etc.

Additionally, the robot 11 can perform actions or travel by controlling the driving unit based on the user's control/interaction. At this time, the robot 11 may acquire interaction intention information according to the user's motion or voice utterance, determine a response based on the acquired intention information, and perform the operation.

<AI+Autonomous Driving>

The self-driving vehicle 12 can be implemented as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying AI technology.

The autonomous vehicle 12 may include an autonomous driving control module for controlling autonomous driving functions, and the autonomous driving control module may refer to a software module or a chip implementing it as hardware. The self-driving control module may be included internally as a component of the self-driving vehicle 12, but may also be configured as separate hardware and connected to the outside of the self-driving vehicle 12.

The self-driving vehicle 12 uses sensor information obtained from various types of sensors to obtain status information of the self-driving vehicle 12, detect (recognize) the surrounding environment and objects, generate map data, or You can determine the movement route and driving plan, or determine the action.

Here, the autonomous vehicle 12, like the robot 11, can use sensor information acquired from at least one sensor among lidar, radar, and camera to determine the movement path and driving plan.

In particular, the autonomous vehicle 12 can recognize the environment or objects in areas where the view is obscured or over a certain distance by receiving sensor information from external devices, or receive recognized information directly from external devices. .

The autonomous vehicle 12 can perform the above operations using a learning model composed of at least one artificial neural network. For example, the self-driving vehicle 12 can recognize the surrounding environment and objects using a learning model, and can determine a driving route using the recognized surrounding environment information or object information. Here, the learning model may be learned directly from the autonomous vehicle 12 or from an external device such as the AI server 16.

At this time, the autonomous vehicle 12 may perform operations by generating results using a direct learning model, but it may also transmit sensor information to an external device such as the AI server 16 and receive the results generated accordingly. You can also perform actions.

The autonomous vehicle 12 determines the movement path and driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and controls the driving unit to determine the determined movement path and driving. The autonomous vehicle 12 can be driven according to a plan.

The map data may include object identification information about various objects placed in the space (eg, road) where the autonomous vehicle 12 drives. For example, map data may include object identification information for fixed objects such as streetlights, rocks, and buildings, and movable objects such as vehicles and pedestrians. Additionally, object identification information may include name, type, distance, location, etc.

Additionally, the autonomous vehicle 12 can perform operations or drive by controlling the driving unit based on the user's control/interaction. At this time, the autonomous vehicle 12 may acquire interaction intention information according to the user's motion or voice utterance, determine a response based on the acquired intention information, and perform the operation.

<AI+XR>

The XR device 13 is equipped with AI technology and can be used for HMD (Head-Mount Display), HUD (Head-Up Display) installed in vehicles, televisions, mobile phones, smart phones, computers, wearable devices, home appliances, and digital signage. , it can be implemented as a vehicle, a fixed robot, or a mobile robot.

The XR device 13 analyzes 3D point cloud data or image data acquired through various sensors or from external devices to generate location data and attribute data for 3D points, thereby providing information about surrounding space or real objects. The XR object to be acquired and output can be rendered and output. For example, the XR device 13 may output an XR object containing additional information about the recognized object in correspondence to the recognized object.

The XR device 13 may perform the above operations using a learning model composed of at least one artificial neural network. For example, the XR device 13 can recognize a real-world object from 3D point cloud data or image data using a learning model and provide information corresponding to the recognized real-world object. Here, the learning model may be learned directly from the XR device 13 or from an external device such as the AI server 16.

At this time, the XR device 13 may perform operations by generating results using a direct learning model, but operates by transmitting sensor information to an external device such as the AI server 16 and receiving the results generated accordingly. You can also perform .

<AI+Robot+Autonomous Driving>

The robot 11 applies AI technology and autonomous driving technology and can be implemented as a guidance robot, a transport robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, etc.

The robot 11 to which AI technology and autonomous driving technology is applied may refer to the robot itself with autonomous driving functions or the robot 11 that interacts with the autonomous vehicle 12.

The robot 11 with autonomous driving function can refer to devices that move on their own according to a given route without user control, or determine the route on their own and move.

The robot 11 and the autonomous vehicle 12 with autonomous driving functions may use a common sensing method to determine one or more of a movement path or a driving plan. For example, the robot 11 and the autonomous vehicle 12 with autonomous driving functions can determine one or more of the movement path or driving plan using information sensed through lidar, radar, and cameras.

The robot 11 interacting with the autonomous vehicle 12 exists separately from the autonomous vehicle 12 and is linked to the autonomous driving function inside or outside the autonomous vehicle 12, or is connected to the autonomous driving function inside or outside the autonomous vehicle 12. ) can perform actions linked to the user on board.

At this time, the robot 11 interacting with the autonomous vehicle 12 acquires sensor information on behalf of the autonomous vehicle 12 and provides it to the autonomous vehicle 12, or acquires sensor information and provides surrounding environment information. Alternatively, the autonomous driving function of the autonomous vehicle 12 can be controlled or assisted by generating object information and providing it to the autonomous vehicle 12.

Alternatively, the robot 11 interacting with the autonomous vehicle 12 may monitor the user riding the autonomous vehicle 12 or control the functions of the autonomous vehicle 12 through interaction with the user. . For example, if it is determined that the driver is drowsy, the robot 11 may activate the autonomous driving function of the autonomous vehicle 12 or assist in controlling the driving unit of the autonomous vehicle 12. Here, the functions of the autonomous vehicle 12 controlled by the robot 11 may include not only the autonomous driving function but also functions provided by a navigation system or audio system provided inside the autonomous vehicle 12.

Alternatively, the robot 11 interacting with the autonomous vehicle 12 may provide information to the autonomous vehicle 12 or assist a function from outside the autonomous vehicle 12 . For example, the robot 11 may provide traffic information including signal information to the autonomous vehicle 12, such as a smart traffic light, and may interact with the autonomous vehicle 12, such as an automatic electric charger for an electric vehicle. You can also automatically connect an electric charger to the charging port.

<AI+Robot+XR>

The robot 11 applies AI technology and XR technology and can be implemented as a guidance robot, a transport robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, a drone, etc.

The robot 11 to which XR technology is applied may refer to a robot that is subject to control/interaction within an XR image. In this case, the robot 11 is distinct from the XR device 13 and can be interoperated with each other.

When the robot 11, which is the object of control/interaction within the XR image, acquires sensor information from sensors including a camera, the robot 11 or the XR device 13 generates an XR image based on the sensor information. And, the XR device 13 can output the generated XR image. And, this robot 11 can operate based on a control signal input through the XR device 13 or user interaction.

For example, the user can check the XR image corresponding to the viewpoint of the remotely linked robot 11 through an external device such as the XR device 13, and adjust the autonomous driving path of the robot 11 through interaction. , you can control movement or driving, or check information about surrounding objects.

<AI+Autonomous Driving+XR>

The self-driving vehicle 12 can be implemented as a mobile robot, vehicle, unmanned aerial vehicle, etc. by applying AI technology and XR technology.

The autonomous vehicle 12 to which XR technology is applied may refer to an autonomous vehicle equipped with means for providing XR images or an autonomous vehicle that is subject to control/interaction within XR images. In particular, the autonomous vehicle 12, which is the subject of control/interaction within the XR image, is distinct from the XR device 13 and can be interoperable with each other.

The autonomous vehicle 12 equipped with a means for providing an XR image can acquire sensor information from sensors including a camera and output an XR image generated based on the acquired sensor information. For example, the self-driving vehicle 12 may be equipped with a HUD and output XR images, thereby providing passengers with XR objects corresponding to real objects or objects on the screen.

At this time, when the XR object is output to the HUD, at least a portion of the XR object may be output to overlap the actual object toward which the passenger's gaze is directed. On the other hand, when the XR object is output to a display provided inside the autonomous vehicle 12, at least a portion of the XR object may be output to overlap the object in the screen. For example, the autonomous vehicle 12 may output XR objects corresponding to objects such as lanes, other vehicles, traffic lights, traffic signs, two-wheeled vehicles, pedestrians, buildings, etc.

When the autonomous vehicle 12, which is the subject of control/interaction within the XR image, acquires sensor information from sensors including cameras, the autonomous vehicle 12 or the XR device 13 detects sensor information based on the sensor information. An XR image is generated, and the XR device 13 can output the generated XR image. Additionally, this autonomous vehicle 12 may operate based on control signals input through an external device such as the XR device 13 or user interaction.

[Extended reality technology]

Extended reality (XR: eXtended Reality) is a general term for virtual reality (VR), augmented reality (AR), and mixed reality (MR: Mixed reality). VR technology provides objects and backgrounds in the real world only as CG images, AR technology provides virtual CG images on top of images of real objects, and MR technology provides computer technology that mixes and combines virtual objects in the real world. It is a graphic technology.

MR technology is similar to AR technology in that it shows real objects and virtual objects together. However, in AR technology, virtual objects are used to complement real objects, whereas in MR technology, virtual objects and real objects are used equally.

XR technology can be applied to HMD (Head-Mount Display), HUD (Head-Up Display), mobile phones, tablet PCs, laptops, desktops, TVs, digital signage, etc., and devices with XR technology applied are called XR Devices. It can be called.

Hereinafter, an electronic device that provides extended reality according to an embodiment of the present invention will be described. In particular, optical devices applied to augmented reality and electronic devices including them are explained in detail.

Figure 2 is a block diagram showing the configuration of an extended reality electronic device 20 according to an embodiment of the present invention.

Referring to FIG. 2, the extended reality electronic device 20 includes a wireless communication unit 21, an input unit 22, a sensing unit 23, an output unit 24, an interface unit 25, a memory 26, and a control unit ( 27) and a power supply unit 28. The components shown in FIG. 2 are not essential for implementing the electronic device 20, so the electronic device 20 described herein may have more or fewer components than those listed above. .

More specifically, among the above components, the wireless communication unit 21 is a wireless communication system between the electronic device 20 and the wireless communication system, between the electronic device 20 and another electronic device, or between the electronic device 20 and an external server. It may contain one or more modules that enable communication. Additionally, the wireless communication unit 21 may include one or more modules that connect the electronic device 20 to one or more networks.

This wireless communication unit 21 may include at least one of a broadcast reception module, a mobile communication module, a wireless Internet module, a short-range communication module, and a location information module.

The input unit 22 includes a camera or video input unit for inputting video signals, a microphone or audio input unit for inputting audio signals, and a user input unit (for example, a touch key) for receiving information from the user. , pushkey (mechanical key, etc.) may be included. Voice data or image data collected from the input unit 22 may be analyzed and processed as a user's control command.

The sensing unit 23 may include one or more sensors for sensing at least one of information within the electronic device 20, information on the surrounding environment surrounding the electronic device 20, and user information.

For example, the sensing unit 23 includes a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor (G- sensor, gyroscope sensor, motion sensor, RGB sensor, infrared sensor, fingerprint scan sensor, ultrasonic sensor, optical sensor ( optical sensor (e.g., imaging device), microphone, battery gauge, environmental sensor (e.g., barometer, hygrometer, thermometer, radiation detection sensor, heat detection sensor, gas detection sensor, etc.), It may include at least one of chemical sensors (eg, electronic nose, healthcare sensor, biometric sensor, etc.). Meanwhile, the electronic device 20 disclosed in this specification can utilize information sensed by at least two of these sensors by combining them.

The output unit 24 is intended to generate output related to vision, hearing, or tactile sensation, and may include at least one of a display unit, an audio output unit, a haptic module, and an optical output unit. A touch screen can be implemented by forming a layered structure with the touch sensor or being integrated with the display unit. This touch screen functions as a user input means that provides an input interface between the augmented reality electronic device 20 and the user, and can simultaneously provide an output interface between the augmented reality electronic device 20 and the user.

The interface unit 25 serves as a passageway for various types of external devices connected to the electronic device 20. Through the interface unit 25, the electronic device 20 can receive virtual reality or augmented reality content from an external device, and can perform mutual interaction by exchanging various input signals, sensing signals, and data.

For example, the interface unit 25 includes a device equipped with a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and an identification module. It may include at least one of a connection port, an audio input/output (I/O) port, a video input/output (I/O) port, and an earphone port.

Additionally, the memory 26 stores data that supports various functions of the electronic device 20. Memory 26 may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and/or other storage devices. The memory 26 may store a plurality of application programs (application programs or applications) running on the electronic device 20, data for operating the electronic device 20, and commands. At least some of these applications may be downloaded from an external server via wireless communication. Additionally, at least some of these applications may be present on the electronic device 20 from the time of shipment for basic functions of the electronic device 20 (e.g., call incoming and outgoing functions, message receiving and sending functions).

The control unit 27 typically controls the overall operation of the electronic device 20 in addition to operations related to application programs. The control unit 27 can process signals, data, information, etc. that are input or output through the components discussed above.

Additionally, the control unit 27 can control at least some of the components by running the application program stored in the memory 26 to provide appropriate information to the user or process functions. Furthermore, the control unit 27 may operate at least two of the components included in the electronic device 20 in combination with each other in order to run an application program. Furthermore, the control unit 27 can perform processing for a driving method described later. This control unit 27 may be expressed as a processor, control unit, etc. Additionally, the control unit 27 may include an application-specific integrated circuit (ASIC), another chipset, logic circuit, and/or a data processing device.

Additionally, the control unit 27 may sense the movement of the electronic device 20 or the user using a gyroscope sensor, gravity sensor, motion sensor, etc. included in the sensing unit 23. Alternatively, the control unit 27 may detect an object approaching the electronic device 20 or the user using a proximity sensor, illuminance sensor, magnetic sensor, infrared sensor, ultrasonic sensor, light sensor, etc. included in the sensing unit 23. there is. In addition, the control unit 27 can also detect the user's movement through sensors provided in the controller that operates in conjunction with the electronic device 20.

Additionally, the control unit 27 may perform an operation (or function) of the electronic device 20 using an application program stored in the memory 26.

The power supply unit 28 receives external or internal power under the control of the control unit 27 and supplies power to each component included in the electronic device 20. The power supply unit 28 includes a battery, and the battery may be provided in a built-in or replaceable form.

At least some of the above components may operate in cooperation with each other to implement operation, control, or a control method of an electronic device according to various embodiments described below. Additionally, the operation, control, or control method of the electronic device may be implemented on the electronic device by running at least one application program stored in the memory 26.

Hereinafter, an electronic device described as an example of the present invention will be described based on an embodiment applied to a Head Mounted Display (HMD). However, embodiments of the electronic device according to the present invention include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, and slate PCs ( Slate PC, tablet PC, ultrabook, and wearable device may be included. In addition to HMD, wearable devices may include smart watches, contact lenses, VR/AR/MR Glass, etc.

Figure 3 is a perspective view of an augmented reality electronic device according to an embodiment of the present invention.

As shown in FIG. 3, an electronic device according to an embodiment of the present invention may include a frame 100, an optical device 200, and a display unit 300.

The electronic device may be provided as a glass type (smart glass). A glass-type electronic device is configured to be worn on the head of the human body, and may be provided with a frame (case, housing, etc.) 100 for it. The frame 100 may be made of a flexible material to make it easy to wear.

The frame 100 is supported on the head and provides space for various parts to be mounted. As shown, the frame 100 may be equipped with electronic components such as an optical device 200, a user input unit 130, or an audio output unit 140. Additionally, a lens covering at least one of the left eye and the right eye may be removably mounted on the frame 100.

As shown in the drawing, the frame 100 may have the form of glasses worn on the face of the user's body, but is not necessarily limited thereto and may have the form of goggles worn in close contact with the user's face. .

Such a frame 100 includes a front frame 110 having at least one opening, and a pair of side frames 120 that extend in the y direction (in FIG. 3) intersecting the front frame 110 and are parallel to each other. It can be included.

The frame 100 may have the same or different length (DI) in the x direction and length (LI) in the y direction.

The optical device 200 is provided to control various electronic components included in an electronic device.

The optical device 200 may generate an image shown to the user or an image containing a series of images. The optical device 200 may include an image source panel that generates an image and a plurality of lenses that diffuse and converge light generated from the image source panel.

The optical device 200 may be fixed to one of the two side frames 120 . For example, the optical device 200 may be fixed to the inside or outside of one of the side frames 120, or may be built into one of the side frames 120 and formed integrally. Alternatively, the optical device 200 may be fixed to the front frame 110 or may be provided separately from the electronic device.

The display unit 300 may be implemented in the form of a head mounted display (HMD). HMD type refers to a display method that is mounted on the head and shows images directly in front of the user's eyes. When a user wears an electronic device, the display unit 300 may be arranged to correspond to at least one of the left eye and the right eye so that an image can be provided directly in front of the user's eyes. In this drawing, the display unit 300 is located in a portion corresponding to the right eye so that an image can be output toward the user's right eye. However, as described above, it is not limited to this and may be placed on both the left and right eyes.

The display unit 300 allows the user to visually perceive the external environment and at the same time allows the user to see the image generated by the optical device 200. For example, the display unit 300 may project an image onto the display area using a prism.

Additionally, the display unit 300 may be formed to be translucent so that the projected image and the general front view (range viewed through the user's eyes) are visible at the same time. For example, the display unit 300 may be translucent and may be formed of an optical member including glass.

Additionally, the display unit 300 may be inserted into and fixed to an opening included in the front frame 110, or may be located on the back of the opening (i.e., between the opening and the user) and fixed to the front frame 110. In the drawing, the display unit 300 is located on the back of the opening and is fixed to the front frame 110 as an example. However, unlike this, the display unit 300 can be placed and fixed at various positions on the frame 100. You can.

As shown in FIG. 3, when image light for an image is incident on one side of the display unit 300 from the optical device 200, the electronic device emits the image light to the other side through the display unit 300, The image generated by the device 200 can be displayed to the user.

Accordingly, the user can view the image generated by the optical device 200 while simultaneously viewing the external environment through the opening of the frame 100. That is, the image output through the display unit 300 may appear to overlap with the general field of view. Electronic devices can use these display characteristics to provide augmented reality (AR), which displays a single image by overlapping a virtual image on a real image or background.

Furthermore, in addition to this operation, the external environment and images generated by the optical device 200 may be provided to the user with a time difference for a short period of time that cannot be perceived by a person. For example, within one frame, an external environment may be provided to a person in one section, and an image from the optical device 200 may be provided to a person in another section.

Alternatively, both overlap and time difference may be provided.

4 to 6 are conceptual diagrams for explaining various display methods applicable to the display unit according to an embodiment of the present invention.

Specifically, FIG. 4 is a diagram for explaining an embodiment of a prism-type optical member, FIG. 5 is a diagram for explaining an embodiment of a waveguide (or waveguide)-type optical member, and FIG. 6 is a surface This is a drawing to explain an embodiment of a reflective optical member.

As shown in FIG. 4, a prism-type optical member may be used in the display unit 300-1 according to an embodiment of the present invention.

In an embodiment, the prismatic optical member is a flat type glass optical member in which the surface on which image light is incident and the surface 300a on which image light is emitted are flat, as shown in (a) of FIG. 4, or , As shown in (b) of FIG. 4, a freeform glass optical member in which the surface 300b from which image light is emitted is formed as a curved surface without a constant radius of curvature may be used.

The flat type glass optical member may receive image light generated by the optical device 200 from its flat side, be reflected by the total reflection mirror 300a provided therein, and be emitted toward the user. Here, the total reflection mirror 300a provided inside the flat type glass optical member may be formed inside the flat type glass optical member using a laser.

The freeform glass optical member is configured to become thinner as the distance from the incident surface increases, so that the image light generated by the optical device 200 is incident on the side having a curved surface, is totally reflected internally, and can be emitted toward the user. there is.

As shown in FIG. 5, the display unit 300-2 according to another embodiment of the present invention includes a waveguide (or waveguide) type optical member or a light guide optical element (LOE). It can be used.

An example of such a waveguide (or waveguide) or light guide type optical member is a partially reflective mirror (segmented beam splitter) type glass optical member as shown in (a) of FIG. 5 , a sawtooth prism glass optical member as shown in (b) of FIG. 5, a glass optical member having a diffractive optical element (DOE) as shown in (c) of FIG. 5, A glass optical element having a hologram optical element (HOE) as shown in (d), a glass optical element having a passive grating as shown in (e) of FIG. 5, FIG. 5 There may be a glass optical member with active grating as shown in (f).

As shown in (a) of FIG. 5, the glass optical member of the segmented beam splitter type has a total reflection mirror 301a and a light image on the side where the light image is incident inside the glass optical member. A partial reflection mirror (segmented beam splitter, 301b) may be provided on the emitting side.

Accordingly, the light image generated by the optical device 200 is totally reflected by the total reflection mirror 301a inside the glass optical member, and the totally reflected light image guides light along the longitudinal direction of the glass and is reflected by the partial reflection mirror 301b. It can be partially separated and emitted and recognized by the user's perspective.

The sawtooth prism glass optical member as shown in (b) of FIG. 5 is provided on the side where the image light of the optical device 200 is incident on the side of the glass in a diagonal direction and is totally reflected inside the glass, and the light image is emitted. The irregularities 302 in the form of sawtooth are projected to the outside of the glass and can be recognized by the user's vision.

The glass optical member having a diffractive optical element (DOE) as shown in (c) of FIG. 5 has a first diffraction portion 303a on the surface of the side where the light image is incident and a side where the light image is emitted. A second diffraction portion 303b may be provided on the surface of . The first and

second diffraction parts

303a and 303b may be provided by patterning a specific pattern on the surface of the glass or attaching a separate diffraction film.

Accordingly, the light image generated by the optical device 200 is diffracted as it enters through the first diffraction unit 303a, is totally reflected, guides light along the longitudinal direction of the glass, and is emitted through the second diffraction unit 303b. , can be recognized by the user's perspective.

The glass optical member having a hologram optical element (HOE) as shown in (d) of FIG. 5 may be provided with an out-coupler (304) inside the glass on the side from which the optical image is emitted. You can. Accordingly, the light image is incident from the optical device 200 in the diagonal direction through the side of the glass, is totally reflected, guides the light along the longitudinal direction of the glass, and is emitted by the out coupler 304, so that it can be recognized by the user's vision. there is. The structure of such holographic optical members can be slightly changed and further divided into a structure with a passive grid and a structure with an active grid.

The glass optical member having a passive grating as shown in (e) of FIG. 5 has an in-coupler (305a) on the surface opposite to the glass surface on which the light image is incident, and the light image is emitted. An out-coupler (305b) may be provided on the surface opposite to the glass surface. Here, the in-coupler 305a and the out-coupler 305b may be provided in the form of a film having a passive grid.

Accordingly, the light image incident on the glass surface on the incident side of the glass is totally reflected by the in-coupler 305a provided on the opposite surface, guides light along the longitudinal direction of the glass, and is guided along the length of the glass by the out-coupler 305b. It projects through the opposite surface and can be recognized by the user's vision.

The glass optical member having an active grating as shown in (f) of FIG. 5 is an in-coupler (306a) formed with an active grating inside the glass on the side where the light image is incident. An out-coupler (306b) formed as an active grid may be provided inside the glass on the side from which light is emitted.

Accordingly, the light image incident on the glass is totally reflected by the in-coupler 306a, guides light along the length direction of the glass, and is emitted out of the glass by the out-coupler 306b, so that it can be recognized by the user's vision. there is.

A pin mirror type optical member may be used as the display unit according to the modified example.

In addition, the surface reflection type optical member of the freeform combiner type as shown in (a) of FIG. 6 has a plurality of flat surfaces with different incident angles of the optical image formed of one glass to perform the role of a combiner. , freeform combiner glass formed to have an overall curved surface can be used. Such freeform combiner glass 300 may emit an optical image to the user at different angles of incidence for each area.

The Flat HOE surface reflection type optical member as shown in (b) of FIG. 6 may be provided by coating or patterning a holographic optical member (HOE, 311) on the surface of flat glass. The light image incident from the device 200 may pass through the holographic optical member 311, be reflected on the surface of the glass, and then pass through the holographic optical member 311 again and be emitted toward the user.

The freeform HOE surface reflection type optical member as shown in (c) of FIG. 6 may be provided by coating or patterning a holographic optical member (HOE, 313) on the surface of freeform glass, and the operating principle is It may be the same as described in (b) of FIG. 6.

*160 FIG. 7 is a conceptual diagram of an electronic device and an optical device according to an embodiment, FIG. 8 is a conceptual diagram of an optical device according to an embodiment, FIG. 9 is a perspective view of an optical device according to an embodiment, and FIG. 10 is a conceptual diagram of an optical device according to an embodiment. It is an exploded perspective view of the optical device, FIG. 11 is a view taken along line AA' in FIG. 9, FIG. 12 is an enlarged view of part K1 in FIG. 11, and FIG. 13 is an enlarged view of part K2 in FIG. 11.

First, referring to FIG. 7 , an electronic device according to an embodiment may include a frame 100, an optical device 200, and a display unit 300 as described above.

And the optical device 200 may be placed on the frame 100. And the optical device 200 is connected to the display unit 300 and can provide an image or video to the display unit 300. This display unit 300 may be located on a light-transmitting lens (eg, eyeglasses) connected to the frame 100.

Furthermore, as will be described later, the optical device 200 includes a barrel 210, a light source device 220 disposed within the barrel and including a light source, an optical signal generator 230 that converts the optical signal into a signal for an image signal, and It may include a detection unit 250. The explanation for this can be applied in the same way as described later.

In particular, in the optical device 200 according to the embodiment, the sensing unit 250 may include a light receiving unit 251. Additionally, the sensing unit 250 may include a barrier unit 252 adjacent to the light receiving unit 251 and a substrate unit 253 connected to the light receiving unit. At this time, in the optical device 200, the barrel 210 may include a through hole 210rs formed on the side.

At this time, the sensing unit 250 may be disposed adjacent to the through hole 210rs. Accordingly, light passing through a plurality of lenses within the barrel 210 may be emitted to the outside through the through hole 210rs. For example, when a user (or wearer) wears an electronic device, light emitted through the through hole 210rs may be emitted onto the wearer's skin. The emitted light (IL) may penetrate the wearer's tissue and move to the blood vessel. At this time, the amount of light absorbed varies depending on the amount of blood (eg, red blood cells) passing through the blood vessel (eg, artery). For example, the amount of blood passing through blood vessels may repeatedly increase or decrease depending on the cycle or frequency of the heartbeat (or pulse). For example, if the amount of blood is large, most of the light (e.g., green wavelength light) transmitted into the skin is absorbed by red blood cells, and the amount of light (RL) reflected to the outside of the skin may be relatively reduced. Accordingly, the light received by the light receiving unit 251 may be relatively reduced. On the other hand, if the amount of blood is small, less light transmitted into the skin is absorbed by red blood cells, and the amount of light reflected outside the skin may relatively increase. Accordingly, the light received at the light receiving unit may relatively increase. In other words, in preparation for cases where the amount of blood is small, the light received by the light receiver may be small. And the light receiving unit 251 may convert the received light into an electrical signal, and the converted electrical signal may be transmitted to the control unit of the optical device or the control unit of the electronic device. Therefore, the wearer's pulse can be detected through the sensing unit 250. Accordingly, the electronic device or optical device can detect whether the user is wearing the device using the sensing unit 250.

The barrier unit 252 is disposed adjacent to the light receiving unit 251 to prevent light passing through the through hole 210rs from being directly provided to the light receiving unit 251. Accordingly, the optical device or electronic device according to the embodiment can perform more accurate detection. For example, optical or electronic devices can accurately sense pulse or oxygen saturation.

And the substrate part 253 can support the barrier part 252 and the light receiving part 251. Furthermore, the substrate portion 253 is electrically connected to the light receiving portion 251 and may be electrically connected to a control portion of an optical device or electronic device.

Referring further to FIGS. 8 to 11 , the optical device 200 according to the embodiment includes a barrel 210, a lens (L), and a first light guide (LG1). Furthermore, the optical device 200 may include a light source device 220 located at or coupled to an opening OP formed on a side of the barrel 210 and an optical signal generator 230 adjacent to the barrel 210. Furthermore, the optical device 200 may further include a cover (CV) surrounding the barrel 210, the light source device 220, and the optical signal generator 230, and a substrate (including a connector, not shown).

There may be a plurality of lenses (L). For example, the lens L may include a plurality of lenses arranged sequentially based on the top of the barrel 210. For example, the lens L may include a first lens L1 disposed first at the top, a second lens L2 disposed behind the first lens, and an N-th lens Ln disposed last at the top. Here, N is a natural number and can be 2 or more. In addition, the N-th lens Ln may be located closest to the optical signal generator 230 located at the rear end of the barrel 210 or at the rear end of the plurality of lenses L among the plurality of lenses L.

Additionally, in an embodiment according to the present invention, the first direction (X-axis direction) may correspond to the optical axis. Additionally, the first direction (X-axis direction) may correspond to the direction in which light emitted from the light source device 220 is reflected by the optical signal generator 230 and is emitted to the display unit described above. And the second direction (Y-axis direction) is a direction perpendicular to the first direction (Y-axis direction). Furthermore, the second direction (Y-axis direction) may correspond to the direction from the first light guide LG1 toward the opening OP. Additionally, in the following specification or this embodiment, the second direction (Y-axis direction) may correspond to the direction from the first light guide LG1 to the second light guide LG2. The first light guide LG1 may be referred to as a first light guide part or a first guide member. Additionally, the second light guide LG2 may be referred to as a second light guide part or a second guide member.

And the barrel 210 may further include a hole corresponding to the opening OP. In an embodiment, the barrel 210 may include a barrel hole or an additional hole 210h. The additional hole 210h may be located facing the opening OP. Alternatively, the additional hole 210h may overlap the opening OP in the second direction (Y-axis direction). Alternatively, the distance of the additional hole 210h from the N-th lens Ln in the first direction may be equal to the distance between the opening OP and the N-th lens Ln. Alternatively, the additional hole 210h may be located in an area corresponding to the position of the opening OP on the inner surface of the barrel 210. With this configuration, the bonding member can be easily applied to the light source device 220 coupled to, inserted into, fixed to, or connected to the opening OP through the additional hole 210h. Accordingly, the coupling force between the barrel 210 and the light source device 220 can be improved. Accordingly, the optical device 200 according to the embodiment may have improved durability, durability, or reliability. Alternatively, optical tests on the light source device 220 located in the opening OP can also be easily performed through the additional hole 210h. Alternatively, fluid (eg, air) can be easily discharged or discharged to the barrel 210 and the light source device 200 through the additional hole 210h. The presence or absence of these additional holes 210h may be applied in various ways depending on the embodiment. For example, the additional hole 210h may be disposed on the side of the barrel 210 as described above. Alternatively, the additional hole 210h may not exist on the side of the barrel 210 in consideration of durability, etc.

And a plurality of lenses (L) may be located within the barrel 210. Additionally, a first light guide LG1 may be located within the barrel 210.

And the barrel 210 according to the embodiment may include an opening OP formed on the side. The shape of the opening OP may include various shapes, such as circular or polygonal. And this opening OP may correspond to the position of the first light guide LG1.

In an embodiment, the opening OP may overlap the first light guide LG1 in a direction perpendicular to the optical axis. With this configuration, light emitted from the light source device 220 located on the side of the barrel 210 can easily enter the first light guide LG1.

And the first light guide LG1 may be disposed between two lenses among the plurality of lenses. For example, the first light guide LG1 may be disposed between the first lens L1 and the N-th lens Ln. Additionally, the first light guide LG1 may be located between the first lens L1 and the optical signal generator 230. Additionally, the first light guide LG1 may be located between the first lens L1 and the second lens L2 or between the second lens L2 and the N-th lens Ln. Various positions of the first light guide LG1 will be described through various embodiments as will be described later.

In this embodiment, the first light guide LG1 may be located between the second lens L2 and the N-th lens Ln. Accordingly, the N-th lens (Ln) may be located between the first light guide (LG1) and the optical signal generator 230. With this configuration, an appropriate optical path can be secured when the light reflected from the first light guide LG1 is provided to the optical signal generator 230. Additionally, refraction of the reflected light may occur. As a result, the first light guide LG1 can be miniaturized.

The first light guide LG1 may include a first prism. The first prism may be a polarizing prism. Additionally, the first prism may be a polarization separation prism.

The first prism may reflect the first polarized light and transmit the second polarized light. For example, a portion of the light (first polarized light) provided from the light source device 220 or incident on the first light guide LG1 may be reflected from the first light guide LG1 and provided to the optical signal generator 230. . Additionally, another part of the light (second polarized light) incident on the first light guide LG1 may pass through the first light guide LG1 and be absorbed in the barrel 210 .

Additionally, light emitted from the light source device 220 may be incident on the first light guide LG1 through the opening OP. To this end, as described above, the opening OP may be disposed in the area where the first light guide LG1 is located. For example, the incident surface of the first light guide LG1 may be positioned to face the opening OP. Alternatively, the position of the first light guide LG1 in the barrel 210 may be the same as the first light guide LG1.

The light source device 220 may include a light source 223 and generate (generate, provide) or emit light. The light source device 220 according to the embodiment may be located or coupled to the opening OP. That is, the light source device 220 may be connected or combined with the barrel 210.

The light source device 220 includes a housing 222 including an opening 222h, a second light guide LG2 disposed within the housing 2220, and a light source 223 that provides light to the second light guide LG2. It can be included.

Furthermore, the light source device 220 includes a light source assembly 221 disposed outside the light source device 220 and surrounding the housing 222, a light source lens 224 adjacent to the light source 223, and an intermediate member located within the housing 222. It may include a lens (MO).

The light source assembly 221 may be disposed on the outermost side of the light source device 220 . When mounting the light source 223 within the housing 222 is difficult or when mounting an additional lens (light source lens) is required, the light source assembly 221 may be located outside the housing 222. The light source assembly 221 may be integrated with the housing 222 or may be separate from the housing 222 .

Housing 222 may include an opening 222h. The housing 222 may be located adjacent to the opening OP of the barrel 210. For example, the opening 222h of the housing 222 may be located corresponding to the opening OP of the barrel 210. Accordingly, the opening 222h of the housing 222 may overlap the opening OP of the barrel 210 in the second direction (Y-axis direction).

Light source 223 may be located within housing 222 or light source assembly 221. The light source 223 may emit light. For example, light emitted from the light source 223 may be incident on the second light guide LG2 within the housing 222. A second light guide LG2 may be located within the housing 222. Accordingly, the second light guide LG2 may transmit the light emitted from the light source 223 to the opening 222h or the first light guide LG1.

And there may be more than one light source 223. That is, in the light source device 220, there may be a single light source 223 or a plurality of light sources 223. For example, the light source 223 may be plural and include a first light source 223a, a second light source 223b, and a third light source 223c. The first light source 223a to the third light source 223c may emit light in the same direction or in different directions. For example, the first light source 223a and the third light source 223c may be positioned to face each other. The first light source 223a and the third light source 223c may be positioned to overlap in the first direction (X-axis direction). And a second light guide LG2 may be located between the first light source 223a and the third light source 223c. Accordingly, the second light guide LG2 may overlap the first light source 223a and the third light source 223c. And the second light source 223b may be located between the first light source 223a and the third light source 223c. The first light source 223a to the third light source 223c may emit light toward the second light guide LG2. And the second light source 223b may overlap the second light guide LG2 in the second direction. With this configuration, the optical device 200 can have a compact light source device 220.

Additionally, the first light source 223a, the second light source 223b, and the third light source 223c may each emit light of the same or different wavelengths or colors. For example, the first light source 223a, the second light source 223b, and the third light source 223c may each emit red, green, and blue light.

The second light guide LG2 may include a second prism. The second prism is a reflective member and may include, for example, an X prism (x-prism). In an embodiment, the second light guide LG2 or the second prism may have a structure in which at least two or more prisms are combined.

And the second prism may include at least two or more coated surfaces (reflecting members or reflective sheets). One of these at least two coating surfaces may reflect light of the first wavelength and light of the second wavelength, and may transmit light of the third wavelength. That is, the coated surface can reflect light in a predetermined wavelength band. Accordingly, for each light emitted from the plurality of light sources 223, lights in a desired wavelength band may be reflected from the second light guide LG2. For example, light passing through the second light guide (LG2) may be provided to the first light guide (LG1) or the intermediate lens (MO).

The light source lens 224 may be located adjacent to the light source 223. For example, there may be a plurality of light source lenses 224. The light source lens 224 may be located on the path of light emitted from each light source 223.

In an embodiment, the light source lens 224 may be located between the second light guide LG2 and the light source 223. Additionally, there may be a plurality of light source lenses 224. A plurality of light source lenses 224 may be positioned between the second light guide LG2 and the light source 223. Alternatively, there may be a plurality of light source lenses 224 corresponding to each of the plurality of light sources 223 .

For example, the light source lens 224 may include a first light source lens 224a, a second light source lens 224b, and a third light source lens 224c. The first light source lens 224a may be located between the first light source 223a and the second light guide LG2. The second light source lens 224b may be located between the second light source 223b and the second light guide LG2. The third light source lens 224c may be located between the third light source 223c and the second light guide LG2.

In addition, the first light source lens 224a, the second light source lens 224b, and the third light source lens 224c are each on the first light source 223a, the second light source 223b, and the third light source 223c. There may be multiple locations. For example, one of the plurality of first light source lenses 224a may be located on the first light source 223a and combined with the light source assembly 221. Another one of the plurality of first light source lenses 224a is located between one of the first light source lenses 224a and the second light guide LG2, and can be combined with the housing 222.

And the first light source lens 224a, the second light source lens 224b, and the third light source lens 224c may include a collimator lens or a collimator.

Additionally, a substrate 225 connected to the light source 223 may be disposed in the light source assembly 221. There may be at least one substrate 225 corresponding to the light source 223 . For example, the plurality of substrates 225 may include a first substrate 225a, a second substrate 225b, and a third substrate 225c. As described above, the plurality of substrates 225 may be one substrate or may be plural in number corresponding to the number of light sources.

Additionally, the substrate 225 may be electrically connected to a control unit or processor within the frame (or display unit) described above. Accordingly, the board 225 may include a connector for communication with an external device or a connected device. Furthermore, the substrate 225 is disposed outside the light source 223 to discharge heat generated from the light source to the outside. Accordingly, the reliability of the light source device 220 can be improved.

The intermediate lens (MO) may be located between the aperture (222h) and the second light guide (LG2). Additionally, the intermediate lens (MO) may be located between the first light guide (LG1) and the second light guide (LG2).

The intermediate lens (MO) may include a plurality of lenses. For example, it may include a first intermediate lens (MO1) and a second intermediate lens (MO2). However, as will be described later, the light reflected from the second light guide LG2 may be directly provided to the first light guide LG1 without the intermediate lens MO.

The first intermediate lens MO1 may include a first surface MO1s1 adjacent to the first light guide LG1 and a second surface MO1s2 corresponding to the first surface MO1s1. Light emitted from the light source 223 may pass through the second light guide LG2 and sequentially pass through the second surface MO1s2 and the second surface MO1s1.

The second surface MO1s2 may be convex toward the second light guide LG2. Alternatively, the second surface MO1s2 may be concave toward the first light guide LG1. Furthermore, the first surface MO1s1 may be convex or concave toward the second light guide Lg2. For example, the first intermediate lens MO1 may have a meniscus shape. In this way, the second surface MO1s2 is convex toward the second light guide LG2, so that light can be concentrated while passing through the intermediate lens MO. Accordingly, the uniformity or uniformity of light provided to the optical signal generator 230 may be improved. In other words, the light incident on the optical signal generator 230 may be surface light. And the uniformity of surface light can be improved. Accordingly, the accuracy or resolution of the image signal or image emitted to the display unit through the optical device 200 according to the embodiment may be improved.

The second intermediate lens (MO2) may include a micro lens array (MLA). Accordingly, surface illumination may be partially performed on the light passing through the second light guide LG2.

Furthermore, the second intermediate lens MO2 may have a different size from the second light guide LG2. Additionally, the first intermediate lens MO1 may have a different size from the second light guide LG2. For example, the intermediate lens (MO2) may be larger in size compared to the second light guide (LG2).

Furthermore, the positions of the plurality of lenses L, the intermediate lens MO, and the light source lens 224 may be maintained by the spacer SP. For example, a plurality of spacers (SP) adjacent to a plurality of lenses (L) may exist in the barrel 210. Additionally, an intermediate lens (MO) and a plurality of spacers (SP) adjacent to the light source lens 224 may be disposed in the housing 222 or the light source assembly 221 of the light source device 220. In an embodiment, a plurality of spacers SP may be disposed on the upper or lower side of the above-described lens to fix or maintain the position of the lens.

The optical signal generator 230 may be located at the rear end of the barrel 210. The optical signal generator 230 may overlap the lens L in the optical axis or the first direction (X-axis direction).

The optical signal generator 230 may convert the light incident on the first light guide LG1, reflected, and then passing through the N-th lens into an optical signal containing image information.

The optical signal generator 230 may reflect light (first polarized light) reflected from the first light guide LG1. The optical signal generator 230 may generate an optical signal including image information. That is, the light reflected from the optical signal generator 230 may be light containing image information.

The optical signal generator 230 may include a liquid crystal on silicon (LCoS) device.

A silicon liquid crystal display device consists of a silicon wafer (a thin disk used as a semiconductor material) with a CMOS (Complementary Metal-Oxide Semiconductor) array and an anti-reflection coating coated with a transparent electrode (Indium Tin Oxide, ITO). It may be a structure where liquid crystal is placed between members (anti-reflection, AR).

And, an alignment layer is formed on the wafer (silicon wafer) to form initial liquid crystal alignment.

Furthermore, a reflective layer or reflective electrode formed of an aluminum layer and having a high optical reflectance may be located below the alignment layer. The reflective electrode may be located on a silicon wafer. And a semiconductor array (CMOS array) can be formed on the silicon wafer. And with this semiconductor array, the transmission of data signals through the panel is achieved.

The optical signal generator 230 may reflect at least a portion of the incident light as it is driven. Additionally, the optical signal generator 230 may reflect light incident on the surface light source for each pixel. Additionally, the intensity of reflected light can also be adjusted depending on the degree of modulation. For example, the optical signal generator 230 may partially modulate the first polarized light into the second polarized light. Accordingly, the light modulated by the second polarization may pass through the first light guide LG1 and the first lens L1 and be provided to the display unit.

That is, the optical signal generator 230 can modulate the retardation of modulated light, that is, polarized light. The optical signal generator 230 may delay the first polarization in various ways. In other words, an electric field can be formed by adjusting the voltage for each pixel (adjusting the voltage of the electrode). And the degree of twist in the liquid crystal can be adjusted according to the adjusted voltage. For example, at the maximum voltage, all of the light reflected from the optical signal generator 230 may be reflected from the first light guide LG1. And at the minimum voltage, all of the light reflected from the optical signal generator 230 may be transmitted through the second light guide LG1. However, it may operate in the opposite direction depending on the electric field. Additionally, when a moderate voltage is applied, some light may pass through the first light guide LG1. That is, the intensity (eg, brightness) of light provided to the display unit may be medium.

In this way, the optical signal generated by the optical signal generator 230 may be transmitted to the first lens L1 through the N-th lens Ln and the first light guide LG1. Furthermore, at least a portion of the optical signal generated by the optical signal generator 230 may pass through the first lens L1 and enter the display unit.

Additionally, a transparent member 240 may be further disposed between the optical signal generator 230 and the N-th lens Ln (or first light guide). The transparent member 240 may be glass. The transparent member 240 may be combined with the barrel 210 or the cover (CV). Furthermore, the transparent member 240 may be located on the optical signal generator 230. Accordingly, the inflow of foreign substances into the optical signal generator 230 can be easily blocked. And the transparent member 240 may have the same or different size from the optical signal generator 230. Furthermore, the transparent member 240 may overlap at least a portion of the optical signal generator 230 in the optical axis direction or the first direction (X-axis direction).

Additionally, a barrier portion 252 may be located between the through hole 210rs of the barrel 210 and the light receiving portion 251. This light receiving unit 251 may be located in the area DA located between the first light guide LG1 and the optical signal generating unit 230. For example, the area DA may be an area between the through hole 210rs and the optical signal generator 230 in the first direction (uniaxial direction). Additionally, the through hole 210rs may also be located within the area DA. With this configuration, a portion of the light emitted before the optical signal is incident on the area 230 can be more efficiently emitted to the outside (eg, the wearer's skin) through the through hole 210rs. That is, the amount of light passing through the through hole 210rs increases, and the sensing accuracy through the sensing unit 250 can be improved.

And the light receiving unit 251 may be located within the barrel 210. For example, the light receiving unit 251 may be located in a groove formed on the outer surface of the barrel 210. Additionally, the outer surface of the light receiving unit 251 may have a level difference from the outer surface of the barrel 210. For example, the light receiving unit 251 may protrude outward from the outer surface of the barrel 210. Accordingly, the light receiving unit 251 can be placed closer to the user. Accordingly, sensing accuracy through the light receiving unit 251 can be further improved.

Likewise, the barrier portion 252 may extend further outward than the outer surface of the barrel 210. As a result, the barrier unit 252 can prevent the light emitted through the through hole 210rs from being directly emitted to the light receiving unit 251. In other words, noise can be reduced.

Furthermore, in the optical device according to the embodiment, the first reference lens (L _n ) and the second reference lens (L _n-1 ), which are two lenses sequentially adjacent to the optical signal generator 230, are located in the plurality of lenses. You can. Although not shown, the second reference lens may be located between the first light guide LG1 and the first reference lens L _n . At this time, the size (d2) of the first reference lens (L _n ) may be larger than the size (d1) of the second reference lens (L _n-1 ). At this time, the through hole 210rs may be located between the first reference lens (L _n ) and the second reference lens (L _n-1 ). Accordingly, divergent light can be more easily emitted from the first reference lens toward the second reference lens through the through hole 210rs. Accordingly, the accuracy of the sensing unit can be improved. Likewise, the light receiving unit 251 may also be located between the first reference lens (Ln) and the second reference lens (Ln-1).

Looking further at FIG. 12, the light (La, Lb, Lc) emitted from each light source 223 of the light source device 220 passes through the light source lens 224, the second light guide LG2, and the intermediate lens (MO). can do. For example, the first light (La), the second light (Lb), and the third light (Lc) emitted from each of the first light source 223a to the third light source 223b are directed in the same direction in the second light guide LG2. can be released as

For example, the second light guide LG2 may include a first coating surface (LG2a) and a second coating surface (LG2b). As described above, one of these at least two coating surfaces may reflect some of the first wavelength light, the second wavelength light, and the third wavelength light. For example, the first wavelength includes a wavelength band of red light. The second wavelength includes a wavelength band of green light. The third wavelength includes a wavelength band of blue light.

Additionally, the first coating surface LG2a may reflect the first light La or light of the first wavelength. That is, the first coating surface LG2a can transmit the second light Lb and the third light Lc. In other words, the first coating surface LG2a can transmit light of the second wavelength and light of the third wavelength.

The second coating surface LG2b may reflect the second light Lb or light of the second wavelength. That is, the second coating surface LG2b can transmit the first light La and the third light Lc. In other words, the second coating surface LG2b can transmit light of the first wavelength and light of the third wavelength.

Accordingly, the first light La may be reflected or incident on the intermediate lens MO or the opening OP in the second light guide LG2. And in the second light guide LG2, the second light Lb may be reflected or incident on the intermediate lens MO or the opening OP. Additionally, the third light Lc may be reflected or incident on the intermediate lens MO or the opening OP from the second light guide LG2.

Accordingly, the light (IL or (La, Lb, Lc)) emitted from the light source 223 may be incident on the first light guide LG. At this time, the light incident on the first light guide LG may be the first incident light IL.

Looking further at FIG. 13 , the first incident light IL may be partially reflected and partially transmitted by the first light guide LG1. That is, the first light guide LG1 may reflect the first polarized light ILa of the first incident light IL and transmit the second polarized light ILb. For example, the first polarization (ILa) and the second polarization (ILb) may each be different types of S/P light. Accordingly, the first polarized light ILa, which is part of the first incident light IL, may be provided to the optical signal generator 230 through the N-th lens Ln. And the second polarized light (ILb) may be absorbed by the barrel 210 or provided through the additional hole 210h.

14 is another example of a light source device according to an embodiment.

Referring to FIG. 14, in the light source device 220, there may be at least one light source 223 as described above. That is, the light source 223 may include a first light source 223a, a second light source 223b, and a third light source 223c. Furthermore, in another example, a plurality of mirrors (M1 to M3) are present instead of the second light guide. That is, the optical device includes a plurality of mirrors (M1 to M3), and can provide light to a plurality of lenses or a first light guide in the barrel using each of the plurality of mirrors (M1 to M3). Other light source devices may have various structures.

FIG. 15 is a flowchart of a method of operating an electronic device or optical device according to an embodiment, and FIG. 16 is a timing diagram explaining the operations of a light source, an optical signal generator, and a light receiver.

First, the light receiving unit 251 can be driven while light is emitted from the light source. However, as will be described later, in detecting whether or not the device is worn, the light receiving unit 251 may be driven even during a time when light is not emitted from the light source. Additionally, the operation method described later may be executed by a control unit (or controller) of an optical device or a control unit (or controller) of an electronic device. That is, processing in each step of the operation method described later can be performed in the control unit except for the content described later. Additionally, the operating method according to the disclosed embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Additionally, an embodiment of the present disclosure may be a computer-readable recording medium on which one or more programs including instructions for executing an operation method are recorded.

Referring to FIGS. 15 and 16, the operating method according to the embodiment includes turning the optical device off (S1010), operating the light receiving unit (S1020), and comparing the first light reception signal with the first set value. Step (S1030), turning on the light source device (S1050), operating the light receiving unit (S1060), comparing the second light receiving signal with the second set value (S1070), and driving the sensing unit. Includes (S1080).

At this time, the control unit may transmit a high signal to the light receiver for operation (eg, on) of the light receiver. That is, the control unit can control the operation of the light receiving unit. Additionally, the control unit can control on (high) or off (low) the light source. Additionally, the controller may control the on/off of the optical signal generator according to the on or off of the light source. In other words, the control unit can control the light source device, the optical signal generating unit, and the light receiving unit.

First, the control unit can keep the light source in an off state when the electronic device is turned on. That is, current may not be applied to the light source. For example, the controller may operate (high) the light receiver when the optical device is turned off as described above in the first operation period T1 (S1020). And the first light receiving signal can be received from the light receiving unit in the first operation period T1. In other words, when there is no current applied to the light source device (low), the control unit can operate the light receiving unit and receive the first light receiving signal from the light receiving unit. And the control unit may compare the first light-receiving signal and the first set value in the first operation period T1 (S1030).

The control unit may determine whether to drive the light source autonomy by comparing the first light reception signal and the first set value. At this time, if the first light reception signal is greater than the first set value, the control unit may determine that the user is not wearing the device, in which ambient light is introduced. Accordingly, the control unit can output a non-wearing alarm, etc. or return to the above-described step (S1010). Furthermore, it may start again from step S1020.

If the first light-receiving signal is smaller than the first set value in the first operation section T1, the operation may proceed to the second operation section T2.

The control unit may transmit a high signal to the light source device in the second operation period T2. In an embodiment, the control unit may transmit a high signal to the light source of the light source device (eg, a second light source emitting green wavelength light). For example, the controller may transmit a high signal to at least one of the light sources of the light source device. As a result, the first to third light sources can emit light that is different from or does not overlap with each other. Accordingly, energy efficiency can be improved.

For example, the first light source may be turned on during the first light source time period T2aa. In other words, the controller may transmit a high signal to the first light source during the first light source time period T2aa. Additionally, the optical signal generator may be turned on by receiving a high signal during the first generator time period T2ba. Furthermore, the control unit may be turned on by receiving a high signal during the first light reception time period T2ca. For example, the light receiving unit may receive red light, and the control unit may detect the user's oxygen saturation using the light receiving signal (second light receiving signal) received from the light receiving unit. In other words, the control unit can drive the light source and receive the second light reception signal from the light reception unit when the first light reception signal is smaller than the first set value.

Additionally, the second light source may be turned on during the second light source time period T2ab. In other words, the controller may transmit a high signal to the second light source during the second light source time period T2ab. Additionally, the optical signal generator may be turned on by receiving a high signal during the second generator time period T2bb. Furthermore, the control unit may be turned on by receiving a high signal during the second light reception time period T2cb. For example, the light receiving unit may receive green light, and the control unit may detect the user's pulse using the light receiving signal (second light receiving signal) received from the light receiving unit.

Additionally, the third light source may be turned on during the third light source time period T2ac. In other words, the controller may transmit a high signal to the third light source during the third light source time period (T2ac). Additionally, the optical signal generator may be turned on by receiving a high signal during the third generator time period T2bc. Furthermore, the control unit may be turned off by receiving a low signal during the third light reception time period T2cc.

For example, the light source time period and the generator time period may be synchronized.

In this way, in the second operation period T2, the control unit can detect or determine whether the user is wearing the device using the light reception signal (second light reception signal) from the light receiver. Additionally, only when a specific light source (at least one light source) is turned on (high), the control unit can drive the light receiving unit to receive a light receiving signal (second light receiving signal).

Additionally, in this embodiment, the optical signal generator may be turned on (high) in response to a light source time period when the light source is on (high). Accordingly, operation accuracy can be improved.

Afterwards, the control unit can receive the second light receiving signal output when the light receiving unit is turned on (S1060). And the control unit can compare the second light-receiving signal and the second set value (S1070). At this time, if the second light-receiving signal is smaller than the second set value, the control unit may determine that the user or wearer is not wearing the device correctly. For example, when at least a portion of the electronic device is not in exact contact with the skin, the second light-receiving signal may be smaller than the second set value. Accordingly, the correct wearing of the user can be determined. In this case, it is determined that the user is not wearing it, especially if it is incorrectly worn, and a standby state can be performed (S1090). For example, it may return to step S1010 or S1050).

In contrast, when the second light reception signal is greater than the second set value, the operation may proceed to the third operation period T3. Accordingly, the control unit can transmit a driving signal (high signal) to the light source and the optical signal generator at a predetermined period.

And the control unit can turn on (high) the light receiving unit in response to turning on (high) the light source and the optical signal generator (S1080). In other words, the control unit may compare the second light reception signal with the second set value and receive a third light reception signal from the light reception unit in response to the operations of the light source and the optical signal generator. That is, the control unit can drive the detection unit (S1080). For example, in order to detect the wearer's pulse in the third operation section T3, the control unit turns on the light source and the optical signal generator at a predetermined period (e.g., the second light source time period, the second generator time section) and turns on the light receiver. It can be turned on (high).

Alternatively, the control unit may detect the oxygen saturation of the wearer in the third operation period T3 by turning on the light source and the optical signal generator at a predetermined period (e.g., the first light source time period, the first generator time period) in response to the light receiving unit. can be turned on (high).

And in the operation method and electronic device according to the various embodiments described above, selection of the inspection image may be performed for the user's right eye and left eye, respectively. Additionally, the selection of the inspection image may be performed for both eyes of the user, or may be performed sequentially for the right eye, left eye, and both eyes.

First, the display method described in FIGS. 17 to 24 can be performed by a control unit in the electronic device described above. That is, processing in each step of the display method described later can be performed in the control unit except for the content described later. Additionally, the display method according to the disclosed embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Additionally, an embodiment of the present disclosure may be a computer-readable recording medium on which one or more programs including instructions for executing a display method are recorded.

FIG. 17 is a flowchart of a display method according to an embodiment, FIG. 18 is an additional flowchart of a display method according to an embodiment, FIGS. 19 and 20 are diagrams explaining visual acuity determination in a display method according to an embodiment, FIG. 21 is a diagram illustrating image output in a display method according to an embodiment.

Referring to Figures 17 and 18, the display method according to the embodiment includes outputting test images with different focal lengths (S1040) and determining the user's visual acuity based on an image selected from test images with different focal lengths (S1040). S1060) and outputting a virtual image based on the determined visual acuity of the user (S1080).

Furthermore, before the step of outputting inspection images with different focal lengths of the display method according to the embodiment (S1040), the step of detecting the user (or the wearer of the electronic device) (S1010) and checking whether an input (button, etc.) is detected Step (S1020) may further include, when detecting input, checking whether the user is a pre-stored user (S1030).

First, looking at FIG. 2, when the user or wearer wears the display device, the control unit 27 (see FIG. 2) determines whether the user is wearing it through a detection signal received through the sensing unit 23 (see FIG. 2). can be detected or judged. The control unit 27 can determine whether an input is detected when the user wears the device based on the signal received from the sensing unit 23 (S1020). For example, a user may generate an input signal through the user input unit 130 (see FIG. 3) or the input unit 22 (see FIG. 2). As the user touches the input unit, it can be recognized that the user is using an electronic device. If the user's input is not detected within a predetermined time, the control unit may re-perform the step of detecting the user's wearing.

Furthermore, when an input is detected through the input unit, the control unit can check whether the user is a pre-stored user (S1030). In the case of a previously stored user, the user's vision information stored in the memory 26 (FIG. 2) can be retrieved. Alternatively, according to the user's input, the control unit may re-perform visual acuity determination and storage of visual acuity information even for users who have previously stored the visual acuity information. Accordingly, changes in the user's body can be easily reflected.

And, if the control unit is not a pre-stored user or the user re-performs the inspection, the control unit may output inspection images with different focal lengths as described above (S1040). For example, the control unit may generate or provide image signals of inspection images with different focal lengths. Additionally, the optical device generates an image based on the image signal of the inspection image generated by the control unit, and the image generated by the optical device may be output on the display unit.

Looking further at FIG. 19, the inspection images TI1, TI2, and TI3 may include images with different focal lengths. Additionally, there may be multiple inspection images (TI1, TI2, and TI3). In an embodiment, the inspection image may include a first inspection image (TI1), a second inspection image (TI2), and a third inspection image (TI3). At this time, the focal distances of the first inspection image TI1, the second inspection image TI2, and the third inspection image TI3 from the user's eye may sequentially increase. For example, the focal length f1 of the first inspection image TI1 may be greater than the focal length f2 of the second inspection image TI2. Additionally, the focal length f2 of the second inspection image TI2 may be greater than the focal length f3 of the third inspection image TI3.

Looking further at FIG. 20 , the first inspection image TI1 , the second inspection image TI2 , and the third inspection image TI3 are described as being the same as the inspection image of FIG. 19 . Furthermore, the first inspection image TI1, the second inspection image TI2, and the third inspection image TI3 may be output according to the user's selection (or input) (S1051). And one of the plurality of inspection images may be output by the control unit. For example, the control unit may output the first inspection image TI1. Afterwards, the user can input an input signal by touching the input unit 130 (S1051). For example, the user may generate an input signal by touching any one of the first button BT1, the second button BT2, and the third button BT3 of the input unit arranged on the frame. At this time, the first button BT1 may respond to an increase in the focal distance for the inspection image. The second button BT2 may correspond to a decrease in the focal distance for the inspection image. The third button BT may correspond to selection of an inspection image. Furthermore, there may be at least one inspection image with different focal lengths, such as the first inspection image TI1, the second inspection image TI2, and the third inspection image TI3. Furthermore, when there are a plurality of inspection images with different focal lengths, each of the plurality of inspection images may have a different focal length.

In an embodiment, when the control unit receives the user's input (S1051), the control unit may adjust the focal length of the inspection image in response to the user's input, that is, the applied input (S1052). And the user's vision can be determined in response to the user's selection input (S1053, S1060)

For example, when the first inspection image T1 is recognized as blurry, the user may touch the first button BT1 or the second button BT2 to change the focal distance for the inspection image. For example, when the user touches the second button BT2, the controller may output a second inspection image TI2 with a smaller focal length than the first inspection image TI1. Additionally, if the second inspection image TI2 is recognized as blurry, the user may touch the second button BT2 again to reduce the focus on the inspection image. Accordingly, the control unit may output the third inspection image TI3 with a smaller focal length than the second inspection image TI2. Thereafter, when the third test image TI3 is clearly recognized, the user can touch the third button BT3 to select the third test image TI3 as an image for determining visual acuity. In this way, the control unit can adjust the focal distance of the inspection image in response to the input by the user (S1050).

At this time, the image or inspection image selected by the user's input may be at least one of a plurality of inspection images. For example, depending on the user's vision, there may be multiple inspection images that are clearly visible to the user. In this case, when there are multiple selected images, the controller may determine the user's visual acuity based on the maximum focal length. Accordingly, the control unit can determine the user's accurate vision and output a virtual image corresponding to the user's vision. Accordingly, the electronic device according to the embodiment can provide the effect of eliminating user inconvenience.

And the control unit can determine the user's vision based on the image selected by the input unit (S1060). After determining the user's visual acuity, the control unit can adjust and output the focal distance for the image based on the user's determined visual acuity (S1080). Accordingly, the user can clearly perceive the virtual image output from the optical unit.

Looking further at FIG. 21, the control unit may transmit a control signal to the optical device to output a plurality of virtual images (VI, VI2 to Vin, VI). Accordingly, the optical device can generate images corresponding to a plurality of virtual images.

As shown, a plurality of virtual images may be composed of different layers depending on the distance. And virtual objects may exist in virtual images of different layers. This virtual image is delivered to the user by the optical device 200, and the control unit can adjust the focal length of the virtual image based on the user's visual acuity determined as described above. For example, for myopic users, a plurality of virtual images can be output by reducing the focal distance.

Furthermore, optical elements OD1 and OD2 may be disposed on the front and rear sides of the display unit 300. Here, the front side refers to the area of the display unit 300 facing an object or real image, and the rear side refers to the area of the display unit 300 facing the user. Alternatively, the display unit 300 may further include optical elements OD1 and OD2. These optical elements OD1 and OD2 may be located on the frame of the electronic device. Additionally, the optical elements OD1 and OD2 may include a variable focus lens in addition to the diffraction unit described above. Accordingly, the curvature of the optical element can be changed in response to the user's determined visual acuity. That is, the focal distance can be changed.

For example, the first optical element OD1 is disposed outside the display unit 300 and may include a diffraction unit or a polarization unit. Accordingly, the real image RI may be provided to the user at a predetermined period of frames or at a predetermined frame. And, as described above, the focus or curvature of the first optical element OD1 may be changed in response to the user's determined visual acuity. Accordingly, the user can clearly perceive real images, etc. without blurring. In other words, the user's eyes may be in a normal state for real images, etc.

Furthermore, the focus or curvature of the second optical element OD2 may also change depending on the user's vision.

As a modified example, there are a plurality of lenses in the optical device 200, and the distance between the plurality of lenses of the optical device may be changed by an actuator or the like in response to the user's vision. Furthermore, the control unit can adjust the focal length of the inspection image by moving the lens in the optical device.

And after correcting the user's vision (S1060), the control unit may store the vision information in memory (S1070). In addition to this vision information, the user's specific detection information through the sensing unit may also be stored in the memory.

FIG. 22 is a flowchart of a display method according to another embodiment, and FIG. 23 is a diagram explaining visual acuity determination in a display method according to another embodiment.

Referring to FIGS. 22 and 23, a display method according to another embodiment includes outputting an inspection image including a plurality of objects (S2010), selecting at least one of the plurality of objects (S2020), and displaying an image based on the selected object (S2020). It may include determining the user's vision (S2030) and storing vision information (S2040).

The step of outputting an inspection image including a plurality of objects (S2010) may correspond to the step of outputting the inspection image described above (S1040). Furthermore, steps prior to the step of outputting the inspection image are also applied to the display method according to this other embodiment, and the content of each step described above can be applied equally, except for the content described later.

According to this embodiment, the inspection image may include a plurality of objects. At this time, the plurality of objects may have different focal distances. That is, inspection images with different focal lengths may include a plurality of objects with different focal lengths in one image.

As shown, the inspection image TI4 may include a plurality of objects OJ1 to OJ3. For example, the plurality of objects may include a first object (OJ1), a second object (OJ2), and a third object (OJ3). At this time, the distance or focal length of the first object OJ1 may be greater than that of the second object OJ2. And the second object OJ2 may have a greater distance or focal length than the third object OJ3.

For example, the selection window (SD) for an object in the inspection image, which is a virtual image, may move in response to the user's input (via the first button, second button, or third button). For example, the first button may correspond to Left. And the second button may correspond to the right side. And it can correspond to the third button (select). According to the user's input, one of a plurality of objects in the inspection image may be selected. Alternatively, any one of inspection images composed of a plurality of layers and having different focal lengths may be selected. Through this, the user's vision can be determined. Afterwards, the user's visual acuity determined as described above may be stored. Alternatively, the focal length of the virtual image can be adjusted and output based on the determined visual acuity. That is, when outputting a virtual image, the control unit may vary or change the focus of the virtual image based on the user's determined visual acuity.

FIG. 24 is a flowchart of a display method according to another embodiment, and FIG. 25 is a perspective view of an augmented reality electronic device according to a modified example.

Referring to FIG. 24, a display method according to another embodiment includes outputting a plurality of inspection images (S3010), selecting at least one image (S3020), determining visual acuity (S3030), and visual acuity information. It may include a step of storing (S3040).

When outputting a plurality of inspection images (S3010), the various examples described above can be applied. Furthermore, selection of at least one image may be performed by the control unit receiving the user's input signal (S3020). And visual acuity can be determined based on the selected test image (S3030). At this time, the step of determining the visual acuity may include selecting at least one image from a plurality of test images with different focal lengths, and determining the visual acuity of the user based on the maximum focal length when there are multiple selected images. You can.

For example, based on the input, the control unit may output a plurality of images selected among the inspection images. Additionally, the control unit may determine the user's visual acuity based on the maximum focal length for a plurality of selected images. In addition, the control unit can vary the focus of the virtual image and output it based on the user's visual acuity, which is determined based on the maximum focal distance.

And referring to FIG. 25 , the electronic device may further include a light receiving element PD disposed on the frame 100. Accordingly, in determining the above-described visual acuity, inspection light (image, etc.) is output to the user's eye through the optical device 200 and the display unit 300 (TL), and light (image) reflected from the human eye is output. may be transmitted to the light receiving element (PD). At this time, the control unit may determine the refractive index or visual acuity of the user's eyeball through the reflected light RL provided by the light receiving element PD. Based on this, the electronic device can output a virtual image by varying the focal distance based on the user's vision as described above.

The control method described in FIGS. 26 to 34 may be performed by a controller or control unit in the control device or electronic device described above. That is, processing in each step of the control method described later can be performed in the controller except for the content described later. Additionally, the control method according to the disclosed embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Additionally, an embodiment of the present disclosure may be a computer-readable recording medium on which one or more programs including instructions for executing a control method are recorded.

FIG. 26 is a block diagram of a control device according to an embodiment of the present invention, FIG. 27 is a conceptual diagram of a control device according to an embodiment of the present invention, FIG. 28 is a flowchart of a control method according to an embodiment of the present invention, and FIG. 29 is a detailed flowchart of a control method according to an embodiment of the present invention, FIG. 30 is a diagram explaining the operation and control method of a control device according to an embodiment of the present invention, and FIG. 31 is a control diagram according to an embodiment of the present invention. It is a diagram explaining the operation and control between a device and an electronic device. Figure 32 is a diagram showing an interface of an electronic device according to an embodiment of the present invention, and Figure 33 is a diagram showing an electronic device using a control device according to an embodiment of the present invention. This is an example diagram explaining the operation of , and Figure 34 is a diagram explaining the operation of the control device according to an embodiment of the present invention.

26 and 27, the control device 500 according to the embodiment includes a touch input unit or a first input unit 510, a button input unit or a second input unit 520, a detection unit 530, a communication unit 540, It may include a battery unit or power unit 550, a memory unit or storage unit 560, and a controller 570.

The touch input unit 510 can detect the user's touch. For example, the touch input unit 510 may include a pressure sensor, etc. Additionally, the touch input unit 510 may output a signal about the detected user's touch to the controller 570, etc. In an embodiment, the touch input unit 510 may detect a user's touch to set the position of the control device 500 and detect various touch operations.

The control device 500 according to the embodiment can be set up, down, left, and right in a manner described later. That is, the control device 500 may not be structurally set up, down, left, and right. For example, the control device 500 may have a circular shape, and may have a symmetrical structure with respect to one virtual line. The touch input unit 510 may be located outside the button input unit 520, which will be described later. For example, the touch input unit 510 may surround the button input unit 520 located in the center. And the touch input unit 510 may be formed as a touch screen. With this configuration, the control device according to the embodiment is a control device (eg, remote control) for an electronic device, and can easily control the operation or control of the electronic device with a simple sense (eg, tactile sense). In other words, the user's perspective is no longer essential for controlling the control device, and electronic devices can be easily controlled in various environments and spaces.

For example, the touch input unit 510 may receive various inputs (swiping input, tap), etc. from the user and output signals corresponding to them.

The button input unit 520 can detect button input by a user, etc. For example, the button input unit 520 may have various button structures and may output a signal about the user's pressure to the controller 570, etc. In an embodiment, an image output from an electronic device or information within an image may be selected through the button input unit 520.

The button input unit 520 may be located in the center of the control device 500. Accordingly, the user can easily find the button input unit 520. Furthermore, the button input unit 520 may protrude outward or have a groove shape. Accordingly, the user can easily control the electronic device using his or her fingers (eg, thumb). For example, whether or not power is input to an electronic device or control device can be controlled through the button input unit 520. Alternatively, whether or not to interoperate with an electronic device may be controlled through the button input unit 520, or whether or not the input standby mode of the electronic device is activated may be controlled.

The detection unit 530 may be made of various motion detection sensors. The detection unit 530 may output a detection signal including movement information. For example, the detection unit 530 may include an IMU sensor. For example, the sensing unit 530 may include a gyro sensor, an acceleration sensor, an angular velocity sensor, and a geomagnetic field sensor. And the exercise information may include speed or acceleration information. The sensing unit 530 may exist within the control device or may correspond to the sensing unit of the electronic device described above. Accordingly, the control unit of the control device may receive a detection signal from the sensing unit of the electronic device.

In an embodiment, the detection unit 530 may detect the direction of gravity and the degree of inclination of the control device 500. Additionally, the posture information from the sensor 530 can be used together with the swiping information (input) by the touch input unit 510. Accordingly, the user can easily control the electronic device regardless of the direction of the control device 500.

For example, by tilting the control device 500 to the left or right, a command corresponding to a direction key may be transmitted to the electronic device.

The communication unit 540 can transmit data with an electronic device. Alternatively, the communication unit 540 may include one or more modules that enable wireless communication between external servers.

The battery unit 550 may supply power to the control device. And the memory unit 560 serves as a storage unit and stores data supporting various functions. The memory unit 560 may include read-only memory (ROM), random access memory (RAM), flash memory, a memory card, a storage medium, and/or other storage devices.

The controller 570 may perform processing for a control method described later. This controller 570 may be expressed as a processor, control unit, etc. Additionally, the controller 570 may include an application-specific integrated circuit (ASIC), other chipset, logic circuit, and/or data processing device.

And in the embodiment, the controller 570 is described based on performing a control method described later.

Referring further to FIGS. 28 and 29, the control method according to the embodiment includes receiving a user input (S1010), activating a standby mode (S1020), receiving a detection signal (S1030), and determining the user status. It includes a step of determining (S1040), a step of determining activation of a gesture action or a touch action in response to the determined user state (S1050), and a step of transmitting a command in response to the activated action (S1060).

The controller 570 may receive user input through a button input unit or a touch input unit (S1010). Accordingly, the controller 570 can wake-up the control device. That is, the controller 570 may activate the standby mode of the control device (S1020).

Afterwards, the controller 570 may receive a detection signal from the detection unit 530 (S1030). At this time, the controller 570 may determine the user's status using the detection signal received from the detection unit 530 (S1040). The user's state may include, for example, a stationary state, a moving state, an exercise state, and a riding state. This is an exemplary state and may be expressed as a plurality of states (eg, first to fourth states).

Afterwards, the controller 570 may determine activation of a gesture operation or a touch operation in response to the determined user state (S1050). And the controller 570 may transmit a command to the electronic device in response to the activated operation (S1060). For example, the controller 570 may activate only gesture operations. In this case, even if the user performs a touch action, an output or command corresponding to the touch action may not be transmitted to the electronic device. Additionally, the controller 570 may activate only touch operations. In this case, even if the user performs a gesture operation, an output or command corresponding to the gesture operation may not be transmitted to the electronic device.

More specifically, in the control method according to the embodiment, the step of determining the user state (S1040) includes comparing whether the amplitude of the detection signal is greater than the second set value when the intensity of the detection signal is greater than the first set value ( S1041) may be included.

First, looking at FIG. 30, in a stationary state (first state, state1), the intensity (e.g., average intensity, A1) and amplitude (AM1) of the detection signal may be the smallest compared to other states.

And in the movement state (second state, state2), the intensity of the detection signal may be greater than the first state, and the amplitude (AM2) may be smaller than the first state and the fourth state. Furthermore, in the moving state, the intensity of the detection signal may be smaller than that in the third and fourth states. And in the moving state, the amplitude (AM2) of the detection signal may be smaller than that in the third state.

In the exercise state (third state, state3), the intensity (A3) of the detection signal may be greater than the first and second states and less than the fourth state. And in the exercise state, the amplitude (AM3) of the detection signal may be the largest compared to other states.

Furthermore, in the boarding state (fourth state, state4), the intensity (A4) of the detection signal may be the greatest compared to other states. And in the riding state, the amplitude (AM4) of the detection signal may be greater than the first state and smaller than the second state and the third state.

Accordingly, when the intensity of the detection signal is greater than the first set value, the controller can compare whether the amplitude of the detection signal is greater than the second set value. At this time, the first set value may be a value between A3 and A2.

And the control method according to the embodiment may include determining the user's state as an exercise state when the amplitude of the detection signal is greater than the second set value (S1042).

That is, if the amplitude of the detection signal is greater than the second set value, the controller may determine the user's state to be an exercise state (S1042). The second set value may be a value between AM3 and AM4.

And, if the amplitude of the detection signal is smaller than the second set value, the control method may include comparing whether the intensity of the detection signal is greater than the third set value (S1043).

For example, if the amplitude of the detection signal is smaller than the second set value, the controller may compare the intensity of the detection signal with the third set value (S1043).

Thereafter, the control method may include a step (S1044) of determining the user's state as a boarding state if the strength of the detection signal is greater than the third set value.

That is, if the intensity of the detection signal is greater than the third set value, the controller may determine the user's state as a riding state (S1044). At this time, the third set value may be greater than A3. And the third set value may be greater than the first set value.

And, if the intensity of the detection signal is less than the first set value, the control method may include comparing whether the amplitude of the detection signal is greater than the fourth set value (S1045). The fourth set value may be a value between AM1 and AM2. And the fourth set value may be smaller than the second set value.

Thereafter, the control method may include a step (S1047) of determining the user state as a moving state if the amplitude of the detection signal is greater than the fourth set value.

If the amplitude of the detection signal is greater than the fourth set value, the controller may determine the user's state as a moving state (S1047). In other words, the controller can determine from the amplitude that the user state is not stopped.

Additionally, the control method may include a step (S1046) of determining the user state as a stopped state if the amplitude of the detection signal is smaller than the fourth set value.

If the detection signal is smaller than the fourth set value, the controller may determine that the user is in a stationary state with almost no movement (S1046)

And as described above, after the controller determines the user's state, it can determine activation of the gesture action and touch action in response.

If the amplitude of the detection signal is greater than the second set value, the controller may determine the user's state to be an exercise state (S1042). Afterwards, the controller may block both gesture operations and touch operations (S1051).

And, if the strength of the detection signal is greater than the third set value, the controller may determine the user's state as a boarding state (S1044). Afterwards, the controller may activate gesture operations and touch operations (S1052).

If the detection signal is smaller than the fourth set value, the controller may determine that the user is in a stationary state with almost no movement (S1046). Afterwards, the controller may activate gesture operations and touch operations (S1052).

If the amplitude of the detection signal is greater than the fourth set value, the controller may determine the user's state as a moving state (S1047). Afterwards, the controller can block the gesture operation and activate the touch operation.

As a result, it is possible to easily prevent cases where touch or gesture input due to movement is not made as intended by the user (incorrect input). In other words, even when the user is in a stationary state without moving, the control device according to the embodiment can accurately transmit a control command to the electronic device according to the user's input without incorrect input.

Looking further at FIG. 31, the electronic device may also include a sensing unit as described above. Accordingly, electronic devices or control devices can transmit and receive detection signals to each other. For example, when the difference between the detection signal sig1 of the sensing unit in the electronic device and the detection signal sig2 of the detection unit in the control device is within a predetermined range, a command by the control device may be provided to the electronic device.

And looking further at Figure 32, the user can recognize various objects on a real image through an electronic device. Furthermore, the electronic device can provide virtual objects, etc., corresponding to objects in real images. For example, an electronic device may output information corresponding to a real image (eg, a building) to a user as a virtual object.

Looking further at FIG. 33, the user may output a command to move or select a virtual object using a control device according to an embodiment. That is, the user can select an object (eg, virtual object, information (memo picture)) based on the user interface shown in FIG. 32 through the control device. For example, if the memo-shaped object shown in FIG. 32 is selected, the screen can be switched to FIG. 33.

Thereafter, for example, when the user is in a stationary state, the controller may activate gesture actions and touch actions as described above. Accordingly, when the user taps the control device, the controller can move the selection object or selection object from 'Yes' to 'No' (SL') or transmit a selection command for a specific object to the electronic device.

In addition, when the user shakes the control device in a specific way (e.g., number of times), the controller moves the selection target or selected object from 'Yes' to 'No' (SL') or selects a specific object. A selection command may be transmitted to the electronic device.

Furthermore, looking further at Figure 34, when the touch operation is activated, the controller can set the position of the user's initial input to any of the top, bottom, left, and right with respect to the center.

For example, a touch operation may be activated in any one of the user states described above. At this time, the controller can select any of up, down, left, and right based on the area (AR1, initial input) that the user initially inputted to the touch input unit. At this time, the controller creates a first virtual line (VL1) connecting the center of the button input unit (520) and the initial input, and a second virtual line that is perpendicular to the first virtual line (VL2) and bisects the touch input unit (510). (VL2) can be set. And the controller can set the direction of the first virtual line (VL1) and the second virtual line (VL2) to one of up, down, and left.

For example, after the initial input, if the user specifically touches 'left' on the second virtual line VL2, the control device may transmit a command to move 'left' to the electronic device. And the electronic dabiashi can execute commands in response to the user's touch (input).

As such, a user may need to input or command an electronic device through a control device even while moving. At this time, the control device can easily prevent incorrect input even if the user's entire body is shaking. Furthermore, it can provide comfort to users even while moving.

The control method according to the disclosed embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Additionally, an embodiment of the present disclosure may be a computer-readable recording medium on which one or more programs including instructions for executing a control method are recorded.

FIG. 35 is a side view of an electronic device according to an embodiment, FIG. 36 is a top view of an electronic device according to an embodiment, FIG. 37 is a conceptual diagram of an optical device according to an embodiment, and FIG. 38 is a perspective view of an optical device according to an embodiment. , FIG. 39 is an exploded perspective view of an optical device according to an embodiment, FIG. 40 is a view cut along AA′ in FIG. 38, FIG. 41 is an enlarged view of portion K1 in FIG. 40, and FIG. 42 is an exploded perspective view of an optical device according to an embodiment. This is an enlarged view of part K2.

Referring to FIGS. 35 and 37 , the electronic device according to the embodiment may include a frame 100, an optical device 200, a display unit 300, and a heat dissipation unit (HD), as described above.

Frame 100 may be a glasses tail as described above.

Additionally, the optical device 200 may be disposed on the frame 100 to generate an image. Accordingly, the optical device 200 may include a light source.

Additionally, the display unit 300 is connected to the optical device 200 and can output an image generated by the optical device 200. For this purpose, the optical device 200 may include an optical signal generator 230.

The heat dissipation unit HD may be connected to the optical device 200. And the heat dissipation unit HD may be disposed inside the frame 100. With this configuration, it is possible to easily prevent phenomena such as fogging caused by the environment by the heat dissipation unit HD.

The heat dissipation portion (HD) may be in contact with the light source of the optical device. Accordingly, heat generated by the light source of the optical device can be easily dissipated. Accordingly, even if the user wears the frame, the skin may not be damaged from the heat caused by the optical device. Furthermore, the reliability of optical devices can be improved.

This heat dissipation unit HD may be in contact with the substrate of the light source device of the optical device 200. More specifically, it may be in contact with a substrate that is in contact with a light source. For this purpose, in the light source device the substrate can be exposed from the housing of the light source device. That is, the heat dissipation portion HD easily contacts the substrate on which the light source is mounted through the exposed surface, and the heat dissipation portion can easily dissipate heat.

With further reference to FIGS. 37 to 40 , the optical device 200 according to the embodiment includes a barrel 210, a lens (L), and a first light guide (LG1). Furthermore, the optical device 200 may include a light source device 220 located at or coupled to an opening OP formed on a side of the barrel 210 and an optical signal generator 230 adjacent to the barrel 210. Furthermore, the optical device 200 may further include a cover (CV) surrounding the barrel 210, the light source device 220, and the optical signal generator 230, and a substrate (including a connector, not shown).

Furthermore, a reflective layer or reflective electrode formed of an aluminum layer and having a high optical reflectance may be located below the alignment layer. The reflective electrode may be located on a silicon wafer. And a semiconductor array (CMOS array) can be formed on the silicon wafer. And data signals can be transmitted through the panel using this semiconductor array.

Furthermore, the heat dissipation portion HD may extend from the outer surface of the housing 222 to the frame 100 . As described above, the heat dissipation unit HD is disposed on the outer surface of the housing 222 and may contact the substrate 225 exposed in the housing 222. At this time, the heat dissipation unit HD may not be located on the inner surface 200IS of the optical device 200. That is, the heat dissipation unit HD may be located on a surface opposite to the inner surface 200IS of the optical device 200. Furthermore, the heat dissipation unit HD may be located on a surface opposite to the inner surface 100IS of the frame 100. That is, the heat dissipation unit HD may not be disposed along the inner surface 100IS of the frame 100.

The heat dissipation unit HD may extend along the outer surface of the housing 222 and extend to the front side frame. Additionally, the heat dissipation unit HD may extend from the inside of the frame along the edge of the glass or lens GL.

Furthermore, the heat dissipation unit HD may extend adjacent to the optical signal generating unit 230. For example, the heat dissipation unit HD may extend from the light source device 220 to the lower side of the optical signal generator 230. Accordingly, not only the light source but also the heat generated by the optical signal generator 230 can be easily discharged to the outside. At this time, as described above, the heat dissipation unit HD may extend along the opposite side, that is, the outer side, rather than the inner side, of the optical device 200IS. Accordingly, the wearer may hardly feel any heat. That is, the heat dissipation unit HD may extend upward from the lower side of the optical signal generating unit 230 along the outer surface of the optical device 200 and along the side of the light source device.

Looking further at FIG. 41, the light (La, Lb, Lc) emitted from each light source 223 of the light source device 220 passes through the light source lens 224, the second light guide LG2, and the intermediate lens (MO). can do. For example, the first light (La), the second light (Lb), and the third light (Lc) emitted from each of the first light source 223a to the third light source 223b are directed in the same direction in the second light guide LG2. can be released as

Looking further at FIG. 42 , the first incident light IL may be partially reflected and partially transmitted by the first light guide LG1. That is, the first light guide LG1 may reflect the first polarized light ILa of the first incident light IL and transmit the second polarized light ILb. For example, the first polarization (ILa) and the second polarization (ILb) may each be different types of S/P light. Accordingly, the first polarized light ILa, which is part of the first incident light IL, may be provided to the optical signal generator 230 through the N-th lens Ln. And the second polarized light (ILb) may be absorbed by the barrel 210 or provided through the additional hole 210h.

FIG. 43 is a side view of an electronic device according to another embodiment, FIG. 44 is a top view of an electronic device according to another embodiment, FIG. 45 is a side view of an electronic device according to another embodiment, and FIG. 46 is another embodiment. This is a top view of an electronic device according to . Except for the content described below, the content of the frame, optical device, display unit, and heat dissipation unit described above may be applied in the same manner as the content described above.

Referring to FIGS. 43 and 44 , the electronic device according to the embodiment may further include a heat supply unit HL disposed on the frame 100 .

The heat supply unit HL may be located inside the frame 100. That is, the heat supply unit HL may surround the edge of the lens GL inside the frame 100 like the heat dissipation unit HD. Accordingly, fogging, etc. can be prevented more effectively.

The heat supply unit HL may be electrically connected to the optical device 200. Furthermore, the heat supply unit HL may be electrically connected to a controller or control unit within the electronic device. Alternatively, the heat supply unit (HL) may be electrically connected to a controller or control unit within the optical device.

The heat dissipation unit HD and the heat supply unit HL may be sealed with a sealing member (eg, epoxy) to reduce the influence of moisture from the outside. Furthermore, when fogging occurs on the lens due to a temperature difference and the fogging is not sufficiently removed by the heat dissipation unit HD, the heat supply unit HL can supplement the insufficient heat.

Furthermore, the heat supply unit (HL) and the heat dissipation unit (HD) may be spaced apart from each other. Accordingly, the heat dissipation unit HD can be minimally affected by the heat generated from the heat supply unit HL. That is, the heat dissipation unit HD can efficiently radiate heat generated from the light source and the substrate rather than heat generated from the heat supply unit HL.

For example, the heat supply unit HL and the heat dissipation unit HD may extend in parallel within the frame even if they are separated by a predetermined distance. Accordingly, the occurrence of fogging phenomenon can be more effectively eliminated.

Looking further at FIGS. 45 and 46 , the heat supply unit HL may be located inside the frame 100 as described above. That is, the heat supply unit HL may surround the edge of the lens GL inside the frame 100 like the heat dissipation unit HD. Accordingly, fogging, etc. can be prevented more effectively.

Furthermore, the heat supply unit (HL) and the heat dissipation unit (HD) may be spaced apart from each other. Accordingly, the heat dissipation unit HD can be minimally affected by the heat generated from the heat supply unit HL.

In this embodiment, the frame 100 may include an upper area (US) and a lower area (BS). The upper area US and the lower area BS may bisect the lens GL. At this time, either the heat dissipation unit HD or the heat supply unit HL may be disposed in either the upper area US or the lower area BS. And the other one of the heat dissipation unit (HD) and the heat supply unit (HL) may be disposed in the other one of the upper area (US) and the lower area (BS). For example, the heat dissipation unit HD may be located in the upper area US. Additionally, a heat supply unit (HL) may be located in the lower area (BS).

In addition, as a modified example, the heat dissipation unit HD is arranged to surround the lens GL along the edge of the lens GL within the frame 100, but the heat supply unit HL is located in a portion of the frame 100. It can be located in . Accordingly, in order to effectively remove fogging, the heat supply unit HL may be located at a relatively high rate in the lower area or upper area.

With this configuration, energy efficiency and effective fogging phenomenon can be achieved.

The operating method according to the disclosed embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Additionally, an embodiment of the present disclosure may be a computer-readable recording medium on which one or more programs including instructions for executing an operation method are recorded.

And the computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be those specifically designed and constructed for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

Here, the device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g. electromagnetic waves), and this term is used when data is semi-permanently stored in the storage medium. There is no distinction between temporary storage and temporary storage. For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored.

According to one embodiment, operating methods according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products can be traded as commodities. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

Specifically, it may be implemented as a computer program product including a recording medium in which a program for performing the operating method according to the disclosed embodiment is stored.

Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. belongs to

The term '~unit' used in this embodiment refers to software or hardware components such as FPGA (field-programmable gate array) or ASIC, and the '~unit' performs certain roles. However, '~part' is not limited to software or hardware. The '~ part' may be configured to reside in an addressable storage medium and may be configured to reproduce on one or more processors. Therefore, as an example, '~ part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card.

Although the above description focuses on the examples, this is only an example and does not limit the present invention, and those skilled in the art will be able to You will see that various variations and applications are possible. For example, each component specifically shown in the examples can be modified and implemented. And these variations and differences in application should be construed as being included in the scope of the present invention as defined in the appended claims.

Claims

multiple lenses;

a barrel on which the plurality of lenses are disposed;

a first light guide disposed on the barrel;

an opening formed on a side of the barrel;

a light source device coupled to the opening;

a through hole formed on a side of the barrel; and

An optical device including a light receiving unit disposed adjacent to the through hole.
According to paragraph 1,

An optical device including a barrier portion disposed between the through hole and the light receiving portion.
According to paragraph 1,

It includes; an optical signal generator that generates an optical signal containing image information,

The light receiving unit is an optical device disposed between the first light guide and the optical signal generating unit.
According to paragraph 3,

The plurality of lenses include a first lens and a second lens disposed between the first light guide and the optical signal generator,

An optical device wherein the first lens is smaller than the second lens.
According to paragraph 4,

The light receiving unit is an optical device disposed between the first lens and the second lens.
According to paragraph 4,

The through hole is an optical device disposed between the first lens and the second lens.
According to paragraph 1,

An optical device wherein the light receiving portion protrudes from an outer surface of the barrel.
According to paragraph 2,

An optical device wherein the barrier portion protrudes from an outer surface of the barrel.
According to paragraph 1,

The light source device is,

A housing having an opening formed therein;

a second light guide disposed in the housing; and

An optical device comprising a light source that emits light toward the second light guide.
According to clause 9,

An optical device in which the light receiving unit is driven while light is emitted from the light source.