KR20220061614A - A lighting control device and method, and smart galsses and method - Google Patents

Abstract

The present invention relates to a lighting control device and method, and smart glasses and method. To this end, the lighting control device according to the present invention obtains information on a surgeon's line of sight based on at least one piece of image data related to surgery, and measures the brightness of an area corresponding to the surgeon's line of sight to control the light emission of at least one light emitting element. Accordingly, the present invention can provide customized brightness to the surgeon.

Description

A LIGHTING CONTROL DEVICE AND METHOD, AND SMART GALSSES AND METHOD

The present invention relates to a lighting control apparatus and method, and smart glasses and method.

In general, an operating room of a hospital may be equipped with various equipment necessary for a patient's operation. These various devices may be used for a patient's surgery, and may measure a patient's body temperature, heart rate, pulse rate, blood pressure, electrocardiogram, electromyogram, and biosignals such as EEG. In addition, Muyeongdeung for irradiating light to the surgical site of the patient may be arranged in the operating room. These muyeongdeung provide a light source necessary for the surgeon to operate by irradiating light to the surgical site.

However, in the operating room, not only the operating surgeon, but also many medical staff such as anesthesiologists, assistant doctors, and nurses, as well as many people who observe the operation may also be present. As such, when many people approach the patient to see the surgical site, not only may the light irradiated by Muyeongdeung not be transmitted to the surgical site well, but also the light necessary for the surgeon to operate may not be provided. .

Therefore, there is a need to track the surgeon's gaze and properly irradiate the surgical site with light through the muyeongdeung.

In addition, there is a need to identify the surgical site where the light cannot be transmitted through the smart glasses worn by the surgeon and to irradiate the light to the surgical site.

In the prior art, there was a problem in that the surgeon's hands could not maintain a sterile state because the surgeon directly moved the muyeongdeung to the surgical site.

In addition, conventionally, a surgical system capable of optimally maintaining lighting in the operating environment of an operating surgeon has not been provided.

Accordingly, the present invention is to provide a surgical system that can optimize the operating environment of the surgeon.

In addition, the present invention is to provide a lighting control device and method for automatically adjusting the position of the muyeongdeung by tracking the gaze of the surgeon even if the surgeon does not directly touch the muyeongdeung.

Another object of the present invention is to provide smart glasses and a method for improving the operating environment of an operating surgeon by providing light through the smart glasses worn by the operating surgeon to a surgical site where light cannot be provided by muyeongdeung.

The objects of the present invention are not limited to the objects mentioned above, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the examples of the present invention. It will also be readily apparent that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the appended claims.

In order to solve this problem, the present invention may provide appropriate lighting through at least one light emitting device to the surgical site by tracking the surgeon's gaze based on image data related to surgery.

In addition, the present invention can monitor the precise brightness of the surgical site through the sensor unit and the camera, and control the driving unit so that at least one light emitting device is directed toward the surgical site to keep the surgical site bright.

In addition, the present invention can optimally control the lighting of the surgical site by providing the brightness through smart glasses to the surgical site to which the light cannot be provided by Muyeongdeung.

To this end, the lighting control apparatus according to the present invention may include a communication unit, a lighting unit, and a processor electrically connected to the communication unit, and the lighting unit. In addition, the processor may receive at least one image data related to surgery through the communication unit, and acquire information about the surgeon's gaze based on the received at least one image data. In addition, the processor may measure the brightness in a region corresponding to the gaze of the operator, and when the measured brightness is less than or equal to a predetermined brightness, control the light emission of at least one light emitting device included in the lighting unit.

In addition, the lighting control device according to the present invention receives at least one image data related to surgery, and based on the received at least one image data, obtains information about the surgeon's gaze, and corresponds to the surgeon's gaze An operation of controlling the light emission of at least one light emitting device included in the lighting unit may be performed by measuring the brightness in the region to be used, and when the measured brightness is less than or equal to a predetermined brightness.

In addition, the smart glasses according to the present invention identify the eye of the surgeon performing the operation, and the brightness at the surgical site corresponding to the eye, and when the brightness is less than or equal to a predetermined brightness, at least one light emitting element included in the lighting unit It is possible to perform an operation of irradiating the light of the surgical site.

The present invention tracks the surgeon's gaze based on at least one image data related to surgery, and controls the light emission of at least one light emitting device at the surgical site corresponding to the surgeon's gaze, thereby providing customized brightness to the surgeon. .

In addition, the present invention can provide brightness adaptive to the surgical environment by emitting light at least one light emitting element when the brightness in the region corresponding to the operator's gaze is less than or equal to a predetermined brightness.

In addition, the present invention can provide active motion during surgery by including the location information of the surgical site toward which the surgeon's gaze is directed in the image data.

In addition, according to the present invention, by including the location information of the surgical site to which the surgeon's gaze is directed in the image data, the image from the surgeon's point of view can be viewed remotely and can be used as educational materials.

In addition, the present invention may perform pairing with the smart glasses to transmit/receive signals or data to and from the smart glasses.

In addition, the present invention identifies the brightness of the surgical site corresponding to the gaze of the surgeon performing surgery through smart glasses, and, when the brightness is less than or equal to a predetermined brightness, emits light from at least one light emitting device to the surgical site By irradiating the light, it is possible to provide optimal brightness for the surgeon to operate.

In addition to the above-described effects, the specific effects of the present invention will be described together while describing specific details for carrying out the invention below.

1 is an exemplary view showing an operating room environment according to an embodiment of the present invention.
2 is an exemplary view of a lighting control device in an operating room according to an embodiment of the present invention.
3 is a block diagram of a lighting control device according to an embodiment of the present invention.
4 is a flowchart illustrating an operation of a lighting control device according to an embodiment of the present invention.
5 is an exemplary view showing the light emission of the lighting unit of the lighting control device according to an embodiment of the present invention.
6 is a block diagram of smart glasses according to an embodiment of the present invention.
7 is a flowchart illustrating the operation of smart glasses according to an embodiment of the present invention.
8 is an exemplary view illustrating light emission of smart glasses according to an embodiment of the present invention.

The above-described objects, features and advantages will be described below in detail with reference to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains will be able to easily implement the technical idea of the present invention. In describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

Although the first, second, etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from other components, and unless otherwise stated, it goes without saying that the first component may be the second component.

In the following, that an arbitrary component is disposed on the "upper (or lower)" of a component or "upper (or below)" of a component means that any component is disposed in contact with the upper surface (or lower surface) of the component. Furthermore, it may mean that other components may be interposed between the component and any component disposed on (or under) the component.

In addition, when it is described that a component is “connected”, “coupled” or “connected” to another component, the components may be directly connected or connected to each other, but other components are “interposed” between each component. It should be understood that “or, each component may be “connected,” “coupled,” or “connected,” through another component.

Throughout the specification, unless otherwise stated, each element may be singular or plural.

As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various components or various steps described in the specification, some of which components or some steps are It should be construed that it may not include, or may further include additional components or steps.

Throughout the specification, when “A and/or B” is used, it means A, B or A and B, unless otherwise stated, and when “C to D” is used, it means that there is no specific opposite description. Unless otherwise specified, it means that it is greater than or equal to C and less than or equal to D.

Hereinafter, a lighting control apparatus and method, and smart glasses and method according to some embodiments of the present invention will be described.

1 is an exemplary view showing an operating room environment according to an embodiment of the present invention.

Referring to FIG. 1 , in an operating room 100 according to an embodiment of the present invention, a medical staff such as an operating surgeon 102 and an assistant surgeon 101 , and a lighting control device 110 may be disposed. Each arm of the lighting control device 110 may be equipped with a monitor 150 , 160 , a camera 140 , or the Muyeongdeung 120 , 130 . Each arm of the lighting control device 110 has a structure capable of moving up, down, left and right by the physical force of a medical staff. The camera 140 may acquire image data for a surgical site, and the monitors 150 and 160 may display an image obtained by the camera 140 or display data according to a request of a medical team.

As such, the lighting control device 110 may be equipped with at least one Muyeongdeung 120 , 130 , at least one monitor 150 , 160 , and at least one camera 140 . And, each Muyeongdeung may include a plurality of light emitting devices, the light emission angle of the light emitting unit may be controlled based on the gaze of the surgeon 102 .

In addition, the lighting control device 110 may transmit and receive a signal or data with the smart glasses 170 worn by the surgeon 102 .

According to an embodiment, the lighting control device 110 may receive at least one image data related to surgery from the smart glasses 170 to obtain information on the surgeon's gaze. And, the lighting control device 110 measures the brightness in the area corresponding to the gaze of the operator, and compares the measured brightness and a predetermined brightness to control the light emission of at least one light emitting element included in the lighting unit. can

According to an embodiment, the smart glasses 170 may detect the gaze of the surgeon performing the operation, and the brightness at the surgical site corresponding to the gaze of the surgeon. In addition, the smart glasses 170 may compare the brightness with a predetermined brightness and irradiate the light of at least one light emitting device included in the lighting unit to the surgical site.

2 is an exemplary view of a lighting control device in an operating room according to an embodiment of the present invention.

Referring to Figure 2, the lighting unit 120 of the lighting control apparatus 200 in the operating room according to an embodiment of the present invention is a plurality of light emitting element groups consisting of a plurality of light emitting elements, and to control the light emitting angle of each group A driving unit 381 may be included. For example, one light emitting device group 220 may include a plurality of light emitting devices 220a, 220b, 220c, and 220d.

According to an embodiment, the lighting unit 120 may include a holding unit 210 for allowing the surgeon 102 or the assistant surgeon 101 to move the lighting unit 120 during surgery. The operating surgeon 102 or the assistant surgeon 101 can move the lighting unit 120 to a desired site through the gripping unit 210 . In addition, each lighting unit 120 may be connected by a link 110 .

3 is a block diagram of a lighting control device according to an embodiment of the present invention.

Referring to FIG. 3 , the lighting control device 200 according to an embodiment of the present invention includes a communication unit 310 , an interface unit 320 , an input unit 330 , a storage unit 340 , a camera unit 140 , and a speaker. 350 , a sensor unit 360 , a link unit 370 , a lighting unit 120 , and a processor 390 may be included. The lighting control device 200 may include at least one lighting unit 120 (eg, Muyeongdeung). The lighting unit 120 and the camera unit 140 may be mounted on any one link (eg, an arm) of the link unit 370 of the lighting control device 200 . The link may move according to at least a part of an upward direction, a downward direction, a left direction, and a right direction under the control of the driving unit in the link unit 370 .

The configuration of the lighting control device 200 shown in FIG. 3 is according to an embodiment, and the components of the lighting control device 200 are not limited to the embodiment shown in FIG. 3 , and as necessary, some components may be added, changed or deleted.

According to an embodiment, the communication unit 310 includes at least one component included in the lighting control device 200 (eg, the interface unit 320 , the input unit 330 , the storage unit 340 , and the camera unit 140 ). ), the speaker 350, the sensor unit 360, the link unit 370, the lighting unit 120, and the processor 390) and at least one that can transmit and receive at least one signal or information through wired communication or wireless communication It may include one circuit.

According to an embodiment, the communication unit 310 communicates with a component (eg, a communication unit (not shown)) of a hospital server (eg, a server that stores and manages patient data in a hospital) (not shown) and wired or wireless communication It may include at least one circuit capable of transmitting and receiving at least one signal or information through the .

According to an embodiment, the communication unit 310 provides information (eg, name, gender, age, blood type, weight, operation date, operation time, underlying disease) on at least one patient to be operated on from a hospital server (not shown). presence or type, etc.), and information on each patient's surgical site (eg, surgical site, disease name, and image data (eg, CT, MRI, X-Ray, etc.) of the surgical site) may be received.

According to an embodiment, the communication unit 310 is worn by the operating surgeon 102 with information about a patient undergoing surgery in the operating room where the lighting control device 200 is located (eg, image data on the surgical site, operation time, etc.). It can be received from one smart glasses 170 . The communication unit 310 may receive the information after pairing with the smart glasses 170 (eg, communication unit). The information may include information on the gaze that the surgeon 102 is looking at (eg, information on the location of the gaze).

According to an embodiment, the interface unit 320 may be configured to communicate with various external electronic devices (eg, an X-ray imaging device, a computer tomography (CT) imaging device, a magnetic resonance imaging (MRI) imaging device, etc.) in the operating room. It can support specified protocols that can be wired or wirelessly connected. The interface unit 320 may include a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

According to an embodiment, the interface unit 320 includes a lighting control device 200 and an external electronic device (eg, an X-ray imaging device, a computer tomography (CT) imaging device, and a magnetic resonance imaging (MRI) device. It may include a connector capable of physically connecting a photographing device, smart glasses 170, etc.), for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

According to an embodiment, the input unit 330 has an interface capable of transmitting/receiving various data about a patient to be operated on to/from an external device (eg, Universal Serial Bus (USB), an external hard drive (not shown), etc.) may include The input unit 330 may provide various types of inputted information about the patient to the processor 390 . To this end, the input unit 330 may include a physical manipulation member such as a switch or a button, or may include an electrical manipulation member such as a touch key, a touch pad, and a touch screen. Alternatively, the input unit 330 may further include a microphone capable of receiving a voice signal of a user (eg, a surgeon, a nurse, an anesthesiologist, etc.).

According to an embodiment, the storage unit 340 may include a volatile memory or a non-volatile memory. For example, the storage unit 340 may store information, data, programs, etc. necessary for the operation of the lighting control device 200 . Accordingly, the processor 390 may perform a control operation to be described later with reference to information stored in the storage unit 340 . The storage unit 340 may store various platforms. The storage unit 340 is, for example, a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg, SD or XD memory), RAM (RAM), ROM (EEPROM, etc.), and a USB memory may include at least one type of storage medium.

According to an embodiment, the storage unit 340 includes at least one component of the lighting control device 200 (eg, the communication unit 310 , the interface unit 320 , the input unit 330 , the storage unit 340 , Camera unit 140, speaker 350, sensor unit 360, link unit 370, lighting unit 120, and various data (eg, software, applications, acquired information, measured information, control signals, etc.) and instructions related thereto. For example, the storage unit 340 may store a control signal or data received from the lighting control device (200).

According to an embodiment, the storage unit 340 includes information about at least one patient obtained through the communication unit 310 (eg, name, gender, age, blood type, weight, presence or absence of an underlying disease or type, etc.); and information on each patient's surgical site (eg, surgical site, disease name, and image data (eg, CT, MRI, X-Ray, etc.) of the surgical site). The storage unit 340 may store information on a plurality of patients scheduled for surgery and information on a surgical site obtained through the communication unit 310 . For example, the storage unit 340 may automatically delete information about a patient for which surgery is completed and information on a surgical site.

According to an embodiment, the camera unit 140 may include at least one camera. The camera unit 140 may acquire an image (or image data) of a person (eg, a patient, an operating surgeon, a nurse, an anesthesiologist, etc.) inside the operating room. The camera unit 140 may acquire an image (or image data) of a surgical site for a patient on an operating bed. The camera unit 140 may include an intelligent image camera (or smart image intelligent camera) in which an artificial intelligence chip (eg, DQ1) is embedded. The face and surgical site of the surgical patient may be recognized through the image intelligence camera, and at least one image data of the surgical site may be acquired.

According to an embodiment, the image intelligence camera is a camera to which a deep learning-based algorithm that can operate on an artificial intelligence chip (eg, DQ1) is applied, recognizes a surgical patient's face or a surgical site, and a person in the operating room (eg, an operating surgeon, a nurse). , anesthesiologists, etc.) can be distinguished. Such an intelligent image camera may be included in the camera unit 140 of the lighting control device (200).

According to an embodiment, the camera unit 140 may include a camera (eg, a Red Green Blue (RGB) camera) capable of acquiring image data for a surgical site even in a situation in which illumination due to illumination is high. . The camera unit 140 may include a device (eg, LED, IR pattern illumination) that emits light necessary to acquire an image.

According to an embodiment, the speaker 350 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. The speaker 350 may output sound through a sound output device (eg, earphone or headphone). The speaker 350 may output an acoustic signal to the outside of the lighting control device 200 . The speaker 350 may output information to be provided to a person (eg, an operating surgeon, a nurse, an anesthesiologist) existing in the operating room by voice.

According to one embodiment, the speaker 350 may output a voice message according to the operation of the lighting control device 200, or brightness. For example, the speaker 350 may output an acoustic signal (eg, a warning sound, etc.) based on an operation of at least one light emitting device (eg, an LED) of the light emitting unit 382 .

According to an embodiment, the sensor unit 360 may detect the intensity of brightness in the operating room. The sensor unit 360 may be disposed on one side of the link unit 370 of the lighting control device 200 to detect the brightness intensity of the patient's surgical site, and transmit the detection result to the processor 390 . The sensor unit 360 may include at least one sensor disposed on one side of the lighting control device 200 .

According to one embodiment, the link unit 370 may include a plurality of links (eg, arms (Arms)) of the lighting control device (200). Each of the plurality of links may be operably connected to each other, and may be formed to mount a device such as a camera 140 , a lighting unit 120 , or a display device (not shown). Each of the links is similar to a joint of the human body, and may be connected to move in at least one direction of up, down, left, and right. In addition, each link may be physically moved according to the operation of each driving unit under the control of the processor 390 .

According to an embodiment, the lighting unit 120 includes at least one driving unit 381 (eg, a second driving unit) and a light emitting unit 382 including at least one light emitting device operated by each driving unit 381 to emit light. ) may be included. The light emitting unit 382 may be composed of at least one light emitting device (eg, LED), or may be grouped in plurality. For example, the lighting unit 120 may be composed of a plurality of light emitting devices to brighten the surgical site. Each of the plurality of light emitting devices may be disposed to be spaced apart from each other by a predetermined distance (eg, 1 cm or 2 cm).

According to an embodiment, the processor 390 drives software to at least one component connected to the processor 390 (eg, the communication unit 310 , the interface unit 320 , the input unit 330 , and the storage unit 340 ). ), the camera unit 140 , the speaker 350 , the sensor unit 360 , the link unit 370 , and the lighting unit 120 ) may be controlled based on wired communication or wireless communication. The processor 390 may perform various data processing and operations based on wired communication or wireless communication.

According to an embodiment, the processor 390 includes a communication unit 310 , an interface unit 320 , an input unit 330 , a storage unit 340 , a camera unit 140 , a speaker 350 , and a sensor unit 360 . , a link unit 370 , and a command or data received from the lighting unit 120 may be loaded into the storage unit 340 for processing, and the processed data may be stored in the storage unit 340 .

According to an embodiment, the processor 390 may have an artificial intelligence chip (eg, DQ1) 391 embedded therein. Alternatively, the algorithm for the artificial intelligence chip may be implemented by the processor 390 . The artificial intelligence chip is a processor that mimics a human brain neural network, and can support a deep learning algorithm that analyzes, recognizes, infers, and judges various data by itself.

According to an embodiment, the processor 390 may obtain patient information on the surgical site of the patient, and surgical information on the surgeon operating the surgical site through the communication unit 310 (or the storage unit 340). The processor 390 includes information on at least one patient (eg, name, gender, age, blood type, weight, presence or absence or type of underlying disease, etc.) and information on each patient's surgical site (eg, surgery). Image data (eg, CT, MRI, X-Ray, etc.) for a region, disease name, and surgical site may be acquired through the communication unit 310 (or the storage unit 340 ) The processor 390 controls the lighting It is possible to obtain image data for the surgical site photographed by the camera unit 140 of the device 200. Alternatively, the processor 390 may be configured to capture images taken with the smart glasses 170 worn by the surgeon 102. Image data for the surgical site may be acquired.

According to an embodiment, the processor 390 receives at least one image data related to surgery through the communication unit 310 and, based on the received at least one image data, obtains information on the surgeon's gaze can do. The processor 390 may receive image data of a patient undergoing surgery from the smart glasses 170 . The processor 390 may acquire image data acquired in real time through at least one camera attached to the smart glasses 170 worn by the surgeon in real time through the communication unit 310 . The image data may include information on the position of the surgical site to which the eye of the surgeon who performs (or proceeds with) the operation faces.

According to an embodiment, the processor 390 may perform pairing with the smart glasses 170 and receive image data from the smart glasses 170 in real time through a channel connected by the pairing. The image data may include information indicating the location of the surgical site to which the surgeon's gaze is directed. The information may include information indicating where the surgical site is currently being watched by the surgeon wearing the smart glasses 170 .

According to an embodiment, the processor 390 may identify the position of the surgeon's gaze through the received image data, and measure the brightness in an area corresponding to the surgeon's gaze. The processor 390 may acquire gaze information included in at least one image data, and determine a point or region to which the surgeon's gaze is directed based on the gaze position included in the obtained image data.

According to an embodiment, when the measured brightness is less than or equal to a predetermined brightness, the processor 390 may control light emission of at least one light emitting device included in the lighting unit 120 . The processor 390 may control the physical operation of the driving unit 381 so that at least one light emitting device is directed toward the surgical site based on the location information of the surgical site. The processor 390 may control the emission angle of at least one light emitting device by transmitting a control signal for controlling the movement of the link to the driving unit 381 .

4 is a flowchart illustrating an operation of a lighting control device according to an embodiment of the present invention. 5 is an exemplary view showing the light emission of the lighting unit of the lighting control device according to an embodiment of the present invention.

Hereinafter, an operation of the lighting control device according to an embodiment of the present invention will be described in detail with reference to FIGS. 4 and 5 .

According to an embodiment, the lighting control device 200 (eg, the processor 390) may receive at least one image data (S410). The lighting control device 200 (eg, the processor 390 ) may receive image data for a patient undergoing surgery from the smart glasses 170 through the communication unit 310 . The lighting control device 200 (eg, the processor 390 ) transmits image data acquired in real time through at least one camera attached to the smart glasses 170 worn by the surgeon in the operating room in real time through the communication unit 310 . It can be obtained in real time. The image data may include information on the position of the surgical site to which the eye of the surgeon who performs (or proceeds with) the operation faces.

According to an embodiment, the lighting control device 200 (eg, the processor 390) performs pairing with the smart glasses 170, and the image from the smart glasses 170 in real time through a channel connected by the pairing data can be received. The image data may include information indicating the location of the surgical site to which the surgeon's gaze is directed. The information may include information indicating where the surgical site is currently being watched by the surgeon wearing the smart glasses 170 .

According to one embodiment, the lighting control device 200 (eg, the processor 390) may identify the position of the surgeon's gaze through the received image data (S412). The lighting control device 200 (eg, the processor 390 ) may acquire gaze information included in at least one image data. The lighting control device 200 (eg, the processor 390 ) may determine a point or a region to which the gaze of the surgeon in operation is directed based on the gaze position included in the at least one image data.

According to an embodiment, the lighting control device 200 (eg, the processor 390) may measure the brightness at the identified gaze position (S414). The lighting control device 200 (eg, the processor 390 ) may measure the brightness of a specific region of the surgical site based on location information included in at least one image data. The lighting control device 200 (eg, processor 390) identifies a specific region of the surgical site that the surgeon is looking at based on location information included in at least one image data, and the sensor unit 360 (eg: light sensor) to detect the intensity of brightness for the specific area. The sensor unit 360 (eg, an optical sensor) may be attached to any link of the lighting control device 200 . The sensor unit 360 (eg, an optical sensor) may be attached to a part of a link that is easy to detect the brightness of the surgical site.

Referring to FIG. 5 , while the surgeon 102 is operating the surgical site of the patient 510 , the smart glasses 170 worn by the surgeon 102 are being operated through at least one mounted camera. image data can be acquired in real time. In addition, the smart glasses 170 may transmit the acquired image to the lighting control device 200 (eg, a communication unit) 310 in real time or periodically. The smart glasses 170 (eg, communication unit) and the lighting control device 200 (eg, communication unit) 310 may perform pairing with each other, and may transmit/receive signals or data through a communication channel by the pairing.

According to an embodiment, the lighting control device 200 (eg, the processor 390 ) may identify whether a shadow (eg, a shadow by a surgeon) exists in a specific area corresponding to the identified gaze position. The lighting control device 200 (eg, the processor 390 ) analyzes an image obtained from at least one camera 140 connected to the link 110 , and a shadow exists in a specific area corresponding to the identified gaze position. can be identified. The lighting control device 200 (eg, the processor 390 ) may identify whether a shadow exists in the specific region through the pixel color of the specific region in the acquired image.

According to an embodiment, the lighting control device 200 (eg, the processor 390) may determine whether the measured brightness is less than or equal to a predetermined brightness (S416). The lighting control device 200 (eg, the processor 390 ) may compare the intensity of light in the specific area with a predetermined intensity (eg, 100,000 Lux). The lighting control device 200 (eg, the processor 390) determines that when the intensity of light in the specific area is less than the predetermined intensity (eg, 100,000 Lux), the brightness of the surgical site is dark for surgery. can judge Alternatively, if the lighting control device 200 (eg, the processor 390) is not less than the predetermined intensity (eg, 100,000 Lux) of the intensity of light in the specific area, the brightness of the surgical site to operate It can be judged that it is not dark in

According to an embodiment, the lighting control device 200 (eg, the processor 390) may control the emission angle of at least one light emitting device (S418). When the lighting control device 200 (eg, processor 390) determines that the brightness of the surgical site is dark for surgery, at least one light emitting device (eg, LED) emits light to the surgical site, An operation of the driving unit (eg, the second driving unit 381) may be controlled. The lighting control device 200 (eg, the processor 390 ) acquires the stored brightness information (eg, information on brightness suitable for surgery) from the storage unit 340 , and the brightness included in the obtained brightness information and the measured brightness may be compared. Then, when it is determined that the brightness of the surgical site is dark for surgery, the lighting control device 200 (eg, the processor 390) emits light so that at least one light emitting element of the lighting unit 120 faces the surgical site. The light emitting angle of the device may be controlled through the light emitting unit 382 .

According to an embodiment, the lighting unit 120 may include at least one driving unit for controlling the emission angle of at least one light emitting device. Alternatively, the lighting unit 120 may include one driving unit for controlling the angle for each light emitting element group composed of a plurality of light emitting elements. For example, the lighting unit 120 may include a plurality of driving units and at least one light emitting element group including a plurality of light emitting elements whose light emission angle is controlled by each driving unit. For example, one light emitting device group may include a plurality of light emitting devices (eg, 4 or 6). Each light emitting device group is assigned a unique identifier, and this identifier is stored in the storage unit 340 . The processor 390 may determine where each light emitting element group (or each light emitting element) is mounted in the lighting unit 120 through the identifier of one light emitting element group. In addition, the number of the plurality of light emitting devices may be variably adjusted.

Referring to FIG. 5 , for example, light emission 530 of at least one light emitting element of the second lighting unit 130 is directly on the surgical site of the patient by a body part (eg, head) of the surgeon 102 . When not irradiated, at least one light emitting element of the first lighting unit 120 may be controlled to emit light 520 to the surgical site under the control of the processor 390 .

According to one embodiment, the lighting control device 200 (eg, the processor 390) is the brightness by the second lighting unit 130 in the identified surgical site based on the gaze position of the surgeon 102 surgery If it is determined that the brightness is less than the easy brightness, the physical operation of the first lighting unit 120 may be controlled through the link unit 370 . And, the lighting control device 200 (eg, the processor 390) is a corresponding driving unit (eg, the second driving unit 381) so that the light emitting angle of at least one light emitting element of the first illumination unit 120 toward the surgical site. )) can be controlled.

According to an embodiment, the lighting control device 200 (eg, the processor 390 ) may emit light based on the controlled light emission angle ( S420 ). The lighting control device 200 (eg, the processor 390 ) controls the at least one first driving unit 371 of the link unit 370 to control the operation of the link, thereby at least one lighting unit 120 , 130 ). operation can be controlled. For example, the first driving unit 371 may control the physical operation of the at least one lighting unit 120 , 130 . In addition, the physical operation of the at least one light emitting device included in the at least one lighting unit 120 and 130 may be controlled by the second driving unit 381 included in each lighting unit 120 .

Referring to FIG. 5 , in a state in which at least one light emitting element of the second lighting unit 130 is emitting light to the surgical site, if it is determined that the brightness by the second lighting unit 130 is less than or equal to the brightness that is easy for surgery, the lighting The control device 200 (eg, the processor 390 ) may emit light to the at least one light emitting device of the first lighting unit 120 to the surgical site.

6 is a block diagram of smart glasses according to an embodiment of the present invention.

Referring to FIG. 6 , the smart glasses 170 according to an embodiment of the present invention include a communication unit 610 , an input unit 620 , a storage unit 630 , a power supply unit 640 , a camera unit 650 , and a sensor unit ( 660 ), a lighting unit 670 , and a processor 680 may be included. The lighting unit 670 may include a light emitting unit 672 including at least one light emitting device and a driving unit 671 for controlling a light emission angle of the at least one light emitting device. The at least one light emitting device may move according to at least a part of an upward direction, a downward direction, a left direction, and a right direction under the control of the driving unit 671 .

The configuration of the lighting control device 200 shown in FIG. 6 is according to an embodiment, and the components of the lighting control device 200 are not limited to the embodiment shown in FIG. 3 , and as necessary, some components may be added, changed or deleted.

According to an embodiment, the communication unit 610 performs pairing with the lighting control device 200 under the control of the processor 680, and includes at least one circuit capable of transmitting a signal or image data through the pairing. can In addition, the communication unit 610 may receive at least some information related to surgery from the lighting control device 200 .

According to an embodiment, the communication unit 310 transmits information about a patient undergoing surgery in an operating room in which the lighting control device 200 is located (eg, image data for a surgical site, surgery time, etc.) to the lighting control device 200 . can be sent to The communication unit 310 may transmit the information after performing pairing with the lighting control device 200 (eg, the communication unit 310). The information may include information on the gaze that the surgeon 102 is looking at (eg, information on the location of the gaze).

According to an embodiment, the input unit 620 is an interface capable of transmitting/receiving various data about a patient to be operated to/from an external device (eg, a Universal Serial Bus (USB), an external hard drive (not shown), etc.) may include The input unit 620 may provide various types of inputted information about the patient to the processor 680 . To this end, the input unit 620 may include a physical manipulation member such as a switch or a button, or may include an electrical manipulation member such as a touch key, a touch pad, and a touch screen. Alternatively, the input unit 620 may further include a microphone capable of receiving a voice signal of a user (eg, a surgeon, a nurse, an anesthesiologist, etc.).

According to an embodiment, the storage unit 630 may include a volatile memory or a non-volatile memory. For example, the storage unit 630 may store information, data, programs, etc. necessary for the operation of the smart glasses 170 . Accordingly, the processor 680 may perform a control operation to be described later with reference to information stored in the storage unit 630 . The storage unit 630 may store various platforms. The storage unit 630 is, for example, a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg, SD or XD memory), RAM (RAM), ROM (EEPROM, etc.), and a USB memory may include at least one type of storage medium.

According to an embodiment, the storage unit 630 may store image data acquired in the operating room in which the lighting control device 200 is located. In addition, the storage unit 630 may store information about a patient undergoing surgery (eg, image data on a surgical site, an operation time, etc.).

According to an embodiment, the power supply unit 640 may include an interface (not shown) for a wired connection or a wireless connection with an external power source. The interface (not shown) may include a high-definition multimedia interface (HDMI), a universal serial bus (USB), an optical interface, or a D-subminiature (D-sub). The interface (not shown) may include, for example, a mobile high-definition link (MHL) interface, a secure digital (SD) card/multi-media card (MMC) interface, or an infrared data association (IrDA) standard interface. there is. The power supply unit 640 includes a battery (not shown), and may supply power necessary for the smart glasses 170 to operate based on the power charged in the battery.

According to an embodiment, the camera unit 650 may include at least one camera. The camera unit 650 may acquire an image (or image data) of a person (eg, a patient, an operating surgeon, a nurse, an anesthesiologist, etc.) inside the operating room. The camera unit 650 may acquire an image (or image data) of a surgical site for a patient on an operating bed. The camera unit 650 may include an image intelligence camera (or smart image intelligence camera) in which an artificial intelligence chip (eg, DQ1) is embedded. The face and surgical site of the surgical patient may be recognized through the image intelligence camera, and at least one image data of the surgical site may be acquired.

According to an embodiment, the image intelligence camera is a camera to which a deep learning-based algorithm that can operate on an artificial intelligence chip (eg, DQ1) is applied, recognizes a surgical patient's face or a surgical site, and a person in the operating room (eg, an operating surgeon, a nurse). , anesthesiologists, etc.) can be distinguished. Such an intelligent image camera may be included in the camera unit 650 of the smart glasses 170 .

According to an embodiment, the camera unit 650 may include a camera (eg, a Red Green Blue (RGB) camera) capable of acquiring image data for a surgical site even in a situation in which illumination due to illumination is high. . The camera unit 650 may include a device (eg, LED, IR pattern illumination) that emits light necessary to acquire an image.

According to an embodiment, the sensor unit 660 may detect the intensity of brightness in the operating room. The sensor unit 660 may be disposed on one side of the smart glasses 170 to detect the brightness intensity of the patient's surgical site, and transmit the detection result to the processor 680 .

According to an embodiment, the lighting unit 670 may include at least one driving unit 671 and a light emitting unit 672 including at least one light emitting device operated by each driving unit 671 to emit light. The light emitting unit 672 may be composed of at least one light emitting device (eg, LED), or may be grouped in plurality.

According to an embodiment, the processor 680 drives software to at least one component connected to the processor 680 (eg, a communication unit 610 , an input unit 620 , a storage unit 630 , and a power supply unit 640 ). ), the camera unit 650 , the sensor unit 660 , and the lighting unit 670 ) may be controlled based on wired communication or wireless communication. The processor 680 may perform various data processing and operations based on wired communication or wireless communication.

According to an embodiment, the processor 680 may have an artificial intelligence chip (eg, DQ1) 681 embedded therein. Alternatively, the algorithm for the artificial intelligence chip may be implemented by the processor 680 . The artificial intelligence chip is a processor that mimics a human brain neural network, and can support a deep learning algorithm that analyzes, recognizes, infers, and judges various data by itself.

According to an embodiment, the processor 680 may perform pairing with the lighting control device 200 . The processor 680 performs pairing with the lighting control device 200 in order to transmit information (eg, image data on the surgical site, operation time, etc.) about the patient undergoing surgery in the operating room to the lighting control device 200 . can

According to an embodiment, the processor 680 may identify the eye of the surgeon performing the operation and the brightness of the surgical site corresponding to the eye through the sensor unit 660 . The processor 680 may obtain at least one image data of the pupil of the surgeon through at least one camera in the camera unit 650, and track the gaze of each pupil of the surgeon through the acquired image data. .

According to an embodiment, the processor 680 may identify a surgical site through the gaze of each pupil of the surgeon, and may emit at least one light emitting device in the light emitting unit 672 to the identified surgical site. For example, the processor 680 may compare the brightness at the surgical site with a predetermined brightness. The processor 680 may compare the intensity of light at the surgical site with a predetermined intensity (eg, 100,000 Lux). When the intensity of light at the surgical site is less than the predetermined intensity (eg, 100,000 Lux), the processor 680 may determine that the brightness of the surgical site is dark for surgery.

According to an embodiment, the processor 680 performs pairing with the lighting control device 200, and information (eg, image data for a surgical site, operation time, etc.) may be transmitted to the lighting control device 200 . The image data may include information indicating the location of the surgical site to which the surgeon's gaze is directed. The information may include information indicating where the surgical site is currently being observed by the surgeon wearing the smart glasses 170 .

According to an embodiment, the processor 680 receives the control signal from the lighting control device 200 through the communication unit 610 in response to transmitting at least one image data to the lighting control device 200. can The processor 680 receives a signal transmitted by the lighting control device 200 to control the operation of the lighting unit 670 (eg, the light emitting unit 672), and based on the received signal, at least one driving unit You can control the physical operation of (671). Also, the processor 680 may control light emission of at least one light emitting unit 672 based on the signal. ,

7 is a flowchart illustrating the operation of smart glasses according to an embodiment of the present invention. 8 is an exemplary view illustrating light emission of smart glasses according to an embodiment of the present invention.

Hereinafter, the operation of the smart glasses according to an embodiment of the present invention will be described in detail with reference to FIGS. 7 and 8 .

According to an embodiment, the smart glasses 170 (eg, the processor 680 ) may identify the position of the surgeon's gaze ( S710 ). The smart glasses 170 (eg, the processor 680 ) may identify the gaze of the surgeon performing the surgery, and the brightness of the surgical site corresponding to the gaze. The smart glasses 170 (eg, the processor 680 ) may track the gaze of each pupil of the surgeon through at least one image data of the pupil of the surgeon identified through the camera.

According to an embodiment, the smart glasses 170 (eg, the processor 680) acquires image data through at least one camera, and determines a point or a region to which the gaze of the surgeon in operation is directed through the obtained image data. can do. The smart glasses 170 (eg, the processor 680 ) may utilize well-known eye tracking technology.

According to an embodiment, the smart glasses 170 (eg, the processor 680 ) may measure the brightness by the light emitting unit at the identified gaze position ( S712 ). The smart glasses 170 (eg, the processor 680 ) may identify the brightness of the surgical site through information about the brightness intensity obtained from the sensor unit 660 attached to the smart glasses 170 . The sensor unit 660 may be disposed on one side of the smart glasses 170 to detect the brightness intensity of the patient's surgical site, and transmit the detection result to the processor 680 .

According to an embodiment, the smart glasses 170 (eg, the processor 680) may identify whether the measured brightness is less than or equal to a predetermined brightness (S714). The smart glasses 170 (eg, the processor 680 ) may determine a point or a region toward which the eye of the surgeon undergoes surgery through at least one image data. In addition, the smart glasses 170 (eg, the processor 680) emits light of at least one light emitting device included in the light emitting unit 672 when the brightness of the surgical site is less than or equal to a predetermined brightness (eg, 100,000 Lux). can be controlled The processor 680 may control the physical operation of the driving unit 671 so that at least one light emitting device is directed toward the surgical site. The processor 680 may control the light emitting angle of at least one light emitting device by transmitting a control signal for controlling the movement of the link to the driving unit 671 .

According to one embodiment, the smart glasses 170 (eg, the processor 680), when the intensity of the light of the surgical site is not less than the predetermined brightness (eg, 100,000 Lux), the brightness of the surgical site Therefore, it can be judged that it is not dark.

According to an embodiment, the smart glasses 170 (eg, the processor 680 ) may emit a light emitting device of the class of the surgeon at the identified gaze position ( S716 ). When it is determined that the brightness of the surgical site is dark for surgery, the smart glasses 170 (eg, the processor 680) include a driving unit ( For example, the operation of the driving unit 671) may be controlled. The smart glasses 170 (eg, the processor 680) acquires the stored brightness information (eg, information on brightness suitable for surgery) from the storage unit 630, the brightness included in the obtained brightness information and The measured brightness may be compared.

According to an embodiment, when the smart glasses 170 (eg, the processor 680 ) determine that the brightness of the surgical site is dark for surgery, at least one light emitting element of the lighting unit 670 illuminates the surgical site. The light emitting angle of the light emitting device may be controlled through the light emitting unit 672 to face the light emitting device.

Referring to FIG. 8 , while at least one light emitting element of each of the first lighting unit 120 and the second lighting unit 130 is irradiating light to the surgical site, a body part (eg, the head) of the surgeon 102 . ) may not be transmitted. In this case, the smart glasses 170 (eg, the processor 680 ) may identify whether a shadow (eg, a shadow by a surgeon) exists at the surgical site through the sensor unit 660 .

Alternatively, the smart glasses 170 (eg, the processor 680 ) may analyze the image obtained from the at least one camera 650 to identify whether a shadow exists in the surgical site corresponding to the identified gaze position. . The smart glasses 170 (eg, the processor 680 ) may identify whether a shadow exists in the specific area through the pixel color of the specific area in the acquired image.

In addition, when the smart glasses 170 (eg, the processor 680 ) are identified as having a shadow in the surgical site, at least one light emitting device in the light emitting unit 672 emits light 810 to the surgical site. (670) can be controlled.

Each step in each of the above-described flowcharts may be operated regardless of the illustrated order, or may be performed simultaneously. In addition, at least one component of the present invention and at least one operation performed by the at least one component may be implemented in hardware and/or software.

As described above, the present invention has been described with reference to the illustrated drawings, but the present invention is not limited by the embodiments and drawings disclosed in the present specification. It is obvious that variations can be made. In addition, although the effects according to the configuration of the present invention have not been explicitly described and described while describing the embodiments of the present invention, it is natural that the effects predictable by the configuration should also be recognized.

120: lighting unit 140: camera unit
200: lighting control device 310: communication unit
320: interface unit 330: input unit
340: storage 350: speaker
360: sensor unit 370: link unit
382: light emitting unit 390: processor

Claims (20)

A lighting control device comprising:
communication department;
lighting unit; and
It includes a processor electrically connected to the communication unit, and the lighting unit,
The processor is
Receive at least one image data related to surgery through the communication unit,
Based on the received at least one image data, obtain information about the surgeon's gaze,
Measuring the brightness in the area corresponding to the gaze of the operator,
When the measured brightness is less than or equal to a predetermined brightness, a lighting control device set to control the light emission of at least one light emitting element included in the lighting unit.
According to claim 1,
The lighting unit,
Lighting control device comprising at least one driver for controlling the light emission angle of the at least one light emitting element.
According to claim 1,
The video data is
Lighting control device including location information of the surgical site to which the gaze of the surgeon performing the operation is directed.
4. The method of claim 3,
The processor is
Based on the location information of the surgical site, the lighting control device for operating the driving unit to direct the at least one light emitting element toward the surgical site.
According to claim 1,
The lighting unit,
a plurality of driving units; and
A lighting control apparatus comprising at least one light emitting element group comprising a plurality of light emitting elements whose light emission angle is controlled by each of the plurality of driving units.
According to claim 1,
including a link,
The processor is
A lighting control device set to control the movement of the lighting unit by controlling at least one driving unit included in the link unit.
According to claim 1,
The processor is
Pairing with smart glasses,
A lighting control device set to transmit and receive signals or data to and from the smart glasses through the communication unit based on the pairing.
In smart glasses,
communication department;
sensor unit;
lighting unit; and
It includes a processor electrically connected to the communication unit, the sensor unit, and the lighting unit,
The processor is
Identifies the eye of the surgeon performing the operation, and the brightness at the surgical site corresponding to the eye through the sensor unit,
When the brightness is less than a predetermined brightness, smart glasses set to irradiate the light of at least one light emitting element included in the lighting unit to the surgical site.
9. The method of claim 8,
Comprising at least one camera for acquiring at least one image data for the pupil of the surgeon,
The processor is
Smart glasses set to track the gaze of each pupil of the surgeon through the obtained at least one image data.
10. The method of claim 9,
The processor is
adding information on the position of the tracked gaze to the at least one image data obtained,
Smart glasses set to transmit at least one image data to which information on the position of the tracked gaze is added to the lighting control device through the communication unit.
11. The method of claim 10,
The processor is
In response to the transmission of the at least one image data, receiving a control signal from the lighting control device through the communication unit,
Smart glasses set to irradiate the light of at least one light emitting device included in the lighting unit to the surgical site based on the received control signal.
11. The method of claim 10,
The processor is
Identifies an area corresponding to the tracked gaze in the surgical site,
Smart glasses configured to emit light to the at least one light emitting element into the identified area.
9. The method of claim 8,
The processor is
perform pairing with the lighting control device;
A lighting control device set to transmit and receive signals or data through the communication unit with the lighting control device based on the pairing.
In the lighting control method of a lighting device,
receiving at least one image data related to surgery;
obtaining information on a gaze of a surgeon based on the received at least one image data;
measuring the brightness in an area corresponding to the gaze of the surgeon; and
and controlling light emission of at least one light emitting element included in the lighting unit when the measured brightness is less than or equal to a predetermined brightness.
15. The method of claim 14,
The method further comprising: operating a driving unit to direct the at least one light emitting device to the surgical site based on the location information of the surgical site to which the surgeon's gaze is directed.
15. The method of claim 14,
the process of performing pairing with smart glasses; and
Lighting control method further comprising the step of transmitting and receiving a signal or data with the smart glasses based on the pairing.
In the lighting control method of smart glasses,
The process of identifying the gaze of the surgeon performing the operation, and the brightness of the surgical site corresponding to the gaze; and
When the brightness is less than a predetermined brightness, lighting control method comprising the step of irradiating the light of at least one light emitting device included in the lighting unit to the surgical site.
18. The method of claim 17,
acquiring at least one image data of the pupil of the surgeon; and
The lighting control method further comprising the step of tracking the gaze of each pupil of the surgeon through the obtained at least one image data.
19. The method of claim 18,
adding information on the position of the tracked gaze to the at least one obtained image data; and
The lighting control method further comprising the step of transmitting at least one image data to which information on the position of the tracked gaze is added to a lighting control device.
20. The method of claim 19,
In response to the transmission of the at least one image data, the process of receiving a control signal from the lighting control device; and
Lighting control method further comprising the step of irradiating light from at least one light emitting device included in the lighting unit to the surgical site based on the received control signal.

Images