KR20210019704A - Wheelchair apparatus for movement and expression of user based on eye movement - Google Patents

Abstract

Disclosed is a wheelchair device for movement and expression of a user based on eyeball movement. According to one embodiment of the present invention, the wheelchair device includes: a sensor for sensing whether the user sits; a camera for photographing an eyeball image of a user′s eyeball in response to sitting of the user; a driving unit including a plurality of wheels and a motor for movement; a monitor for outputting a screen; and a control unit for controlling the driving unit based on the eyeball image or controlling the monitor to output the user′s expression.

Description

Wheelchair device for movement and expression of intention based on eye movement {WHEELCHAIR APPARATUS FOR MOVEMENT AND EXPRESSION OF USER BASED ON EYE MOVEMENT}

The following embodiments relate to a wheelchair device for movement and expression of a user's intention based on eye movement.

Wheelchairs are used as auxiliary means for moving indoors and outdoors for patients with disabilities, the disabled, and the elderly. Wheelchairs are divided into manual wheelchairs that are driven by a user's hand holding and rotating the wheels and electric wheelchairs that are moved by rolling the wheels with the rotational force of a motor.

The electric wheelchair is a wheelchair comprising a driving device such as a motor. Since the electric wheelchair has a separate driving device, unlike a manual wheelchair, even a user with discomfort can move with a simple operation.

However, in the case of patients with amyotrophic lateral sclerosis (ALS), commonly referred to as Lou Gehrig's disease, it is difficult to move without the help of a guardian even when using an electric wheelchair.

The embodiments may control a wheelchair based on the user's eye movements and output the user's expression of intention, so that even a user with extremely difficult body movements, such as a patient with Lou Gehrig's disease, can move and express intentions only with the user's eye movements. .

A wheelchair device according to an embodiment includes a sensor for sensing whether the user is seated, a camera for photographing an eyeball image of the user's eye in response to the user's seating, and a plurality of wheels and motors for movement And a driving unit for controlling the driving unit, a monitor for outputting a screen, and a control unit for controlling the driving unit based on the eyeball image or for controlling the monitor to output the user's expression of intention.

The control unit provides a selection screen for a movement mode for controlling the driving unit or an input mode for outputting the user's expression of intention to the user through the monitor in response to the user's seating, and the selection screen When the movement mode is selected as the first movement of the pupil region of the eyeball, the driving unit is controlled in response to the second movement of the pupil region of the eyeball, and the input mode is selected as the movement of the pupil region , It is possible to control the monitor in response to a second movement of the pupil region of the eyeball.

When the input mode is selected, the control unit provides a keyboard for inputting the user's expression of intention and a shortcut button for providing the user's expression of intention according to the frequency of use, and the pupil region of the eyeball According to the second movement, the user's expression of intention input from at least one of the keyboard and the shortcut button may be output through the monitor.

The sensor senses an inclination angle from the horizontal ground of the wheelchair device, and the controller controls the driving unit to stop when the inclination angle is greater than or equal to a critical inclination angle, outputs an emergency signal, and sends an emergency message to the guardian of the user. Can be transmitted.

The control unit may control the driving unit to move to the position of the pager when a call message is received from the pager of the user.

1 is a view showing a wheelchair device according to an embodiment.
FIG. 2 is a flowchart illustrating an operation of the controller illustrated in FIG. 1.
3 is a diagram illustrating an example in which a wheelchair device is implemented.
4 shows an example in which the camera shown in FIG. 1 is implemented in glasses.
5 to 8 are diagrams for explaining an operation of a controller to process an eyeball image.
9 to 11 are diagrams for explaining an example in which a controller provides a moving mode and an input mode to a user.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the rights of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes to the embodiments are included in the scope of the rights.

The terms used in the examples are used for illustrative purposes only and should not be interpreted as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Terms such as first or second may be used to describe various components, but the components should not be limited by terms. The terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of rights according to the concept of the embodiment, the first component may be named as the second component, and similarly The second component may also be referred to as a first component.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of the reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the embodiments, the detailed description thereof will be omitted.

1 is a view showing a wheelchair device according to an embodiment.

The wheelchair device 10 includes a sensor 100, a camera 200, a control unit 300, a monitor 400, and a driving unit 500.

The wheelchair device 10 controls the wheelchair and outputs the user's expression of intention based on the user's eye movement, so that even a user with extremely difficult physical movements, such as a Lou Gehrig's disease patient, can move and express his or her intention without the help of a guardian. Can

The sensor 100 may sense whether the user is seated. For example, the sensor 100 may include an infrared sensor and/or a pressure sensor.

The sensor 100 may sense the angle of inclination of the wheelchair device 10 from the horizontal ground. For example, the sensor 100 may include an acceleration and a gyro sensor.

The sensor 100 may sense an obstacle existing in a proximity distance of the wheelchair device 10. For example, the sensor 100 may include an ultrasonic sensor.

The sensor 100 may transmit the sensed signal to the controller 300.

The camera 200 may capture a user's eyeball image. For example, the camera 200 may continuously capture an eyeball image of the user's eye in response to a seating on the user's wheelchair device 10. The camera 200 may capture a user's eyeball image when receiving a shooting start signal from the controller 300. However, it should be noted that the camera 200 does not necessarily receive a photographing start signal from the control unit 300 to capture a user's eye image.

The camera 200 may be implemented independently from the sensor 100, the control unit 300, the monitor 400, and the driving unit 500. For example, the camera 200 may be implemented solely on the user's glasses or the like. When the camera 200 is implemented alone, the camera 200 may wirelessly communicate with the controller 300.

The camera 200 may transmit the photographed eyeball image to the controller 300.

The controller 300 may control the driving unit 500 based on the eyeball image or the monitor 400 to output the user's expression of intention.

The controller 300 may determine whether the user is seated in a wheelchair based on a signal received from the sensor 100. When it is determined that the user is seated in a wheelchair, the controller 300 may transmit a photographing start signal to the camera 200.

The controller 300 may receive an eyeball image from the camera 200. The controller 300 may extract the movement of the pupil region from the eyeball image. For example, the controller 300 may extract the movement of the pupil region from a plurality of eyeball images continuously received from the camera 200.

In response to the user's seating, the controller 300 may provide a selection screen for a movement mode for controlling the driving unit 500 or an input mode for outputting a user's expression of intention to the user through the monitor 400. The controller 300 may receive a mode desired by the user through the first movement of the pupil region of the eyeball on the selection screen. For example, the user may select a desired mode through a gesture of gazing at a desired mode area or blinking an eye with respect to a movement mode and an input mode selection button displayed on the monitor 400.

When the movement mode is selected as the movement of the pupil region, the controller 300 may control the driving unit 500 in response to the second movement of the pupil region of the eye. When the input mode is selected for the movement of the pupil region, the controller 300 may control the monitor 400 in response to the movement of the pupil region.

When the input mode is selected, the controller 300 may provide a keyboard for inputting the user's expression of intention on the monitor 400 and a shortcut button for providing the user's expression of intention according to the frequency of use. For example, the controller 300 may store words, phrases, emoticons, and the like that the user frequently inputs. The controller 300 may provide stored words, phrases, emoticons, and the like as a shortcut button according to the frequency of use.

The controller 300 may output a user's expression of intention input from at least one of a keyboard and a shortcut button through the monitor 400 according to the second movement of the pupil region of the eyeball. For example, the control unit 300 may output a user's expression of intention through the monitor 400 as text, an emoticon, a voice-transformed sound, or the like.

The control unit 300 switches the camera 200, the control unit 300, the monitor 400, and the driving unit 500 to a sleep mode when the user's seating status is not sensed by the sensor 100 for a certain period of time. can do. The sleep mode may mean a low power mode. The sleep mode may be a mode in which only the constant power of the controller 300 is used and all other power is cut off. That is, when the controller 300 is in the sleep mode, power for controlling the driving unit 500 (eg, power of the motor driver when the controller 300 includes a motor driver) may also be cut off to save power. .

The controller 300 may calculate an inclination angle of the wheelchair device 10 from the horizontal ground based on a signal received from the sensor 100. The controller 300 may control the driving unit 500 to stop when the inclination angle is greater than or equal to the critical inclination angle. The controller 300 may output an emergency signal when the inclination angle is greater than or equal to the critical inclination angle. For example, the control unit 300 may output sound and/or light by using a speaker or LED included in the control unit 300 for an emergency signal. The controller 300 may output an emergency signal using a speaker and a screen included in the monitor 400.

The controller 300 may transmit an emergency message to a guardian of the user when the inclination angle is greater than or equal to the critical inclination angle. When the inclination angle is greater than or equal to the critical inclination angle, the controller 300 may perform a telephone connection to the guardian of the user. The user's guardian may refer to both a person who assists/carers a user with extremely inconvenient mobility and a person whose contact information is stored in advance.

The control unit 300 may control the driving unit 500 to stop when an obstacle approaches a threshold distance or less based on a signal received from the sensor 100.

The controller 300 may receive a call message from a pager (not shown) of the user. The controller 300 may control the driving unit 500 to move to a position of a pager (not shown) included in the call message.

The controller 300 may receive a remote manipulation signal from a remote manipulation device (not shown) of a parent. The controller 300 may provide a parental control mode capable of controlling the driving unit 500 as the parental controls operate according to a remote manipulation signal. Due to the parental control mode, the guardian can easily move the wheelchair device 10 by remotely controlling it.

The controller 300 may provide an Internet Of Things (IoT) control mode capable of controlling other devices capable of communication (eg, lighting equipment, air conditioners, TVs), and the like. Users can control IoT devices in a hospital room or their home through the IoT control mode.

The monitor 400 may output a screen, a sound, etc. according to a signal received from the controller 300. That is, the monitor 400 may include a speaker.

The driving unit 500 may include a plurality of wheels and motors for movement. For example, the motor may be implemented as a'BMGC90B300B' model. The driving unit 500 may move and stop by driving a motor under the control of the control unit 300. The driving unit 500 may further include a brake for stopping.

FIG. 2 is a flowchart illustrating an operation of the controller illustrated in FIG. 1.

The controller 300 may determine whether the user is seated in a wheelchair based on a signal received from the sensor 100 (210 ).

When it is determined that the user is seated in a wheelchair, the controller 300 may transmit a photographing start signal to the camera 200. The camera 200 may capture a user's eyeball image in response to a shooting start signal. The camera 200 may transmit the user's eyeball image to the controller 300. The controller 300 may extract the movement of the pupil region from the eyeball image (220).

The controller 300 may provide a selection screen for a movement mode or an input mode in response to the user's seating (230).

When the movement mode is selected as the first movement of the pupil region of the eyeball on the selection screen, the controller 300 may control the driving unit 500 in response to the second movement of the pupil region of the eyeball (240). When the user ends the movement mode, the controller 300 may provide a movement mode or an input mode selection screen again (250).

When the input mode is selected as the first movement of the pupil region of the eyeball on the selection screen, the controller 300 may control the monitor 400 in response to the second movement of the pupil region of the eyeball (260 ). When the user ends the input mode, the controller 300 may provide a moving mode or an input mode selection screen again (270).

FIG. 3 is a diagram illustrating an example in which a wheelchair device is implemented, and FIG. 4 illustrates an example in which the camera shown in FIG. 1 is implemented in glasses.

As shown in FIGS. 3 and 4, the wheelchair device 10 may implement the camera 200 independently of the sensor 100, the control unit 300, the monitor 400, and the driving unit 500. For example, the camera 200 may be implemented on the user's glasses. When the camera 200 is implemented in the user's glasses, the camera 200 may perform wireless communication with the controller 300.

For example, the camera 200 may have a noise cut filter removed. The camera 200 may be attached with a photosensitive film serving as an infrared cut filter. The camera 200 may include an infrared LED (InfraRed Light Emitting Diode) to capture a user's eyeball image even in a dark environment.

5 to 8 are diagrams for explaining an operation of a controller to process an eyeball image.

The controller 300 may receive an eyeball image (510). The controller 300 may extract pixels of the pupil region from the eyeball image (520). An example in which the control unit 300 extracts a pixel of the pupil region may be as shown in FIG. 6.

The controller 300 may convert the extracted pixels into a red green blue (RGB) image to a Heu Saturation Value (HSV) image (530). The controller 300 may binarize the image using the extracted HSV range (540). An example of the image binarized by the controller 300 may be as shown in FIG. 7.

The controller 300 may form a border using the coordinate value of the pupil region and label the pupil region (550).

9 to 11 are diagrams for explaining an example in which a controller provides a moving mode and an input mode to a user.

9 is a diagram illustrating an example of a movement mode 930 provided by the controller 300. For example, the controller 300 may also display the cursor 950 on the screen of the monitor 400 according to the movement of the labeled pupil region 910.

The controller 300 may control the driver 500 in response to the movement of the labeled pupil region 910. For example, the control unit 300 controls the driving unit 500 when the cursor 950 stays on the left button 970 commanding the movement to the left or when there is a gesture such as blinking of an eye on the left button 970 Thus, the wheelchair device 10 can be moved to the left.

10 and 11 are diagrams for explaining an example of an input mode 1030 provided by the controller 300.

The controller 300 may also display the cursor 1050 on the screen of the monitor 400 according to the movement of the labeled pupil region 1010.

The controller 300 may control the monitor 400 in response to the movement of the labeled pupil region 1010. For example, when the cursor 1050 stays on the Hello button 1070 or there is a gesture 1110 such as blinking an eye on the Hello button 1070, the user's expression of intention is input (1150). It can be judged that it has become.

The controller 300 may control the monitor 400 for the input intention expression 1150 to display the text “Hello” on the screen of the monitor 400.

In FIGS. 10 and 11, the expression of the intention that the controller 300 outputs through the monitor 400 is described simply as text, but the controller 300 uses an emoticon, a sound converted from text to speech, etc. Note that other expressions of opinion can also be printed.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

As described above, although the embodiments have been described by the limited drawings, a person of ordinary skill in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments and claims and equivalents fall within the scope of the following claims.

Claims (5)

A sensor for sensing whether a user is seated;
A camera for taking an eyeball image of the user's eye in response to the user's seating;
A driving unit including a plurality of wheels and motors for movement;
A monitor that outputs a screen; And
A control unit that controls the driving unit based on the eyeball image or controls the monitor to output the user's expression of intention
Wheelchair device comprising a.
The method of claim 1,
The control unit,
In response to the user's seating, a selection screen for a movement mode for controlling the driving unit or an input mode for outputting the user's expression is provided to the user through the monitor, and the eyeball for the selection screen When the movement mode is selected as the first movement for the pupil region, controlling the driving unit in response to the second movement for the pupil region of the eyeball,
When the input mode is selected as the first motion for the pupil region of the eyeball for the selection screen, controlling the monitor in response to the second motion for the pupil region of the eyeball
Wheelchair device.
The method of claim 2,
The control unit,
When the input mode is selected, a keyboard for inputting the user's expression of intention and a shortcut button that provides the user's expression of intention according to the frequency of use are provided,
Outputting the expression of the user's intention inputted from at least one of the keyboard and the shortcut button through the monitor according to the second movement of the pupil region of the eyeball
Wheelchair device.
The method of claim 1,
The sensor,
Sensing the inclination angle from the horizontal ground of the wheelchair device,
The control unit,
When the inclination angle is greater than or equal to a critical inclination angle, controlling the driving unit to stop, outputting an emergency signal, and transmitting an emergency message to the guardian of the user.
Wheelchair device.
The method of claim 1,
The control unit,
When receiving a paging message from the pager of the user, controlling the driving unit to move to the position of the pager
Wheelchair device.

Images