KR20210100922A - A Method of operation of a terminal providing a variable setting user interface to compensate for visual constraints - Google Patents

Abstract

According to an embodiment of the present invention, an operation method of a terminal providing a user interface includes: a user activity information analysis step of collecting and analyzing user interface adjustment information including at least one of font size adjustment, screen brightness adjustment, and color adjustment input in response to the user interface at regular intervals; a step of determining variable setting information, which compensates for visual constraints of a terminal user regarding the user interface, based on the information obtained by analyzing the user activity information; and a step of outputting the variable interface that the variable setting information is applied to the user interface.

Description

A method of operation of a terminal providing a variable setting user interface to compensate for visual constraints

The present invention relates to a terminal providing an interface and an operating method thereof. More specifically, the present invention relates to a method of operating a terminal that provides a variable user interface that supplements a user's visual constraint.

A user interface is software that receives graphic and sound outputs and a user input corresponding thereto to drive an electronic device such as a terminal, and has been implemented and improved in various forms according to the development of user input means. Terminal applications using such a user interface are widely used based on information technology and information processing fields in almost all fields of information technology, especially in the fields of telecommunications, entertainment electronics and industrial production and processing plants.

In particular, a function of an application provided through a currently released mobile terminal device and an Internet service is provided through an installed graphic interface software. Graphical interface software is developed in close connection with the functions provided by the application.

In general, the user interface arranges and outputs an appropriate range input based on a general user without physical limitations and graphic input elements corresponding thereto on the display to facilitate the use of terminal functions or the user's access to functions, The touch or keyboard input is implemented as a basic input device and an acoustic device can be used as an auxiliary input and output device.

In order to obtain maximum input efficiency based on the diverse and rich experiences of these general users, the manipulation input of the current user interface is processed by measuring and analyzing very minute changes in the body due to the user's touch range and voice analysis.

However, it is very difficult for users, such as the disabled, the elderly, or the elderly, who are not general users to have physical discomfort or have physical restrictions on specific user input, to obtain normal input and desired response results for the graphic user interface.

In particular, recently, it is also possible to reflect individual preferences or physical characteristics by providing a function to change setting values such as font size, basic sound size, and screen mode of the graphic user interface. This makes it a very difficult task for users with physical limitations.

Furthermore, technologies such as domestic patent 10-2010-0087938 that support a color change function on a graphic interface in consideration of color weakness among these physical limitations in computer game interfaces, etc. have been proposed, but the majority of users use smartphones and tablets. It is not yet known whether such a setting function exists in the basic OS software such as

In addition, only users who do not have physical restrictions on basic input such as touch can make detailed settings such as color, font, screen, and voice change. It is almost impossible.

The present invention has been devised to solve the above problems, and by providing a variable user interface that supplements the user's physical constraints, an interaction pattern input by users such as the disabled, the elderly, and the elderly, who are inconvenient to input to the interface itself, The purpose of this is to provide a terminal and an operation method thereof, which can understand their physical limitations through a user interface, and provide a variable interface that responds accordingly, so that a user interface such as a terminal can be conveniently used without a separate optimization setting. .

In the method according to an embodiment of the present invention for solving the above-described problems, in the method of operating a terminal providing a user interface, font size adjustment, screen brightness adjustment, and color adjustment input corresponding to the user interface are performed. a user activity information analysis step of collecting and analyzing user interface adjustment information including at least one in a predetermined period unit; determining, based on the information obtained by analyzing the user activity information, variable setting information that supplements the visual constraint of the terminal user for the user interface; and outputting a variable interface to which the variable setting information is applied to the user interface.

In addition, indexing the user activity information, performing trend analysis based on the indexing information for each predetermined period; It may further include; providing a guide service for preventing visual diseases by performing statistical analysis on the trend analysis result.

An apparatus according to an embodiment of the present invention for solving the above-described problems, in a terminal providing a user interface, analyzes one or more user inputs input in response to the user interface, and thereby physically constrains the user of the terminal a physical constraint information processing unit for obtaining information; a variable interface setting unit configured to determine variable setting information supplementing the physical constraint of the terminal user for the user interface based on the physical constraint information; and an interface output unit configured to output a variable interface to which the variable setting information is applied to the user interface.

According to an embodiment of the present invention, by analyzing one or more user inputs input in response to the user interface, physical restriction information of the terminal user is obtained, and based on the physical restriction information, the terminal for the user interface As the variable setting information that supplements the physical constraint of the user is determined, a variable interface to which the variable setting information is applied to the user interface may be output.

Accordingly, the terminal according to an embodiment of the present invention provides a variable graphic user interface to compensate for the user's physical limitations through various analysis of the user's terminal activity information or test interface, so that the input to the interface itself It makes it possible to use a user interface such as a terminal more conveniently in spite of the individual constraints of users such as the disabled, the elderly, and the elderly.

In addition, according to an embodiment of the present invention, even if users do not specifically recognize their physical constraints environment, disease, change due to aging, etc., or do not directly perform individual detailed settings of the interface, they Accordingly, users can experience a naturally supplemented user interface, making it more convenient to use user interface-based terminals such as smartphones and tablets.

1 is a conceptual diagram schematically illustrating a terminal according to an embodiment of the present invention.
2 is a block diagram illustrating in more detail a variable interface service unit of a terminal according to an embodiment of the present invention.
3 is a flowchart illustrating a method of operating a terminal according to an embodiment of the present invention.
4 to 8 are exemplary diagrams for explaining a test interface according to an embodiment of the present invention.
9 is a flowchart illustrating a method of operating a terminal according to another embodiment of the present invention.
10 to 12 are diagrams for explaining an activity information analysis method according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art will be able to devise various devices which, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. Moreover, it is to be understood that all conditional terms and examples listed herein are, in principle, expressly intended solely for the purpose of enabling the concept of the present invention to be understood, and not limited to the specifically enumerated embodiments and states as such. should be

Moreover, it is to be understood that all detailed description reciting the principles, aspects, and embodiments of the invention, as well as specific embodiments, are intended to cover structural and functional equivalents of such matters. It is also to be understood that such equivalents include not only currently known equivalents, but also equivalents developed in the future, i.e., all devices invented to perform the same function, regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of illustrative circuitry embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc. may be tangibly embodied on computer-readable media and be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly shown. should be

The functions of the various elements shown in the figures including a processor or functional blocks represented by similar concepts may be provided by the use of dedicated hardware as well as hardware having the ability to execute software in association with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, clear use of terms presented as processor, control, or similar concepts should not be construed as exclusively referring to hardware having the ability to execute software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM (RAM) and non-volatile storage. Other common hardware may also be included.

In the claims of the present specification, a component expressed as a means for performing the function described in the detailed description includes, for example, a combination of circuit elements that perform the function or software in any form including firmware/microcode, etc. It is intended to include all methods of performing the functions of the device, coupled with suitable circuitry for executing the software to perform the functions. Since the present invention defined by these claims is combined with the functions provided by the various enumerated means and combined in a manner required by the claims, any means capable of providing the functions are equivalent to those contemplated from the present specification. should be understood as

The above-described objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram specifically illustrating a terminal according to an embodiment of the present invention.

The terminal 100 described in this specification includes a tablet PC, a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation device, etc. may be included.

In addition, the terminal 100 may be connected to a network by wire or wirelessly, and for mutual communication between networks, the terminal 100 is a local area network, Internet network, LAN, WAN, PSTN (Public Switched Telephone Network), PSDN (Public Switched) Data Network), cable TV network, WIFI, mobile communication network, and other wired and wireless communication networks can transmit and receive data.

And, according to an embodiment of the present invention, the terminal 100 is a terminal of a user having physical limitations, such as a disability, and may be, for example, a terminal of a senior who is a protection target. The guardian's terminal may be pre-authenticated and registered, or may be pre-authenticated and registered in the terminal 100 .

In addition, the terminal 100 outputs various senior contents or messages by a senior service preset in a service providing device or a guardian terminal for a senior or driven according to preset setting information according to an input of the terminal itself, and corresponding to it. It can receive and process the user input of the senior. Here, the senior service may be installed in the terminal 100 in the form of application software or a separate application, or may be pre-loaded in the terminal 100 in the form of system software.

In addition, the terminal 100 may provide various senior content services utilizing the unique functions of the portable terminal. The content service includes a video call service with the guardian terminal 300, a message conversation service, a photo album service, a dementia prevention program service, a health information management service, a medication management service, an activity report service, a music service, a radio service, a video service, and an Internet service. , a game service, a song service, etc. may be exemplified, and the senior may perform various activities through a content service using the terminal 100 .

Accordingly, the terminal 100 may collect activity information according to the senior's use of the content service and activity information corresponding to the sensing and user input of the terminal 100, and such activity information is stored and updated and analyzed continuously. and may be used for variable provision of a user interface, which will be described later in more detail.

In addition, the service providing device may also be a device that performs pre-registration for the terminal 100 , and inputs setting information for senior activity information management and variable interface provision, and may be an institution server such as a nursing institution or a medical institution. can be In addition, the service providing device may be a senior service providing server that registers and manages the terminal 100 and the guardian terminal, and relays setting information.

In addition, the service providing apparatus may be a server that provides a service or a third service organization server. Also, the service providing apparatus 200 may use information received from an external service provider server to provide content suitable for a senior to be protected.

Accordingly, in the present specification, the processes according to an embodiment of the present invention are mainly described as an operation of the terminal 100 itself, but the present invention is not limited thereto.

The terminal 100 according to this embodiment of the present invention can generate and output various user interfaces for providing the content service as described above, and receives a user input corresponding to the user interface and performs a reaction process corresponding thereto. can be done

Here, the user interface may be an input/output interface provided to the user using at least one of visual, auditory, and tactile sense, for example, a touch screen user interface supporting graphic and audio output, etc. may be exemplified.

And, the terminal 100 according to an embodiment of the present invention, by analyzing one or more user inputs input in response to the user interface, obtains the physical constraint information of the terminal user, based on the physical constraint information, As variable setting information that supplements the physical constraint of the terminal user with respect to the user interface is determined, a variable interface to which the variable setting information is applied to the user interface may be output.

More specifically, the physical constraint information may include at least one of visual constraint information, auditory constraint information, and input freedom constraint information. By reflecting the variable setting information on the user interface, physical restrictions on using the user interface can be supplemented.

For example, the variable setting information may include a font size setting, a color setting, a brightness setting, a color vision abnormality setting, etc. that supplements the visual constraint on the graphic user interface. The terminal 100 may vary at least one of a font size setting, a color setting, a brightness setting, and a color vision abnormality setting of the graphic user interface according to the variable setting information determined to compensate for the physical constraint.

Also, for example, the variable setting information may include a volume setting, a notification setting, a TTS (TEXT to SPEECH) setting, a STT (SPEECH to TEXT) setting, etc. that supplement the auditory constraint for the voice user interface. The terminal 100 may vary at least one of the volume setting of the voice user interface, the notification setting, the TTS (TEXT to SPEECH) setting, and the STT (SPEECH TO TEXT) setting according to the variable setting information determined to compensate for the physical constraint. have.

And, for example, the variable setting information may include setting an input range that supplements the input freedom constraint corresponding to the user interface. The terminal 100 may vary at least one of setting an input range, such as an input area for input of a user interface, a soft button size, and an interaction execution time, according to the variable setting information determined to compensate for the physical constraint.

In addition, the terminal 100 may collect and analyze terminal activity information indicating the user's cognitive ability and responsiveness in order to determine such physical constraints and variable setting information, and the collection of such activity information is performed at a predetermined period or before use. Since it is performed naturally according to the measurement of the terminal 100 test performed and the analysis of general activities such as user input when using the contents of the terminal 100, users do not specifically recognize their physical constraints, diseases, changes due to aging, etc. Even if it is not possible or individual detailed settings of the interface are not directly performed, a user interface that is naturally supplemented according to a predetermined test or activity can be experienced, making it more convenient to use a user interface-based terminal such as a smartphone or tablet.

The terminal 100 according to this embodiment of the present invention includes a wireless communication unit 110 , an A/V (Audio/Video) input unit 120 , a user input unit 130 , a sensor unit 140 , and an output unit 150 . , a memory 160 , an interface unit 170 , a control unit 180 , a power supply unit 145 , and a variable interface service unit 190 , and the like. Since the components shown in FIG. 1 are not essential, a terminal having more or fewer components may be implemented.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the terminal 100 and the wireless communication system or between the terminal 100 and the network in which the guardian terminal or service providing device is located. For example, the wireless communication unit 110 may include a broadcast reception module 111 , a mobile communication module 112 , a wireless Internet module 113 , a short-range communication module 114 , and a location information module 115 . .

The broadcast reception module 111 receives a broadcast signal and/or broadcast-related information from an external broadcast management server through a broadcast channel, and the mobile communication module 112 includes at least one of a base station, an external terminal, and a server on a mobile communication network. and transmits and receives wireless signals. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text/multimedia message, and the server may include a payment server or a store management server.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built-in or external to the terminal 100 . As wireless Internet technologies, wireless LAN (WLAN) (Wi-Fi), wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), etc. may be used.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, etc. may be used as short range communication technologies.

The location information module 115 is a module for acquiring location information of the portable terminal, and basically a global positioning system (GPS) module may be included.

Meanwhile, referring to FIG. 1 , the A/V (Audio/Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122 . The camera 121 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit 151 .

The image frame processed by the camera 121 may be stored in the storage 160 or transmitted to the outside through the wireless communication unit 110 . Two or more cameras 121 may be provided according to the use environment.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, and the like, and processes it as electrical voice data.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may include a keypad, a dome switch, a touch pad (static pressure/capacitance), a jog wheel, a jog switch, and the like. Also, the camera 121 may be connected to the user input unit 130 .

The sensor unit 140 detects the current state of the terminal 100, such as the opening/closing state of the terminal 100, the position of the terminal 100, the presence or absence of user contact, the orientation of the terminal, acceleration/deceleration of the terminal, and the like. It generates a sensing signal to control the operation of a person or stores it as activity information of a senior. The sensor unit 140 may include a proximity sensor 141 .

Also, the sensor unit 140 may include, for example, an altitude sensor, an acceleration sensor, a geomagnetic sensor, a gyro sensor, a barometric pressure sensor, and the like, to detect a movement of the current position according to the movement of the terminal 100 .

The output unit 150 is for generating an output related to visual, auditory or tactile sense, which includes a display unit 151 , a sound output module 152 , an alarm unit 153 , and a haptic module 154 . can

The display unit 151 displays (outputs) information processed by the terminal 100 . For example, in the content providing mode, the terminal 100 displays a user interface (UI, User Interface, or GUI, Graphic User Interface) related to content service provision.

The display unit 151 is a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display (flexible). display) and at least one of a three-dimensional display (3D display).

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the storage unit 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 also outputs a sound signal related to a function performed by the terminal 100 . The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event according to the operation of the notification service of the terminal 100 . The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. The video signal or audio signal may also be output through the display unit 151 or the audio output module 152 , and thus they may be classified as a part of the alarm unit 153 .

The haptic module 154 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 are controllable. For example, different vibrations may be synthesized and output or may be output sequentially.

The memory 160 may store a program for the operation of the controller 180 and may temporarily store input/output data. The memory 160 may include a flash memory type, a hard disk type, a multimedia card micro type, and a card type storage unit (eg, an SD or XD storage unit, etc.). ), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), Magnetic It may include at least one type of storage medium among a storage unit, a magnetic disk, and an optical disk. The terminal 100 may operate in relation to a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a passage with all external devices connected to the terminal 100 . The interface unit 170 receives data from an external device, receives power and transmits it to each component inside the terminal 100 , or allows data inside the terminal 100 to be transmitted to an external device. For example, wired/wireless headset port, external charger port, wired/wireless data port, memory card port, port for connecting devices equipped with identification module, audio input/output (I/O) port , a video I/O (Input/Output) port, an earphone port, etc. may be included in the interface unit 170 .

The controller 180 generally controls the overall operation of the portable terminal. For example, it performs related control and processing for variable interface service, voice call, data communication, video call, and the like. The controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented within the controller 180 or may be implemented separately from the controller 180 .

The power supply unit 175 receives external power and internal power under the control of the controller 180 to supply power required for operation of each component.

In addition, the controller 180 according to an embodiment of the present invention may generate user interface variable setting information according to the driving of the variable interface service unit 190, and display the user interface to which the user interface variable setting information is applied on the display unit 151 ) can be printed. To this end, the controller 180 may execute a process according to the operation of the variable interface service unit 190 and process a user interface creation function. This will be described in more detail with reference to FIG. 2 .

2 is a block diagram illustrating in more detail the variable interface service unit 190 of the terminal according to an embodiment of the present invention.

Referring to FIG. 2 , the variable interface service unit 190 according to an embodiment of the present invention includes a service providing unit 191 , a setting information management unit 193 , an activity information analysis unit 195 , and a physical restriction information processing unit 197 . ), and a variable interface setting unit 199 .

First, the service providing unit 191 may provide various content services and variable interface services utilizing the unique functions of the terminal. As described above, the content service includes a video call service with a guardian terminal, a message conversation service, a photo album service, a dementia prevention program service, a health information management service, a medication management service, an activity report service, a music service, a radio service, and a video service. , Internet service, game service, song service, etc. may be exemplified, and a user with physical limitations such as a senior may perform various activities through a content service using the terminal 100 .

Also, the service providing unit 191 may provide a message service, and the message service may include a conversation message service and a notification message service. The chat message service may include a service for transmitting and outputting a chat message or a call message with a guardian terminal or a service providing device.

To provide such a service, the service providing unit 191 obtains the user interface setting information set in the setting information management unit 193, generates a user interface to which the obtained setting information is applied, and outputs it to the output unit 150. have.

The setting information management unit 193 may receive setting information in advance and provide it to the service providing unit 191 , or may receive the variable setting information determined by the variable interface setting unit 199 and store and manage it as user setting information. For example, the setting information may be input from at least one of the user input unit 130, the guardian terminal, or the service providing server, and may include at least one of a visual setting for a user interface, a sound setting, and an input freedom setting. have.

In order to configure the setting information, the activity information analysis unit 195 may collect and analyze the user's activity information for processing the physical restriction information of the physical restriction information processing unit 197 . Here, the activity information includes user input pattern information input through the user input unit 130 with respect to the content service provided by the terminal 100, sensor information sensed by the sensor unit 140, body input information, location information, etc. 100) may include a variety of measurable activity information.

In addition, the activity information analysis unit 195 may transmit the activity information to the service providing server 200 and process it to be stored and managed by the service providing server 200 . For example, short-term activity information within a predetermined period may be stored and managed by the activity information analysis unit 195 , and long-term activity information greater than or equal to a predetermined period may be transmitted to the service providing apparatus 200 to be stored and managed.

Accordingly, the activity information analysis unit 195 collects various activity information and transmits it to the physical restriction information processing unit 197, or processes a separate statistical analysis to provide data for providing content services such as disease prevention to the service providing unit (191) can be forwarded. For example, the service providing unit 191 processes a notification message for recommending specific content according to the content use record of the activity information analysis unit 195, processes a health care notification message according to body measurement, etc., or talks for a certain period of time. It is also possible to process a notification message that recommends conversation with family members who have not been there, or a notification message that recommends using a learning service to prevent dementia.

In addition, user activity information according to each physical constraint is collected and analyzed to be indexed, trend analysis is performed based on the indexing information for each period, and the trend analysis result is statistically analyzed and processed to guide the prevention of diseases corresponding to the physical restriction. service may be provided. For trend identification and statistical processing by analyzing activity information, a database may be used, or a disease prevention guide may be provided using an artificial intelligence learning method.

In addition, a visual test interface is provided for each content including text, image, and video for determining visual interface setting information, and variable setting information for a user interface for each content is provided based on a user input to the visual test interface for each content. By determining the variable setting information, it is possible to output a variable interface applied to the user interface for each content.

Similarly, even when the auditory test interface is provided, the auditory test may be provided for each content, the A1us setting information may be determined according to the user input for each content, and the variable setting information for each content may be applied to the user interface and output.

In addition, the physical restriction information processing unit 197 may acquire physical restriction information according to the activity information of the user analyzed by the activity information analysis unit 195 , and the obtained physical restriction information is transmitted to the variable interface setting unit 199 . can be transmitted.

Here, the physical restriction information processing unit 197 may acquire physical restriction information for setting a variable interface according to one or more preset methods. For example, the one or more methods may include a test method and a usage log analysis method.

First, the test method is described. The physical constraint information processing unit 197 performs a separate user input test before the user's terminal 100 is used, or a separate event occurrence condition based on a predetermined cycle or activity information (eg, For example, a user input test may be performed according to a specific interface or a case in which a button erroneous input occurs at a predetermined frequency or more).

In addition, the physical restriction information processing unit 197 may determine the physical restriction information from the activity information collected and analyzed by the activity information analysis unit 195 according to a user input test.

More specifically, the physical constraint information processing unit 197 may provide a test interface for the physical constraint test, and determine the physical constraint information according to a user input corresponding to the test interface.

The physical constraint may be classified into, for example, a visual constraint, an auditory constraint, and an input freedom constraint, and a test for each classified attribute may be individually performed.

For example, the test interface may include a visual test interface for testing at least one of font size, screen brightness, and color vision abnormality of the user interface.

In addition, the test interface may include an auditory test interface for testing a voice output corresponding to the user interface.

In addition, the test interface may include an input freedom test interface for testing the input freedom corresponding to the user interface.

Here, the input freedom measurable on a test basis is a physical button input freedom corresponding to one or more physical button input units provided in the terminal, or a touch input freedom corresponding to a soft button output through the output unit 150 of the terminal. At least one of the drawings may be included.

On the other hand, if the usage log analysis method is described, the physical restriction information processing unit 197 receives the activity information accumulated and analyzed by the activity information analysis unit 195 according to the user's use of the content service of the terminal 100 without a separate test. Based on the physical constraint information may be determined.

More specifically, for example, in the activity information of the elderly or the disabled who are not free to use their hands and are restricted, a touch interaction input for a specific interface situation may be shifted in a predetermined pattern or may be incorrectly input.

Accordingly, the physical constraint information processing unit 197 according to an embodiment of the present invention includes the user's touch interaction information included in the activity information analyzed by the activity information analyzing unit 195 and an interface state mapped to the touch interaction information. Based on the information, the physical constraint information may be determined.

In particular, the analysis of the user's touch interaction information and the interface state information mapped to the touch interaction information may include a LOG analysis indicating the user's usage record. The purpose of this log analysis technique is to identify the user's original intention for each input failure or erroneous input situation, and to provide a more optimized interface accordingly.

For example, the usage log information collected by the activity information analyzer 195 may include tap information, drag information, and swipe information as touch interaction information. In addition, the usage log information may be exemplified as interface status information, such as screen situation information, interface type information, usage time zone information, and the like.

The variable setting information of the variable interface setting unit 199 determined according to this may include, for example, a variable setting of a soft button size, a variable extension or adjustment of a button recognition area, a change of an interaction reference time for determining button execution, and the like.

In this way, the variable interface setting unit 199 may determine the variable target object identification information and the variable value of the user interface that supplement the physical restriction information processed by the physical restriction information processing unit 197, and the determined variable target object identification information and the variable value is transmitted to the service providing unit 191 so that the user interface to which the variable interface setting is applied is output through the output unit 150 .

For example, the variable setting information that supplements the user's physical constraint may include visual, auditory, or input freedom variable setting information that complements the physical constraint determined by performing the above-described test.

In addition, the variable setting information to compensate for the user's physical constraint may include touch area extension information for compensating for the user's soft button selection input failure determined from the touch interaction information and the interface state information.

In addition, the variable setting information to compensate for the physical constraint of the user may include graphic handler extension information for compensating for the user's graphic handler control input failure determined from the touch interaction information and the interface state information.

In addition, the variable setting information to supplement the physical constraint of the user may include touch duration setting information for compensating for the user's touch continuation impairment determined from the touch interaction information and the interface state information.

According to the test execution or usage log analysis of the physical constraint information processing unit 197, the variable setting information of the variable interface setting unit 199 is determined, and the variable setting information is transmitted to the setting information management unit 193, and the service providing unit ( 191), users who are naturally supplemented even if they do not specifically recognize their physical constraints, diseases, or changes due to aging, or do not directly perform individual detailed settings of the interface. An interface can be set, and user convenience can be improved by providing a user interface-based terminal use supplemented or compensated for by physical limitations.

3 is a flowchart illustrating a method of operating a terminal according to an embodiment of the present invention.

Referring to FIG. 3 , the terminal 100 according to an embodiment of the present invention first outputs a user interface through the output unit 150 ( S100 ).

Here, the user interface is provided visually or aurally through the output unit 150 based on graphics, voice, or a combination thereof, and may receive and process a user's manipulation input or voice input to the user input unit 130 . An input/output interface may be exemplified.

Then, the terminal 100 analyzes one or more user inputs input in response to the user interface, and obtains physical restriction information of the terminal user ( S101 ).

Here, the user input analysis may be processed by the activity information analysis unit 195 , and the physical restriction information processing unit 197 may obtain the physical restriction information by processing the activity information obtained according to the user input analysis.

Then, the terminal 100 determines, through the variable interface setting unit 199, variable setting information that supplements the physical constraint of the user of the terminal 100 for the user interface based on the physical constraint information (S103). ).

Then, the terminal 100 outputs a variable interface to which the variable setting information is applied to the user interface through the output unit 150 (S105).

Then, the terminal 100 determines whether it is necessary to update the physical restriction information (S105).

Here, whether or not the physical constraint information needs to be updated may be determined according to various condition events. For example, the condition event may include a change of a user, the lapse of a test validity period, occurrence of a decrease of a threshold value or more of a specific cognitive ability according to activity information, and a change of more than a threshold value of visual, auditory or input freedom.

When it is necessary to update the physical constraint information, the terminal 100 performs again from step S101, and if not necessary, maintains the current setting information, and stores and manages the current setting information in the setting information management unit 193 .

4 to 8 are exemplary diagrams for explaining a test interface according to an embodiment of the present invention.

First, referring to FIG. 4 , the physical constraint information processing unit 197 according to an embodiment of the present invention is configured to test the degree of freedom of sight, hearing, or input according to the first use of the terminal 100 or according to a predetermined period or event occurrence condition. A test interface may be generated and output through the output unit 150 . As shown in FIG. 4 , the test interface may include an interface for selecting at least one of a visual test, an auditory test, and an input freedom test, and one or more tests selected according to the user's selection may be sequentially performed. have.

5 to 6 are diagrams for explaining a visual test interface according to an embodiment of the present invention. First, as shown in FIG. 5, the visual test interface according to an embodiment of the present invention selects a font size that is most convenient for a user to see. A selection menu including test text for each font size for selection may be included. By selecting the most visible text, the user can set the font size variable setting information to compensate for the visual body constraint.

Also, as shown in FIG. 6 , the visual test interface according to an embodiment of the present invention may include a color theme selection menu for selecting a screen color theme that is most convenient for a user to view. By selecting the most convenient color theme, the user can set the color theme variable setting information that complements the visual body constraint. Here, although a dark theme and a light theme are exemplified as the color theme, various additional color themes such as a red theme and a blue theme may be added according to the subdivision of the menu to be selectable.

Meanwhile, FIG. 7 exemplifies an auditory test interface according to an embodiment of the present invention. First, as shown in FIG. 7A , the auditory test interface according to an embodiment of the present invention provides a voice output that is most comfortable for the user to listen to. may include a sound selection menu for selection. The user may set the audio variable setting information that compensates for the auditory body constraint by selecting the audio output amount that is most comfortable for the user to listen to. However, if all sounds are not well heard, it is determined that there is a hearing impairment, and a button interface for entering the hearing aid setting menu may be further displayed at the bottom.

7(B) is an example of a hearing aid setting menu, and the hearing aid setting menu may include detailed setting menus that can be set when there is a large auditory physical constraint. The terminal 100 may generate an interface for additionally setting the detailed setting menus through the service providing unit 191 and output it through the output unit 150 . The hearing aid setting menu may include, for example, a notification sound switching setting, external sound detection notification setting, TTS conversion output setting according to external voice detection, STT conversion output setting according to text reception detection, and the like.

And, FIG. 8 illustrates an input freedom test interface according to an embodiment of the present invention. The input freedom test interface according to an embodiment of the present invention is a button input graphic interface that induces a user to input a physical button input. may include The user may press each physical button one after another in order and then select the completion button, and the terminal 100 may determine the input freedom of the physical button.

However, when the input freedom of the specific physical button is low, it is determined that the input of the physical button is difficult, and the generation setting of the soft button capable of a touch input may be performed. As illustrated in the lower part of FIG. 8 , a touch button creation menu for generating a soft button may be output through the output unit 150, and accordingly, touch button generation setting information that compensates for the input freedom constraint is a variable interface setting. It may be generated through the unit 199 and stored and managed in the setting information management unit 193 .

On the other hand, although only the process of determining the input freedom of the physical button is disclosed in FIG. 8, the input freedom of the soft button such as the touch button can also be tested, and for this, the test soft button and the user input corresponding to the test soft button are analyzed Accordingly, the extension direction and range of the touch area may be determined.

9 is a flowchart for explaining a method of operating a terminal according to another embodiment of the present invention, and FIGS. 10 to 12 are diagrams for explaining an activity information analysis method according to an embodiment of the present invention.

As described above, the physical constraint information processing unit 197 according to an embodiment of the present invention predicts the physical constraint condition only by analyzing the normal terminal 100 usage log according to the activity information analysis of the activity information analyzing unit 195, and , a user interface variable setting may be performed accordingly.

To this end, first, the physical constraint information processing unit 197 may determine a touch interaction according to the user log analysis of the activity information analysis unit 195 ( S201 ), and analyze the interface state information mapped to the touch interaction ( S203 ). .

Through such analysis processing, the physical constraint information processing unit 197 predicts the physical constraint condition according to the user intention analysis, and the variable interface setting unit 199 generates user interface variable setting information optimized in response to the predicted physical constraint condition. It can be determined (S205).

More specifically, the physical constraint information processing unit 197 according to an embodiment of the present invention predicts a physical constraint condition according to the user's intention to execute a button, and sets variable extension range and extension direction information of the touch area corresponding to the constraint condition. What can be done, this will be described in more detail with reference to FIG. 10 .

As shown in FIG. 10 , a touch position or area input by a user on a specific soft button may be formed to be different from a range that the user actually perceives according to a physical constraint according to input freedom.

Accordingly, the physical constraint information processing unit 197 according to an embodiment of the present invention divides the basic touch area into quadrant coordinate information, and determines the user's intention from the touch location of the usage log collected according to the user's activity information analysis. The input error information according to the input error information is statistically calculated, and a touch extension area (indicated by a dotted line) reflecting the calculated error information may be determined. For example, the error information may be exemplified as (+10 pixels, -10 pixels) as quadrant-based coordinate information, may be different for each individual, and may be updated according to a certain period.

In addition, the variable interface setting unit 199 may determine variable interface setting information for setting a region in which the determined extended region and the basic region are combined as a touchable activation region.

According to this setting, the terminal 100 can compensate for the user's freedom of input of the touch button without a separate user setting menu, thus enabling smooth execution of the touch button of the terminal 100 according to the user's intention. (100) It becomes possible to increase usability and usability.

Meanwhile, FIG. 11 shows that the physical constraint information processing unit 197 according to an embodiment of the present invention predicts a physical constraint condition according to the user's intention to execute the graphic handler, and determines whether or not the graphic handler is variably expanded and the extension range corresponding to the constraint condition. It is a drawing to explain the setting.

As shown in FIG. 11 , the user may fail to input a drag gesture input to a specific graphic handler at a certain frequency or more according to a physical constraint according to the input freedom.

For example, when the user's drag gesture is input several times in the activity information analysis unit 195 in the physical constraint information processing unit 197, but there are not many execution histories corresponding to the drag gesture, there is a restriction on the input freedom for the drag intention. existence can be confirmed.

Accordingly, the physical constraint information processing unit 197 according to an embodiment of the present invention determines constraint information of a user drag gesture on a specific graphic handler, and the determined constraint information is transmitted to the variable interface setting unit 199, and the variable interface The setting unit 199 may determine setting information for setting an additional handler 155 for assisting a user drag gesture on a specific graphic handler, and transmit it to the service providing unit 191 and the setting information managing unit 193 .

Accordingly, the terminal 100 can supplement the drag input freedom constraint for the graphic handler, thus enabling smooth control of the graphic handler of the terminal 100 according to the user's intention, the terminal 100 usability and usability can be improved.

And, FIG. 12 shows that the physical constraint information processing unit 197 according to an embodiment of the present invention predicts the physical constraint of the touch interaction time according to the user's touch input intention, and varies the touch interaction reference time corresponding to the constraint. It is a drawing for explaining that.

As shown in FIG. 12 , the user may not be able to easily perform a tap input in which a touch must be maintained for a short time or a long press input in which a touch must be maintained for a long time, depending on physical constraints according to input freedom. An adjusted reference time may be required to select a desired content or execute a menu according to the intention of the user.

Accordingly, the physical restriction information processing unit 197 obtains the user's touch residence time statistical information analyzed from the activity information analysis unit 195, and obtains physical restriction information corresponding to touch maintenance from the obtained residence time statistical information. In addition, the variable interface setting unit 199 may determine variable setting information capable of adjusting the touch holding reference time range in response to the physical constraint information.

Here, the touch time may be set in response to a tap and a long press, respectively, and the time may be set in a different range according to the user's physical constraint information. However, since physical constraints are caused by a decrease in cognitive ability, it is preferable that the normal touch reference time is increased than the basic time.

According to such a variable touch reference time setting, the terminal 100 can compensate for the restrictions on the degree of freedom of tap and long press input, and thus it is possible to make a tap input and long press input of the terminal 100 smoothly according to the user's intention. Bar, similarly, the usability and usability of the terminal 100 can be improved.

On the other hand, the above-described method according to various embodiments of the present invention is implemented in the form of installation data to be executed in a terminal device and stored in various non-transitory computer readable media to each server or device. may be provided. Accordingly, the user terminal 100 may access the server or device and download the installation data.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims In addition, various modifications may be made by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (5)

A method of operating a terminal providing a user interface, the method comprising:
a user activity information analysis step of collecting and analyzing user interface adjustment information including at least one of font size adjustment, screen brightness adjustment, and color adjustment input in response to the user interface in a predetermined period unit;
determining, based on the information obtained by analyzing the user activity information, variable setting information that supplements the visual constraint of the terminal user for the user interface; and
outputting a variable interface to which the variable setting information is applied to the user interface
A method of operating a terminal providing a user interface. According to claim 1,
providing a visual test interface for determining visual interface setting information; and
Further comprising the step of determining variable setting information that supplements the visual constraint according to a user input corresponding to the test interface
A method of operating a terminal providing a user interface. 4. The method of claim 3,
The test interface includes a visual test interface for testing at least one of font size, screen brightness, and color vision abnormality of the user interface.
A method of operating a terminal providing a user interface. According to claim 1,
indexing the user activity information and performing trend analysis based on indexing information for each period;
providing a guide service for preventing visual diseases by statistically analyzing the trend analysis result; further comprising
A method of operating a terminal providing a user interface. According to claim 1,
Classifying and providing a visual test interface for each content including text, an image, and a video for determining visual interface setting information;
determining variable setting information for a user interface for each content based on a user input to the visual test interface for each content;
outputting a variable interface in which the variable setting information is applied to the user interface for each content
A method of operating a terminal providing a user interface.

Images