KR101357946B1 - Computer image processing system for blind person - Google Patents

Abstract

The present invention discloses an image processing apparatus for computer screen recognition for the visually impaired.
The computer image processing apparatus according to the present invention has a voice guidance macro function for the visually impaired, and by removing a cumbersome input device, a USB connection structure to perform flag and play functions, free control of a mouse and keyboard, and high-speed data processing. A Human Interface Device (HID) driver having a; An image processing software module that implements an enlargement, reduction, and rotation function using Euclidean geometric transformation technology on a computer screen in response to starting the HID driver, and performs screen recognition on an RGB image using an index image technique; And voice processing software module based on 2,351 audio data including consonants and vowels, and guiding arbitrary information on the screen through analysis of mouse position and image data output on the monitor. It characterized in that it comprises a USB firmware consisting of.
Accordingly, the present invention provides a structure that can be visually impaired regardless of the computer system, portable to secure mobility, and based on the image processing filter for fast screen capture, screen information in real time By transmitting, there is an effect that can maximize the computer use ability of the disabled.

Description

COMPUTER IMAGE PROCESSING SYSTEM FOR BLIND PERSON}

The present invention relates to an image processing method for a computer user with a partial visual impairment, and more particularly, by using an index imaging technique for RGB images having a plurality of colors, to maximize the efficiency of screen recognition, The resolution and color value of the user can be set according to the user. By using the DLL injection technology, the user can capture and analyze and notify the images of all the application softwares that the user wants, so that all the changes in the monitor can be delivered to the blind in real time. The present invention relates to a computer image processing device for the visually impaired.

In general, computers are indispensable in living an information society, and how proficient and effective use of such computers is regarded as the ability of modern society to live in an information society. Therefore, in recent years, due to the development of OS programs such as Windows, which is not only highly trained to learn a computer, but also the development of an OS (operation system) program that connects a computer and a user, the computer is easier to learn and very simple to use. The computer population is growing exponentially.

However, despite the rapid development of computers and OS programs as described above, the visually impaired people cannot use computers with OS programs such as Windows. There is no problem.

In recent years, algorithms for the visually impaired have been developed to solve these problems. For example, when the user accesses the Internet by executing a web browser, various objects on the homepage, a list, the address of the homepage, the address input, and the homepage are used. Disclosed is a method of reading a general-purpose program screen for the visually impaired, which allows the visually impaired to read the information of the application program by voice according to the user's setting conditions, so that the visually impaired person can easily access the Internet and obtain various information. This will be described in detail with reference to FIG. 1 as follows.

First, the illustrated computer booting step (S100) is to turn on the computer, the OS program starting step (S110) is to run the OS program so that the user can easily use the computer is completed booting. The message hook setting step (S120) executes a program to set a message hook, and the message hooking step (S130) starts the message hooking when the message hook setting step (S120) is completed. When the message hooking step (S130) is completed as described above, the general program execution step (S140) is carried out by the user selectively executes the general program.

When the web browser, which is a general-purpose Internet program, is executed as described above, the address input step (S200) is performed. The address input step (S200) is to input the address of the server to be connected to the address input box on the general-purpose program, the address input field focus step (S210), the keyboard event generation step (S220), the keyboard spelling buffering step (S230), And transmitting the stored keyboard spell to the voice engine (S240) and outputting the keyboard spell to the voice (S250).

The address input field focusing step (S210) is to place a cursor in an address input box when the web browser is executed, and the keyboard event generating step (S220) allows a user to input a keyboard address to access an address input box where a cursor is located. The keyboard spelling buffering step (S230) is to store the spelling of the keyboard hit by the user in the buffer, and the step of transmitting the stored keyboard spelling to the voice engine (S240) to the keyboard spelling stored in the buffer to the voice engine And the keyboard spelling information transmitted to the voice engine as described above is outputted as a voice through an output means by outputting the keyboard spell (S250), so that the visually impaired person inputs a correct address. It can be checked immediately by so that the wrong input can be corrected immediately.

When the address input step (S200) is completed as described above, the server connection confirmation step (S150) is performed. The server access check step (S150) checks whether the server corresponding to the address input in the address input step (S200) is connected, and if not connected or inputs the address again during the connection, the address input step ( It is returned to (S200).

The document parsing step S160 parses the document contents so that the document on the home page accessed using the web browser can be output by the voice engine. The object existence check and object information voice output step (S300) is to check whether the object exists in the home page of the connected server, and when the object exists by outputting the object information by voice, the object presence check step (S310) and , An object information buffering step (S320), an object information transmitting step (S330), and an object information voice output step (S340).

The object existence checking step S310 checks whether an object exists in the homepage of the connected server, and if the object does not exist, returns to the document parsing step S160. The object information buffering step (S320) stores the information of the objects in the buffer when there is an object in the object existence checking step (S310), and the object information transmitting step (S330) for voice output of the object information stored in the buffer. It transmits to the voice engine, the object information voice output step (S340) outputs the object information transmitted to the voice engine as a voice through the output means.

The object information may include the number of frames on a page to be accessed, picture information and application components, a flashing picture or text or a table. In the object content reading order selection step (S400), by checking the objects with contents among the objects existing in the homepage and allowing the user to randomly order the objects with contents, the contents of each object are output by voice. The verification step (S410), the reading order selection step (S420), the object content buffering step (S430), the object content transmission step (S440), and the object content voice output step (S450).

The object content existence check step (S410) checks whether there is an object content when there is an object on the page parsed in the document parsing step (S160), and if there is no content, a message indicating that there is no content of the object. The voice is output to. In the reading order selection step (S420), the user selects the reading order of the object content when there is an object content in the object content checking step (S410), and the object content buffering step (S430) is the reading order selection step. In step S420, the contents of the objects selected in order are sequentially buffered according to the order, and the object contents transmission step S440 is transmitted to a voice engine to output the contents of the objects stored in the buffer to the voice engine. S450 outputs the object content transmitted to the voice engine as voice through the output means.

The link presence confirmation and information output step (S500) is to check whether there is a link on the page of the server connected to, and if there is a link to create the link purpose, and outputs the created link list, the link presence confirmation Step S510, link list creation step S520, link list buffering step S530, link list transmission step S540, and link list voice output step S550.

The link presence checking step (S510) checks whether there is a link on the page of the connected server, and if there is no link, return to the document parsing step (S160) or go to the frame presence check and output step (S600). do. The link list creation step (S520) creates a link list when there is a link in the link presence checking step (S510), and the link list buffering step (S530) creates a link list created in the link list creation step (S520). Store in a buffer, the link list transmission step (S540) is transmitted to the voice engine to output the link list stored in the buffer, the link list voice output step (S550) outputs the link list transmitted to the voice engine Sound is output through the means.

In the conventional method of reading the Internet web browser screen for the visually impaired person having the above configuration, the user inputs various information (object information, screen composition, picture information, application program information, etc.) existing on the accessed page from address input when accessing the Internet. Change to to print in detail. However, the above-described method of reading the web browser screen includes information provided on the web screen, for example, "transfer completed", "transmitting", "transmission failure," transmitted amount ", amount of data remaining", and "name" of the file. By converting text information such as "," location "," capacity ", creation date", etc. into voice data, it is impossible to actually recognize the screen, that is, office automation OS because it does not recognize the change on the screen. There are a lot of crowds to run.

1. Republic of Korea Patent Registration No. 10-0421566, Registration Date February 24, 2004, Application No. 10-2000-0058922, Application Date October 06, 2000

The present invention was created to solve the above problems, and an object of the present invention is to enable a visually impaired person to handle a desired task while viewing a screen as a normal person, so that the visually impaired person can promote employment of office occupations for the disabled. It is to provide a computer image processing apparatus for.

Another object of the present invention, the visually impaired can be handled without regard to the computer system, propose a portable structure to secure mobility, and based on the image processing filter for fast screen capture, screen information By delivering in real time, to provide a computer image processing device for the visually impaired that can maximize the computer ability of the disabled.

Still another object of the present invention is equipped with a virtual mouse and a virtual keyboard driver to recognize the characters on the screen through the function of voice guidance to enable a variety of information to send and receive mail, messenger, various documents, etc. In addition, the present invention provides a computer image processing apparatus for the visually impaired, which may be used by the visually impaired even in a task that is repeatedly performed or requires precision through macro setting and a shortcut key setting function.

Another object of the present invention is to provide a computer for the visually impaired who can activate global information sharing by disabling the desired screen to facilitate information access when a low vision person uses the Internet in a public institution or at home. An image processing apparatus is provided.

Computer image processing apparatus for the visually impaired according to an aspect of the present invention for achieving the above object, in the image processing apparatus for computer screen recognition for the visually impaired, has a voice guidance macro function for the visually impaired, cumbersome input A Human Interface Device (HID) driver having a USB connection structure to perform flag and play functions, free control of a mouse and keyboard, and high-speed data processing by erasing the device; An image processing software module that implements an enlargement, reduction, and rotation function using Euclidean geometric transformation technology on a computer screen in response to starting the HID driver, and performs screen recognition on an RGB image using an index image technique; And voice processing software module based on 2,351 audio data including consonants and vowels, and guiding arbitrary information on the screen through analysis of mouse position and image data output on the monitor. It characterized in that it comprises a USB firmware consisting of.

USB firmware according to an embodiment of the present invention, the USB communication module connected to the user computer capable of data exchange in accordance with the USB communication protocol; A virtual mouse / virtual keyboard driver for processing complex tasks using a simple macro setting without an input signal of an actual disabled person using a device; A macro setting module for performing voice output on messenger or email information, including various program control; Short cut key setting module to facilitate data cleanup for a repetitive operation using a shortcut key; A partial magnification module in which an arbitrary screen is processed by a user based on a window source program using a StretchBlt function for resizing an original picture; An image processing library for recognizing a character or an image according to a 2D image recognition technique and for enlarging, reducing, or rotating a corresponding screen by using a Euclidean geometric transformation technique by interworking with the partial enlargement module; And a voice processing library for guiding various information by guiding with the virtual mouse / virtual keyboard driver and analyzing the position of the mouse and the image data output to the monitor.

As described above, the computer image processing apparatus for the visually impaired according to the present invention is capable of office work without regard to the computer system, and suggests a portable structure for securing mobility, and speed of screen capture Based on the image processing filter for, by delivering the screen information in real time, there is an effect that can maximize the computer use ability of the disabled. In addition, in the present invention, when a low-sight person uses the Internet in a public institution or at home, the desired screen is partially enlarged to facilitate access to information, thereby enabling global information sharing for the disabled to improve the quality of life of the disabled. It has an effect.

1 is a flowchart illustrating a conventional method for reading a screen of an Internet web browser for the visually impaired.
2 is a block diagram showing a computer image processing apparatus for the visually impaired according to the present invention.
3 is a flowchart for explaining an operation procedure of the HID driver according to the present invention.
4 is a flowchart for explaining a screen recognition algorithm according to the present invention.
5 is a diagram illustrating experimental data obtained by measuring a screen analysis time for image processing according to the present invention.
6 is a diagram illustrating a tab page for setting a macro according to the present invention.
FIG. 7 is a tab page for explaining the macro recording / playback function of FIG. 6.
FIG. 8 is a tab page for describing the remote control command-client & server function of FIG. 6.
9 is a screen showing a screen shot setting for screen recognition according to the present invention.
FIG. 10 is a diagram for describing screen recognition registration of FIG. 9.
11 is a view for explaining a method for comparing the recognition point according to the present invention.
12 is a view showing a condition setting window for recognizing a time variable according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, a computer image processing apparatus for the visually impaired according to the present invention is to enable a visually impaired person to process a desired task while viewing a screen as a normal person, and is a portable device for securing mobility so that office work can be performed at any time. . In addition, it is equipped with a virtual mouse and a virtual keyboard driver, it includes a function to recognize the characters on the screen to guide the voice, and the visually impaired can also be used for tasks that are repeatedly performed or require precision through the macro setting and hot key setting function Do it. In addition, the present invention includes a screen magnification function, and is designed to be able to provide upgrade service without additional cost by mounting firmware.

2 is a block diagram showing a computer image processing apparatus for the visually impaired presented in the present invention. It covers both hardware and software configurations, and can be used for all OSs such as WindowXP and Window7 without any limitations. The USB module enables flag and play functions to automatically use the device. In addition, the mouse and keyboard can be freely controlled without direct control, and high-speed data processing of 100 or more times per second is possible.

In other words, HID (Human Interface Device) has a voice guidance macro function for the visually impaired, and has a USB connection structure to perform flag and play functions, free control of a mouse and keyboard, and high-speed data processing by eliminating cumbersome input devices. It uses the driver and Euclid's geometric transformation technology to implement zooming in, zooming out, and rotation on the screen, and image processing software that performs screen recognition on RGB images using indexed image techniques. Based on a total of 2,351 audio data, including a voice analysis software to guide a variety of information through the analysis of the location of the mouse and the image data output to the monitor.

On the other hand, as shown, the USB firmware consisting of a hardware structure and software includes a USB communication module 221 that can be connected to the user computer to exchange data in accordance with the USB communication protocol, without the input signal of the actual disabled using the device Performs voice output for messenger or e-mail information, including virtual mouse driver 211 and virtual keyboard driver 213 for controlling complex tasks quickly and accurately with simple macro settings, and controlling (starting and closing) various programs. Based on a Windows source program that uses a macro setting module 215 to perform the data, and a shortcut setting module 217 to perform data cleanup for repetitive operations using a shortcut key, and a StretchBlt function for resizing the original picture. Partial magnification module 21 in which an arbitrary screen is processed by the user 9) and image processing for interworking with the partial magnification module 219 to recognize a character or an image according to a two-dimensional image recognition technique and to implement enlargement, reduction, and rotation of a corresponding screen using Euclidean geometric transformation technology. A voice processing library for guiding various information by guiding the library 221 and the virtual mouse driver 211 and the virtual keyboard driver 213 through analysis of the position of the mouse and the analysis of the image data output to the monitor ( 223).

Here, the hardware applied to the USB firmware is an AVR USB MCU that uses an AT90USB162 chip as an example of the present invention and receives a frequency clock of 16 MHz. It has 16K FLASH memory and uses a TQFP32 socket adapter board as a SMD type. EEPROM memory is 512 bytes, RAM memory is 512 bytes and it has a built-in USB controller.

Therefore, a program (firmware) is stored and executed using the internal memory of the MCU, and each hardware is controlled through an input port or communicated with a computer through a USB port and a contactor. The USB controller allows the computer to recognize the USB device as a USB device. The USB controller recognizes the USB mouse as a firmware and controls the actual mouse through an event according to a command.

As described above, including the hardware structure and software is to prevent illegal copying, and to enhance security by periodically changing through firmware update, that is, changing the product's VenderID and ProductorID and USB authentication method. to be.

On the other hand, the HID (Human Interface Device) driver is a device that can maximize the convenience of the user in place of the existing cumbersome input device in the case of the voice guidance macro for the disabled, so that it can be used in all OS such as WindowXP and Window7 without limitation have. In addition, by using the flag and play function to automatically use the device, it is not affected by the operating system depending on the development environment, and eliminates the need to install additional driver files.

In addition, by allowing the mouse and the keyboard to be freely controlled without direct control, the device for the visually impaired leads to other complicated and various things to be handled with a simple operation of the user. Therefore, the user can control this to handle the actual input functions through the mouse and keyboard with a single button.

In addition, the human interface device (HID) driver is designed to enable high-speed data processing of 100 or more times per second, thereby increasing speed and accuracy. When processing data using a window message, this method can minimize delay time in data processing, prevent errors in input processing part, and minimize communication size to reduce the communication load to achieve maximum performance. To make it possible.

3 is a flowchart for explaining an operation procedure of the HID driver according to the present invention. As shown, in step S301 the USB stick is mounted on the computer, and the connection with the computer is made through the USB communication module 221. In step S303, a HID keyboard and mouse virtual driver is generated, which is a driver generation step for performing operations of the virtual mouse driver 211 and the virtual keyboard driver 213.

The device is initialized for communication in step S305, the device is opened through step S307, and then the device enters step S309 to perform stick authentication (VID, PID) for security, and in step S311 the mouse and keyboard Controlled. That is, the operation of the virtual mouse driver 211 and the virtual keyboard driver 213 is performed. This performs an operation according to the preset shortcut key setting module 217 through the virtual keyboard driver 213 or performs a macro function by the macro setting module 215.

After the control is completed through the step S313, the device is closed. This procedure is based on flag and play gestures and recognizes pressing, soldering, clicking, double-clicking, dragging, wheeling, etc. for the keyboard and mouse.

On the other hand, the image processing software is to quickly and accurately capture and analyze the user's behavior patterns and movements on the screen to guide the data, and performs screen recognition for all the application software running on the user computer. To do this, DLL injection technology is used to capture and analyze the images of any application software that the user desires, and to perform screen recognition of game and video playback applications.

The DLL injection inserts a specific DLL file into another process (a program currently running on the computer), and instructs another process to call the API function LoadLibary () to load the DLL I want. The DLL injection according to the present invention transmits and receives data using a pipe communication technique, where pipe communication is a first-in, first-out queue form written by a process and read by another process.

The image processing software is performed based on the operation of the image processing library 221. The image processing library 221 recognizes a character or an image using a 2D image recognition technique and enlarges a screen using a Euclidean geometric transformation technique. Implement, reduce, rotate, etc. In addition, filtering is applied for 100 [fps] image processing per second to process accurate color values. That is, an index image technique is used in which an RGB image having 16.9 million kinds of colors has index information in a two-dimensional array and stores color values corresponding to the information in a color map.

Accordingly, the image processing library 221 recognizes images and text using different algorithms, and determines whether the images are the same by comparing them with the screen captured based on data set by the user. It compares the shape of each icon in the task bar to perform a simple function such as checking the execution and status of the program, and performing precise work such as inspection for product defect inspection. For example, when the network is disconnected, the corresponding icon is changed to the disconnected state, so that a quick action is possible by guiding the user after detecting the change of state.

Text, on the other hand, is obtained by comparing the string captured with the font. This method can recognize texts in various areas, from character strings used in pictures and videos to image fonts used in games. For example, the contents of the Internet, an email, a message, and the like are texted and guided by voice.

In addition, the image processing library 221 automatically adjusts the range of color values according to the screen resolution of the user computer, so that the visually impaired people use the computer under various conditions according to their environment. Therefore, the screen resolution and color value are automatically set in the program to maximize the efficiency of screen recognition, the most important function for them.

Here, the screen recognition algorithm according to the present invention will increase the screen recognition speed very much, the screen recognition algorithm is shown in FIG. As shown, the thread is created in step S401, and enters step S403 to monitor the start of the target process. Thereafter, when the process is executed in step S405, DLL injection is performed, and in step S407, the screen is captured and stored in the memory. Color information is collected through step S409, and filtering is performed to improve speed in step S411. In step S413, the color analysis value is returned.

According to such a procedure, the image processing method of the present invention can process about 10 times faster than the screen capture method that is generally used. That is, the general screen analysis method uses the basic API GetPixel function, which has a large delay time in the processing speed part, which causes a problem that cannot be used in a device for the visually impaired device that requires real-time data analysis. The image processing library 221 overcomes the speed delay according to the general screen analysis described above, thereby minimizing unnecessary time when processing the change value of the screen.

Therefore, all changes in the situation that occur on the monitor can be delivered to the visually impaired in real time to provide a feeling of working with the eyes, and by removing unnecessary parts of the change value of the screen, about 30% It is possible to improve speed.

5 is experimental data obtained by measuring a screen analysis time for image processing according to the present invention.

This experiment is an internet explorer screen recognition speed test to compare the speed difference between the general screen recognition and the screen recognition method according to the present invention. The test method uses different methods on the same computer and checks the time from screen recognition to data processing. (Repeat 10 times each time and check the average of 20 times except the fastest 5 times and the slowest 5 results after 30 tests in total) Calculation).

Test environment, CPU: Intel (R) Core (TM) i3 CPU M339 @ 2.13GHz, OS: Microsoft Windows XP Home Edition K Service Pack 3, RAM: 2.0GB, GPU: NVIDIA GeForce 310M, Resolution: 1366 * 768 ( 16Bit), and the measured results show that the screen recognition method according to the present invention shows a speed improvement of approximately 10 times compared to the general screen recognition method.

On the other hand, the speech recognition program proposed in the present invention is operated based on the speech processing library 223, the speech processing library 223 is a variety of information through the analysis of the position of the mouse and the image data output to the monitor Voice guidance allows the visually impaired to feel as if they are looking at and controlling the monitor. The voice processing library 223 is produced based on Korean characters and is produced based on a total of 2,351 voice data including consonants and vowels.

That is, after checking the current mouse pointer position and the set mouse pointer position, if it matches with each other, it plays the pre-produced Hangul voice guidance data by one character unit. Therefore, the voice guidance for each text is made, the visually impaired people have a feeling to see and control the monitor.

6 shows a tab page for setting a macro according to the present invention, which is presented as an embodiment of the present invention. Therefore, the structure and shape of the tab page provided in the present invention will not limit the technical spirit of the present invention. 6A is an action edit macro. After setting 10 action edit macros, the start key is pressed to execute the action edit macro stored. That is, in order to create an action edit macro, after confirming a preset value by pressing a file open, if a key is set and executed, it is moved by the preset value.

6A, when the Create Action Edit Macro button is clicked, it is provided on the screen as shown in FIG. 6B, and includes a keyboard operation, a mouse button operation, a mouse movement operation, and a time delay operation. You can set the Save, Open, and New actions. First, the keyboard operation may set a key press, a key release, and a key press and release.

The section repetition is used when a specific section is repeated a predetermined number of times. An operation setting is performed by inserting a start number and an end number from a data number on the left side of the figure and inputting a repetition number. The file storage is a function of storing a currently registered action list, and the file open is a function of recalling a stored action list. In addition, the new function is a button used to create one action list and another action list, and when the button is pressed, all data registered to create new data is deleted.

FIG. 6C is a page for starting the "combination macro" function located at the top of FIG. 6A. Five combination macros are set via a key, and when the set key is pressed, the stored combination macro is executed. That is, the combination macro can be executed by loading the value set by the combination macro creation located at the bottom of the screen through the file open button and executing the key setting.

FIG. 7 is a tab page provided when a user clicks the "macro recording / playback" button located at the top of FIG. 6A. This page records 10 macros to set keys, and when the set key is pressed, the recorded data is played back. 'Recording / playing status' on the upper left of the screen is a button for selecting the recording / playing status, and 'option management' is a button for selecting and recording a keyboard or mouse. In other words, if you want to record only the keyboard, click the keyboard record button. If you want to record only the mouse, click the mouse button. In addition, the 'play count' located at the upper right of the screen is a button for setting the number of times the macro is played.

8 is a tab screen obtained by clicking "Remote Control Command-Client & Server" located at the top of FIG. 6A. This pops up in the upper part of the main screen and is executed in connection with the server by selecting the access button after entering the IP of the server. It enters the server's IP in the Client IP blank and connects to it. From this, the remote control command can be sent from the server PC to the client PC.

9 is a screen illustrating a screen shot setting for screen recognition according to the present invention, and a user may set a normal screen shot or a precision screen shot through a 'capture method' of the screen. Here, the precision screen shot is a normal screen shot and is used when a screen shot is not taken where desired, and the caption is set accordingly. If you select the precision capture method, the caption name part on the right side of the screen is activated and enter the caption name in the corresponding position.

After setting the normal method and the precision method, the recognition point operation is performed. The method of creating the recognition point is to open a program, set a screenshot key, and press the key to take a screenshot. It is automatically saved in the set folder.

Then, open the file of the recognition point setting section and load the screen shot in the previously saved folder. When the desired position is clicked on with a mouse as shown in FIG. 10, the click position becomes a recognition point, and a recognition name and a recognition number are set. The recognition name is a name value for the position clicked with the mouse, and the recognition number is arbitrarily assigned with a recognition number of 1 to 200. Therefore, when the desired identification number is set, the registration window is raised. When the identification number is clicked on the registration window, the position clicked with the mouse can be checked.

The above-described recognition point comparison is to compare the recognition point in operation on the screen with the recognition point taken by the screen shot. As shown in FIG. 11, the position point comparison, the center point comparison, the center point comparison, or the average value are compared. The method can be used. That is, when comparing positions, it is determined whether two or more points are similar by comparing N1-P1 to N9-P9. Alternatively, the center point and the comparison method may be used to determine whether a desired color exists by comparing N1-P5, N2-P5, N3-P5 ... N9-P5. In addition, center point comparison is the simplest comparison method by comparing N5-P5, and average value comparison is comparing the average value of N1-N9, and the average value of P1-P9.

12 illustrates a condition setting window for recognizing a time variable. As shown, time setting is performed by checking a check box in item '1', inputting time in seconds from item '2', and setting a comparison condition through item '3'. The comparison condition can be selected as 'small', 'large', 'greater than or equal', etc., and set the time in '4'. Set the time from 1 to 10 hours. Item '5' is for setting the larger comparison condition. You can change the setting to 'not compare', 'small', 'equal', 'small or equal', 'different', etc. The item '6' is a part of inputting the larger time by comparing the conditions, and is described in seconds. Item '7' is a button for deleting the set value. As a result, the present invention performs screen recognition according to the above-described method, and greatly increases the speed according to the screen recognition.

211: virtual mouse driver 213: virtual keyboard driver
215: macro setting module 217: shortcut key setting module
219: loupe expansion module 221: USB communication module
221: image processing library 223: sound processing library

Claims (6)

In the computer image processing device for the visually impaired to which the USB firmware is applied,
The USB firmware may include a USB communication module connected to a user computer and capable of exchanging data according to a USB communication protocol;
A virtual mouse / virtual keyboard driver for performing a macro function without an input signal of an actual disabled person using a device;
A macro setting module for performing voice output on messenger or email information, including various program control;
A partial magnification module in which an arbitrary screen is processed by a user based on a window source program using a StretchBlt function for resizing an original picture;
An image processing library for recognizing a character or an image according to a 2D image recognition technique and for enlarging, reducing, or rotating a corresponding screen by using a Euclidean geometric transformation technique by interworking with the partial enlargement module; And
Computer image processing for the visually impaired, comprising a voice processing library for guiding various information by guiding the virtual mouse / virtual keyboard driver and analyzing the mouse position and the image data output to the monitor. Device. The method of claim 1,
The image processing library recognizes images and text using different algorithms, and determines whether the images are identical to the screen captured based on data set by the user, and uses fonts for the text. Computer image processing apparatus for the visually impaired, characterized in that to recognize the data by comparing with the captured character string. The method of claim 1,
The USB firmware communicates with a computer through a USB port, recognizes the USB firmware as a USB mouse, and controls an actual mouse through an event according to a command;
And a VenderID, a ProductorID, and USB authentication information of a product, wherein the computer image processing apparatus for the visually impaired is maintained by changing the information. The method of claim 1,
The image processing library is a computer image processing apparatus for the visually impaired, characterized in that to perform screen recognition of the video playback application by capturing and analyzing the image using a DLL injection. 5. The method of claim 4,
The DLL injection inserts a specific DLL file into another process, instructs another process to call LoadLibary (), which is an API function, and then loads a desired DLL. The method of claim 5, wherein
The DLL injection is a computer image processing apparatus for the visually impaired, characterized in that to perform data communication using a pipe communication technique.

Images