KR20140049525A - System and method for displaying visual information based on haptic display for blind person - Google Patents

Abstract

The present invention relates to a system and a method, both for displaying visual information based on a haptic display for a visually impaired person so that learning material including visual data along with texts is converted to tactile information to be provided. Both the system and the method of the present invention comprise the steps of: extracting an image part and a text part from general learning material, wherein the image part and text part for each piece of information mixed in the general learning material are extracted; making image information tactile by degree of complexity, wherein the extracted image part is analyzed so as to separate it into a low complex image and a high complex image to treat each image by degree of complexity, and converts each image into tactile information; and recognizing texts and converting them into Braille, wherein text information in the text part made tactile and text information in the image part made tactile are respectively recognized to be converted into Braille information. Therefore, people who are visually impaired can utilize various types of special learning material, and many resources that are required to make things tactile can be reduced. [Reference numerals] (10) Learning material input unit; (110) Image/text part extracting unit; (120) Unit for making images tactile by degree of complexity; (130) Unit for recognizing texts and converting them into Braille; (20) Tactile information display unit

Description

TECHNICAL FIELD The present invention relates to a haptic display based visual information display system and method for a visually impaired person,

The present invention relates to a haptic display-based visual information display system and method for a visually impaired person, and more particularly, to a visual information display system and method for displaying visual information in a general education material displayed through a multimedia device, The present invention relates to a haptic display-based haptic information conversion and presentation technology for a visually impaired person, which enables a visually impaired person to realistically accommodate educational materials provided through the haptic display.

Recently, access to digital multimedia information has become common due to the spread of various electronic and portable devices such as computers, mobile communication terminals, PDAs, and PMPs. These conventional devices are designed based on non-disabled persons without physical difficulties Therefore, it was difficult for disabled people to access these multimedia information.

Accordingly, a variety of braille output devices capable of providing tactile information to the visually impaired, or an output device providing tactile information by applying haptic technology, have been commercialized.

On the other hand, current general education materials include a large amount of visual information that can not be expressed as braille, such as photographs, illustrations, cartoons, diagrams, and graphs. The content of general textbooks is often based on the use of visual materials, and it helps to understand the contents of the text more fully by presenting additional visual information. However, in the special education data, such visual information can not be converted into Braille, and most of it is deleted. On the other hand, in converting to tactile information, many resources such as manpower and time are required.

Recently, as people's interest in private education for education, employment, separation or certification has increased, they have been registered in private institutes such as entrance examination institute, private tutoring institute, certificate institute and English institute, There are increasing numbers of learners who study language in academies, but visually impaired people do not want to receive education such as certification and language because of communication and difficulty in walking.

Therefore, it is necessary to develop a technology that enables the blind people to provide educational information using multimedia data using tactile information.

The technical problem to be solved by the present invention is to convert the educational material including the visual material together with the text into the tactile information so as to provide it as a special education material similar to the general education material, The present invention provides a haptic display-based visual information display system and method for a visually impaired person who can give an opportunity to utilize special education materials similar in level to a haptic display.

According to an aspect of the present invention, there is provided a method for extracting a character region and an image region from general education data, A complexity-based image information hapticization step of separating a low-complexity image and a high-complexity image and converting each image into a tactile information by processing each image according to the complexity, and recognizing character information in the tactile- And a braille conversion step of converting the braille information into braille information.

In the visual information display method according to the present invention, the image region and character region extraction step in the general education data classify each information in the general education data into a character region other than the image region, a character region in the image region, and an image region .

In the visual information display method according to the present invention, the image region and character region extraction step in the general education data may include a step of extracting a brightness image and a label for analyzing character and visual data of the general education data, A general education data input step of generating base information, an image region extraction step of extracting the image region by classifying the character region and the image region based on the characteristics of the character and visual data appearing in the generated brightness image, And a character extracting step of detecting a character area in the image area.

In the visual information display method according to the present invention, the general education data input step may include generating gray level information on all pixels of the scanned general education data to generate base information for a brightness image, And generating base information for the label through binarization and labeling of the base information for the brightness image.

In the time information display method according to the present invention, the base information generation step of the label may include a binarization processing step of reconstructing the generated brightness image into only black pixels and white pixels, a connection state between adjacent black pixels of the binarized brightness image, And judges the pixels connected to each other as a single entity to distinguish at least characters classified as entities (minimum units constituting a character) and visual data classified as one or more entities according to presence or absence of white papers And a labeling processing step.

In the time information display method according to the present invention, the image region extracting step may calculate the change amount (G) with respect to the brightness value between neighboring pixels of the character and the visual data in the brightness image of the inputted general education data, Generating a GD (Difference Difference) image by only a change amount (G) with respect to a brightness value between neighboring pixels of the calculated character and the visual data; Calculating an average width ( LW _average ) of labels of characters with the outlier of the label for the visual data, using equation (2), determining the _average value of the labels of the characters width (LW _average) as the mathematical image for MGD one step, the reconstruction to reconstruct the maximum change amount difference (MGD, maximum difference Griadient) image by applying equation (3) for the reference image GD Through 4 determining the boundary value (Tb) to perform binarization and obtain a whitened bovine width associated with each row plus the two characters wide (2 LW _average) than the characters to be expected within MGD image by removing the white pixel of the small width And a morphology operation of dilate and erode is performed on the MGD image in which the candidate region of the predicted character region is determined. The morphology operation of the dilation and the erode is performed in the MGD image, The merging of the character regions and the labels of the data calculated in the input step of the general education data are checked to see if they are included in the character region of the candidate group, And classifying the region into regions,

Equation 1

Equation 2

Equation 3

Equation 4

In Equation 1 to 4, x, y are the location of the rows and columns of pixels, w, h are the width and height, (x, y) B of the image is the brightness value of the corresponding pixel locations, G (x, y) is Q ₁ and Q ₃ are the first quartile and third quartile for the width of the label, LW and NL are the label width and the total number of labels, respectively, and T _b is the boundary for binarization Value.

In the time information display method according to the present invention, when the label of the data calculated in the input step of the general education data is included in the character area of the candidate group, the text area and the image area classification step are determined as labels for the characters, If not, it is determined as a label for the image area, and character and visual data in the general education data are classified into a character area and an image area, respectively.

In the time information display method according to the present invention, the step of extracting characters in the video region may include determining whether the characters exist in the video region determined in the video region extracting step, The search range is limited to only the image region under the same environment (the average width of the label of characters, the filter of the morphology operation), and the character region in the image region is extracted as a result.

In the temporal information display method according to the present invention, the tactile information of the image information according to the complexity is obtained by analyzing a cumulative histogram of the brightness of the calculated image region, and classifying the image into a low-complexity image and a high- A step of discriminating and extracting a center object of a high-complexity image that determines whether or not the classified high-complexity image includes a main object, extracting a main object, and simplifying and reducing the complexity of the image to help the visually impaired And noise cancellation and simplification of high complexity images.

In the time information display method according to the present invention, the image classification step may include calculating an average cumulative value of brightness for an image through a conditional expression 1, calculating an average cumulative value of the calculated brightness by a conditional expression 2 (I = 0, cnt = 1, 2, 3, 4, 5, 6, 7, 8, 0, sum = 0; i <256; i ++) {if ((Q 1 -1.5 (Q 3 - Q 1) <= NH i _{<= (Q 3 +1.5 (Q} 3 - Q 1)) then cnt ++, sum + = NH i;} NH average = sum / cnt; and, wherein the conditional expression 2, for (i = 0; i <256 _{; i ++) NH i - =} NH average; and, each calculation formula NH _i is the accumulation value for the brightness, NH _average is an average accumulated value of the brightness of an image, Q ₁ and Q ₃ are of the cumulative value of the brightness of each 1 quartile and 3 quartiles.

In the time information display method according to the present invention, the image segmentation step may include a step of dividing the change rate ( D _NH ) with respect to the remaining interval of the brightness cumulative value of the image before and after eliminating the average accumulated value of brightness, And classifying the low-complexity image and the high-complexity image according to the inspection result,

Equation 5

In Equation (5), the threshold value ( D _th ) for the change rate ( D _NH ) for classifying the high-complexity and low-complexity images is a value that is frequently displayed in the general education data including figures, graphs, illustrations, Cnt _before and cnt _after are the residuals of the brightness before and after removing the cumulative value, and I is the image area.

In the method of displaying time information according to the present invention, the step of discriminating and extracting the center object of the high-complexity image measures the color similarity of the pixels constituting the high-complexity image and quantizes similar colors as representative colors A step of determining the presence or absence of a center object by measuring a distribution of colors corresponding to the center object area in the image converted into the representative color, and a step of extracting a contour line of the center object when the center object exists / RTI &gt;

In the method of displaying time information according to the present invention, the step of converting similar colors to representative colors includes the steps of: determining a quantization level using a PGF (Peer Group Filtering) technique; And reducing the complexity with respect to the color of the high-complexity image.

In the method of displaying time information according to the present invention, the step of discriminating the presence of a center object may be performed by assuming that a color having the highest frequency is centered on a central region of an image defined in Equation (6) Calculating a distribution in an image by calculating a variance with respect to a direction and a horizontal direction, and determining the presence or absence of a center object according to the calculated distribution map,

Equation 6

In Equation (6), w and h are respectively the width and height of the high-complexity image, and AreaX _obj and AreaY _obj are the ranges of the x and y axes of the central region, respectively.

In the method of displaying time information according to the present invention, in the step of determining whether a central object is present, a case in which colors representative of a central object are horizontally widely distributed or a distribution is low through a conditional expression formula 3, And in the other cases, it is determined that there is a central object,

Conditional expression 3

s _max = max ( s _verticality , s _{horizontality} )

if (( s _max == s _{horizontality} ) && ( s _max > T ₁ )) ( s _max < T ₂ )

then I is a non-object image

otherwise I is an object image

( T ₁ = 0.06, T ₂ = 0.01)

In the conditional expression 3, s _verticality And s _{horizontality} is the maximum vertical and maximum horizontal variance, s _max for the representative color of the central object in the high complexity image ( I ) s _verticality and s _{horizontality,} among which the maximum variance and the maximum variance, T ₁ and T _{2, which} are constants, are optimal test values calculated through repeated experiments on various high-complexity images.

In the method of displaying time information according to the present invention, the extraction of the contour of the center object extracts the representative colors of the background region for the four corners of the image defined by Equation (7) in the quantized high- Removing the colors from the entire image to leave only the information about the center object in the image; performing binarization and labeling on the high-complexity image to remove color information appearing outside the center area, and generating labels for the representative color of the center object; And extracting the center object of the image to be converted into the tactile information by detecting the contour of the center object left in the high-complexity image by checking whether the position for each of the generated labels is included in the center area, Comprising the steps of:

Equation 7

In Equation 7, AreaX AreaY _n and _n is the background area, w and h for each of the four corners of the image is characterized in that the width and height of the high complexity image.

In the method of displaying time information according to the present invention, the step of removing and simplifying noise of a high-complexity image includes the steps of calculating a size of a label for contours constituting an image by performing binarization and labeling on a high-complexity image, When the ratio of width to height is calculated by comparing the resolution of the image and the resolution of the haptic display, and the respective labels are reduced according to the resolution of the haptic display through Equation (8), if the size of the pixels is less than one cell of the haptic display Classifying the image information of the reduced label into low-importance noise and removing the image information,

Equation 8

In Equation (8), H _width and H _height are the _width and _height of the haptic display, I _width and I _height are the width and _height of the image, and T _w and T _h are the width and height of the image LW , LH and n are the width, height and total number of the label, and S is the size of the pixel when the label is converted into the haptic display resolution.

In the method of displaying time information according to the present invention, noise removal and simplification of a high-complexity image may include detecting corner points of contour lines to reduce complexity of the detected contour lines and creating connection lines using the corner points, And a step of removing a corner point shared by the connecting line when the intersecting angle is greater than the critical angle and lowering the complexity of the contour of the image by creating a new connecting line at the remaining two points, The critical angle is an optimal test value measured by performing an experiment on various images having high contour complexity.

In the time information display method according to the present invention, the character recognition and the braille conversion step may be performed by applying optical character recognition (OCR) to the extracted character area, converting the information into computer-recognizable information, And converting it into a braille form recognizable to a person with a disability.

In the visual information display method according to the present invention, the character recognition and the braille conversion step may be realized by performing optical character recognition on a text area of a paragraph or a paragraph excluding visual data in general education materials, A character recognizing step of generating character information that can be performed by the user, performing optical character recognition on the characters included in the visual material in the general education data, and generating computer-recognizable character information from the character area in the image area And character recognition in the image area outside the generated image area and character information in the image area is performed in the braille expression step of recognizing the character to be displayed as braille information which can be recognized by the blind people through the braille expression algorithm .

In the braille display step of the recognized character in the time information display method according to the present invention, the braille length is calculated to represent the braille information in the haptic display from character information other than the video area, And performing line wrapping in accordance with the result of the comparison.

In the method of displaying time information according to the present invention, the step of expressing the braille of the recognized character is performed by inspecting whether or not the braille information, which is obtained from the character information in the image area, is superimposed on the surrounding image information after the braille information is arranged in the image area, And arranging an index corresponding to the braille information in the video area and outputting braille information with the index outside the video area.

In the visual information display method according to the present invention, the braille expression step of the recognized character is performed in accordance with the Korean Braille character rule and the English Braille character rule, and is composed of concatenation words, acronyms, basic arithmetic operation symbols, And &lt; / RTI &gt;

According to the present invention, it becomes possible to provide special education materials at a level similar to general education materials by converting education materials including visual characters into text information into tactile information. Therefore, it is possible to provide various kinds of special education materials It is advantageous in that a large amount of resources required for generating the existing touch can be saved and a large amount of data can be easily generated and provided to the visually impaired.

1 is a block diagram schematically illustrating an overall configuration of a haptic display based visual information display system for a visually impaired person according to the present invention.
FIG. 2 is a schematic operation flowchart illustrating an overall processing procedure of a haptic display based visual information display method for a visually impaired person according to the present invention.
3 is a detailed flowchart of the image region extracting step of FIG.
FIGS. 4A and 4B are detailed flowcharts of the step of discriminating and extracting the center object and the noise elimination and simplification steps of the high-complexity image of FIG.
FIGS. 5A to 5D are diagrams for explaining an operation of separating an image region and a character region in general education data. FIG. 5A is an example of general education data, FIG. (C) is a result of extracting pixel information of a specific row in the image of (b), converting the brightness value of the pixel for each column of the row into a histogram, and (d) Is the result of converting the difference in the amount of change with respect to the brightness value of (c) into a histogram.
6 (a) and 6 (b) are reference views illustrating cumulative histograms of brightness of a low-complexity image and a high-complexity image, respectively.
FIGS. 7 to 9 are reference views illustrating an image region and character extraction step in general education data according to the present invention.
FIG. 10 is a reference diagram for explaining the tactualization step of an image according to the present invention.
FIG. 11 is a reference diagram for explaining a noise reduction and simplification step of a high-complexity image according to the present invention.
FIG. 12 is a reference diagram for explaining a character recognition step excluding an image area in the character recognition and extraction step according to the present invention.
13 is a reference diagram illustrating character recognition and indexing steps in an image region in the character recognition and extraction step according to the present invention.
14 is a reference diagram illustrating the final result of transforming general education data into tactile information through the present invention.

Hereinafter, a configuration of a haptic display based visual information display system for a visually impaired person according to an embodiment of the present invention and an operation flow of the method will be described in detail with reference to the accompanying drawings.

It is to be understood that the words or words used in the present specification and claims are not to be construed in a conventional or dictionary sense and that the inventor can properly define the concept of a term in order to describe its invention in the best possible way And should be construed in light of the meanings and concepts consistent with the technical idea of the present invention. Therefore, it should be understood that the embodiments described herein and the configurations shown in the drawings are only the most preferred embodiments of the present invention, and that various equivalents and modifications may be substituted for them at the time of the present application shall.

1 is a block diagram schematically illustrating the overall configuration of a haptic display-based visual information display system for the visually impaired according to the present invention, the tactile information conversion system 100 of the present invention, each mixed in the general education material Image / text area extraction unit 110 for extracting a text area and image area for information, and a complex image tactile unit 120 for converting a low complexity image and a high complexity image into tactile information by analyzing the extracted image area. And character recognition in the extracted character area and the character information in the image area, and the character recognition and the point area unit 130 for converting into braille information that can be recognized by the visually impaired. Education data input unit 10 for, may be configured to further include a tactile information display unit 20 for expressing the braille.

FIG. 2 is a schematic flowchart illustrating an entire process of a haptic display-based visual information display method for a visually impaired person according to the present invention. FIGS. 3 and 4A and 4B illustrate an image region extracting step a2 of FIG. And a step (b2) for discriminating and extracting a center object for a high-complexity image and a noise elimination and simplification step (b3), respectively.

As illustrated in FIG. 2, the method of the present invention includes a step of extracting an image region and a character region (a100) in a general education data, a step of haptening an image information by a complexity (b100), and a step of character recognition and braille transformation (c100) .

The step of extracting the image region and the character region in the general education data extracting step (a100) is a step of extracting the character region and the image region for each information mixed in the general education data, (A1), an image region extraction step (a2), and a character region extraction step (a3) in the image region.

In the general education data input step (a1), brightness information and label information are generated for character and visual characteristics of the general education data. The base information for the brightness image is generated by performing gray scale processing on all pixels of the scanned general education data, and the base information for the label is generated by binarizing and labeling the base information for the generated brightness image Lt; / RTI &gt; The base information generation step for the label is performed through binarization and labeling processing. The generated brightness image is reconstructed using only black pixels (pixel value: 0) and white pixels (pixel value: 255) through binarization processing, A process of checking the connection state between adjacent black pixels of the binarized brightness image through the process, determining the pixels connected to each other as one entity, and comparing the characters classified into entities of at least a character (minimum unit) It distinguishes visual data classified as one or more objects according to the presence or absence of white papers. The brightness image and label generated at this stage are used as input data for the image region extraction process and the image character extraction process.

In the image region extraction step (a2), the image region is extracted by classifying the character region and the image region on the basis of the characteristics of the characters and the visual data appearing in the generated brightness image. As shown in the operation flow chart of FIG. 3, the image region extracting step a2 is a step of calculating a change amount G for a brightness value a21, a difference amount difference (GD) image creating step a22, for the average width (LW _average) calculation step (a23), the maximum change amount difference (MGD) image reconstruction step (a24), candidate determining step (a25), the merge phase in scattered character areas MGD image within estimated character region MGD image (a26), and a character region and a video region classification step (a27).

In the step of calculating the amount of change (G) with respect to the brightness value, a change amount (G, Gradient) with respect to the brightness value between neighboring pixels of the character and the visual data in the brightness image of the inputted general education data is calculated by the following equation Lt; / RTI &gt;

In Equation 1, x, y are the location of the rows and columns of pixels, w, h are the width and height, B (x, y) of the image brightness values, G (x, y) of the pixel location is adjacent Represents the amount of change for the pixel.

In the GD image generation step (a22), a GD (Gradient Difference) image is generated only by the amount of change (G) with respect to the brightness value between neighboring pixels of the calculated character and the visual data.

In the average width ( LW _average ) calculation step (a23) of the label of the character, the label of the generated visual data is determined as an outlier in order to determine the candidate group of the character region expected in the generated GD image The average width ( LW _average ) with respect to the label of the character is calculated by the following equation (2).

In Equation (2), Q ₁ and Q ₃ denote the first quartile and the third quartile for the width of the label, and LW and NL denote the width of the label and the total number of labels, respectively.

In the maximum change amount difference (MGD) image reconstruction step (a24), the following equation (3) is applied to the GD image on the basis of the average width ( LW _average ) for the labels of the calculated characters, Difference) reconstruct the image.

The variables in the above Equation (3) are the same as those in Equation (1).

In the determination step (a25) of the candidate region of the character region expected in the MGD image, the boundary value (Tb) is determined by the following Equation (4) for the reconstructed MGD image and binarization is performed. ( 2 LW _average ), and determines the candidate region of the expected character region in the MGD image.

The variables of Equation (4) are the same as those of Equations (1) and (2), and T _b represents a boundary value for performing binarization.

In the step (a26) of merging the character regions scattered in the MGD image, a morphology operation of dilate and erode is performed on the MGD image in which the candidate region of the predicted character region is determined, And character regions scattered by the spacing are merged.

In the character area and the image area classification step (a27), it is checked whether or not the labels of the data calculated in the input step of the general education data are included in the character area of the candidate group, Are classified into a character area and an image area, respectively. In this classification step, if the label of the data calculated in the input step of the general education data is included in the character area of the candidate group, the label is determined as a label for the character. If the label is not included in the candidate group, Classify text and visual materials in educational materials into character area and image area, respectively.

In the character region extracting step (a3), it is determined whether or not a character exists in the image region determined in the image region extracting step. When there is a character, the character region is extracted as a character region. The average width of the label of the image, and the filter of the morphology operation), the search range is limited to only the image region, and the character region in the image region is extracted as a result.

In the hapticization step (b100) of the complexity, the extracted image region is analyzed to distinguish low complexity images including graphics and graphs, high complexity images including illustrations and photographs, (B1), a step of discriminating and extracting the center object of the high-complexity image (b2), and a step of removing and simplifying the noise of the high-complexity image (b3) .

In the image classification step b1, the cumulative histogram of the brightness of the calculated image region is analyzed to classify the image into low complexity images (graphics and graphs) and high-complexity images (illustrations, cartoons, maps, and photographs) according to the complexity .

For this purpose, in the image classification step (b1), first, an average cumulative value of brightness for an image is calculated by the following conditional expression (1), and the average cumulative value of the calculated brightness is calculated by the following conditional expression Eliminates the phenomenon that the cumulative value of brightness in the low-complexity image is subtracted from each cumulative brightness value and appears in successive intervals.

The conditional expression (1)

for (i = 0, cnt = 0, sum = 0; i <256; i ++) {if ((Q 1 -1.5 (Q 3 - Q 1) <= NH i _{A,; - <= (Q 3} +1.5 (Q 3 Q 1)) then cnt ++, sum + = NH i;} NH average = sum / cnt

The conditional expression (2)

* for (i = 0; i <256; i ++) NH _i - = NH _average ;

In the above equations, NH _i is an accumulated value for the brightness, NH _average is an average cumulative value of brightness for an image, and Q ₁ and Q ₃ are a first quartile and a third quartile for the cumulative value of brightness, respectively.

Next, in the image classification step (b1), the rate of change ( D _NH ) with respect to the remaining interval of the brightness cumulative value of the image before and after eliminating the average cumulative value of the brightness is examined through the following equation (5) And low-complexity images and high-complexity images.

In Equation (5), the threshold value ( D _th ) for the change rate ( D _NH ) for classifying the high-complexity and low-complexity images is a value that is frequently displayed in the general education data including figures, graphs, illustrations, Cnt _before and cnt _after are the residuals of the brightness before and after removing the cumulative value, and I is the image area.

In the step (b2) of discriminating and extracting the center object of the high-complexity image, it is determined whether or not the classified high-complexity image includes the main object and extracted. As shown in the operation flow chart of FIG. 4A, the step b2 of discriminating and extracting the center object of the high-complexity image includes a representative color conversion step b21 of similar colors, a presence / absence discrimination step b22 of center objects, And an outline extraction step b23 for the object.

The representative color conversion step b21 of similar colors measures the color similarity of pixels constituting a high complexity image and converts similar colors into representative colors through quantization. In this step, similar colors are converted into representative colors The quantization level is first determined using a Peer Group Filtering (PGF) technique. Next, quantization is performed according to the determined level to convert similar colors to representative colors, thereby reducing the complexity with respect to colors of the high-complexity image.

In the step b22, the presence or absence of a central object is determined by measuring a distribution of colors corresponding to the central object region in the high-complexity image converted into the representative color, and in this step, It is assumed that the color having the highest frequency in the central region of the image defined in Equation 6 is a representative color of the central object, and the distribution in the vertical direction and the horizontal direction is calculated to calculate the distribution in the image. Or the like.

In Equation (6), w and h are the width and height of the high complexity image, respectively, and AreaX _obj and AreaY _obj are the ranges of the x and y axes of the central region, respectively.

Here, the determination of presence or absence of a central object is performed by the following conditional expression 3. The color representing the central object is horizontally wide or low in distribution, such as a feature appearing in background images such as forest or sea, It is determined that there is no central object, and in other cases, it is determined that there is a central object.

The conditional expression (3)

s _max = max ( s _verticality , s _{horizontality} )

if (( s _max == s _{horizontality} ) && ( s _max > T ₁ )) ( s _max < T ₂ )

then I is a non-object image

otherwise I is an object image

( T ₁ = 0.06, T ₂ = 0.01)

In the conditional expression 3, s _verticality And s _{horizontality} is the maximum vertical and maximum horizontal variance, s _max for the representative color of the central object in the high complexity image ( I ) s _verticality and s _{horizontality} , the threshold values T ₁ and T _{2, which} are compared with the maximum variance and the maximum variance, are constants, which are optimal values calculated through repeated experiments on various high-complexity images.

The contour extraction step b23 of the center object extracts the contour of the center object when the center object exists as a result of the determination of whether the center object exists. In this step, first, in the quantized high-complexity image, The representative colors of the background region for the four corner regions of the image defined by the extracted regions are extracted and the extracted colors are removed from the entire image. This leaves a representative color for the center object in the high complexity image.

In the equation 7 AreaX AreaY _n and _n denotes a background area of the four corners of each image, w and h represent the width and height of the high complexity image.

However, at this time, the same color information may remain in an area other than the center area. Thus, the color information appearing outside the center area can be determined as noise, not information representing the object. Therefore, in order to remove this, in this step, binarization and labeling are performed on the high-complexity image to remove color information appearing in areas other than the central area, labels for representative colors of the central object are generated, And extracts the center object of the image to be converted into the tactile information by detecting the contour of the center object remaining in the high-complexity image.

In the noise reduction and simplification step (b3), the complexity of the image is lowered and simplified in order to assist the visually impaired. This noise elimination and simplification step b3 includes the steps of calculating the label size for the contours b31, removing the noise b32, creating the connecting line b33, and delineating the contour line as illustrated in the operation flow chart of Fig. And a step (b34) for lowering the degree of complexity.

In the label size calculation step b31 of the outlines, binarization and labeling processing are performed on the high-complexity image to calculate the size of the label for the outlines constituting the image.

In the noise removal step (b32), when the ratio of the width and the height is calculated by comparing the resolution of the image and the resolution of the haptic display, and the respective labels are reduced according to the resolution of the haptic display through the equation (8) The image information of the label which is reduced to one cell or less of the haptic display is classified into low-importance noise and removed.

In Equation (8), H _width and H _height are the _width and _height of the haptic display, I _width and I _height are the width and _height of the image, and T _w and T _h are the width and height of the image LW , LH, and n represent the width, height, and total number of the label, and S represents the size of the pixel when the label is converted to the haptic display resolution.

In the connection line creation step (b33), the corner points of the contours are detected to lower the complexity of the detected contours, and connection lines connecting the corner points are generated.

In the complexity lowering step (b34) for the contour line, the crossing angle between the connecting lines is checked. When the crossing angle is greater than the critical angle (preferably 150 degrees), the corner point shared by the connecting line is removed, And the complexity of the contour of the image is lowered. At this time, the critical angle (150 degrees) is an optimal experimental value that can be measured by performing experiments on various images having high complexity of contours.

The character recognition and braille conversion step c100 is a step of analyzing the character information in the extracted character area and the character information in the image area and converting the character information into braille information recognizable by the blind person, Optional Character Recognition) can be applied to convert it into computer-recognizable information, and then convert it into braille, which can be recognized by the visually impaired. The character recognition and braille conversion step c100 includes a character recognition step c1 outside the image area, a character recognition step c2 in the image area, and a braille rendering step c3 of the recognized character.

In the character recognizing step (c1) outside the video area, optical character recognition is performed on a character area of a paragraph or a paragraph except for the visual data in the general education data to generate computer-recognizable character information from the image information on the character area .

In the character recognizing step (c2) in the video region, optical character recognition is performed on the characters included in the visual data in the general education data, thereby generating computer-recognizable character information from the character region in the video region.

In the step (c3) of recognizing the recognized character, the blind person can recognize the character information outside the generated image area and the character information in the image area through the braille expression algorithm conforming to Korean Braille Specification and English Braille Specification Braille information is displayed in braille. Such braille expression algorithms preferably include brackets for conjunctions, abbreviations, basic arithmetic symbols, and general symbols. Particularly, in this braille expression step, braille information obtained from character information outside the image area is firstly calculated by calculating the length of the braille to express it on the haptic display, performing line wrapping according to the width of the haptic display resolution, Output. In addition, for the braille information to be transferred from character information in the image area, first, the braille information is arranged in the image area, and then it is inspected whether or not the braille information overlaps with the surrounding image information. An index corresponding to the braille information is placed in the corresponding image area, and braille information is output along with the index outside the image area.

The overall operation of the present invention constructed as described above, and the operation and effects thereof will be described below.

First, in the video / character region extracting unit 110, in order to convert the text information and the time information mixed in the general education data inputted through the training data input unit 10 into tactile information, a character region and an image region (A100) of extracting general education data and a character area extracting step (a100) in the general education data extracting step a100 in the general education data extracting step a1, a2), and a character extraction step (a3) in the video region are sequentially executed.

In the general education data input step (a1), the base information such as the brightness image and the label is generated in order to analyze characteristics of the character and visual data of the general education data. First, in the case of a brightness image, a gray scale process can be performed on all the pixels of the scanned general education data. Next, a label for analyzing characteristics of character and visual data can be generated by performing labeling processing after binarizing the brightness image. At this time, the brightness image is reconstructed using only the black pixel (pixel value: 0) and the white pixel (pixel value: 255) through the binarization process, and the connected state of adjacent black pixels is checked through the labeling process, As shown in FIG. Through this, the characters are classified into at least units (minimum units) and the visual data are classified into one or more objects according to the presence or absence of white papers. The brightness image and label generated in this step are used as input data of the image region extraction process and the image character extraction process.

In the image region extraction step (a2), the characteristics of the brightness between the character and the visual data are analyzed, and the general education data inputted using the area of the label calculated through the preceding process is classified into the character area and the image area.

FIGS. 5A to 5D are diagrams for explaining an operation of separating an image region and a character region in general education data. FIG. 5A is an example of general education data, FIG. (C) is a result of extracting pixel information of a specific row in the image of (b), converting the brightness value of the pixel for each column of the row into a histogram, and (d) Shows the result of converting the difference in the amount of change with respect to the brightness value of (c) into a histogram. At this time, in the pixels constituting the character, it is seen that a brightness value close to 0 (a black pixel and a black printed character) intersects with a brightness value (white pixel, blank space) close to 255, In the constituent pixels, the brightness value changes continuously. In order to determine this more clearly, when the histogram of (d) representing the difference in the amount of change with respect to the brightness value of (c) is generated, the difference between the brightness values of the pixels constituting the character and the visual data is very different . Therefore, a change amount (G) for a neighboring pixel is calculated using Equation (1) by using the feature of the brightness value of the character and the visual data, and the change amount difference (GD) image is reconstructed using only the change amount.

In the region where the characters in the generated GD image exist, the amount of change in brightness between the pixels is high, so that white pixels are crowded. In the margins representing the space between characters and in margins other than the printing region, Results appear. And, in the case of visual data, the whole brightness of the image is lowered, and the appearance of the gray scale image is enhanced.

Next, the boundaries between the characters in the paragraph or paragraph are removed to determine the candidate group of the character region expected in the generated GD image. For this, the width of the average character is calculated using the label generated in the previous step under the same conditions as in Equation (2), and the maximum variation amount difference image (MGD) is calculated by applying Equation (3) Lt; / RTI &gt;

Generally, labels generated from general education materials are mostly letters, labels for visual materials are relatively large in size, while frequencies are very small. Therefore, by using Equation (2), it is possible to calculate the average width ( LW _average ) with respect to the label of the character after excluding the label for the visual data as an outlier, and assigning the calculated LW _average to the expression Thereby generating an MGD image.

In order to more clearly classify the character region with respect to the MGD image reconstructed from the GD image, binarization is performed through determination of the threshold value of Equation (4), and then the width of white pixels connected to each row is obtained, ( 2 LW _average ). This is in order to eliminate the noise of the scanning process of the data and the image area by using the property that the character does not exist alone.

The morphology operation of dilate and erode is performed in order to merge character regions scattered by the glyphs and spacing in the MGD image that determines the candidate region of the last expected character region. At this time, the filter size of the morphology operation is set to be equal to the LW _average of Equation (2), and the character region can be clearly determined through the result of this morphology operation. Therefore, it is checked whether or not the label of the data calculated in the input process of the general education data is included in the determined character area, and when it is included, it is determined as a label for the character, do. Finally, it is possible to classify text and visual materials in general education materials into character area and image area, respectively.

In the character extraction step (a3), the presence or absence of a character in the image area determined in the image area extraction process is determined, and when there is a character, the character area is extracted. This process is performed by limiting the search range to only the image area under the same environment (average width of characters, filter of morphology operation, etc.) as the image region extraction process, and as a result, character region in the image can be extracted.

Next, the complexity-based image hunting unit 120 divides the extracted image region into a low-complexity image and a high-complexity image to determine whether or not the main object is included. The complexity-based image hunting unit 120 processes the complexity- (Graphics and graphs) and high-complexity images (such as illustrations, cartoons, and maps) according to the degree of complexity in the tactile information visualization step (step b100) (B2) for determining whether a high-complexity image includes a main object, extracting and extracting the high-complexity image, and (b2) A noise removal and simplification step (b3) of the high-complexity image that reduces the complexity of the image is executed sequentially.

In the image classification step (b1), a low-complexity image and a high-complexity image are classified by analyzing a cumulative histogram of the brightness of the image region.

6 (a) and 6 (b) are reference views illustrating cumulative histograms of brightness of a low-complexity image and a high-complexity image, respectively. As illustrated in the figure, a low- Brightness, and the cumulative value of the brightness appears only in certain intervals. On the other hand, the high-complexity image as shown in (b) consists of various brightness, and the cumulative value of the brightness appears in a continuous interval. Therefore, the image is classified by calculating the frequency of the interval in which the brightness value appears based on these characteristics. In this case, even in the case of the low-complexity image, there are minute brightness values and the brightness values appear in the continuous interval like the high- . Accordingly, the cumulative value of the average brightness is calculated through the conditional equations 1 and 2, and the brightness value of the low-complexity image is removed from each brightness value.

Next, the change rate with respect to the remaining interval of the brightness value before and after removing the cumulative value of the average brightness is examined as shown in Equation (5), and classified into a low-complexity image and a high-complexity image.

In the step (b2) of discriminating and extracting the center object of the high-complexity image, color similarity of pixels constituting the high-complexity image is measured, and similar colors are converted into representative colors through quantization. Then, the distribution of the color corresponding to the center object area is measured in the representative color image, and the presence or absence of the center object is determined, and when the center object exists, the outline is extracted.

In order to quantize the high complexity image which transforms similar colors into the representative color, the quantization level is first determined using the Peer Group Filtering (PGF) technique in the step b2 of discriminating and extracting the center object of the high complexity image By performing quantization according to the determined level, similar colors can be converted into colors representing them to reduce complexity with respect to colors of high-complexity images.

Next, in the step b2 of discriminating and extracting the center object of the high-complexity image, it is assumed that the color having the highest frequency in the central region of the image defined in Equation (6) is the representative color of the center object, And the degree of distribution in the image is calculated. At this time, if the color representative of the central object is horizontally wide or if the distribution is low, it is judged that the image has no central object. This is because it is a feature that appears in background images such as forests and seas. In other cases, it is determined that there is a central object, and such an operation can be performed through the conditional expression 3.

Finally, in the step (b2) of discriminating and extracting the center object of the high complexity image, the representative colors of the background region are extracted from the quantized high-complexity image, and the extracted colors are removed from the entire image. In this case, the background region of the image is defined as a section with respect to four corners of the image as shown in Equation (7). By doing this, only the representative color for the center object remains in the image. In this case, the same color information may remain in an area other than the center area. Thus, color information appearing outside the center area can be determined as noise, not information representing an object, and binarization and labeling are performed to remove the noise. From the result of performing binarization and labeling, it is possible to generate labels for the representative color of the central object, find its position, and check whether the position of each label is included in the center area to remove noise. As a result, only the information of the center object remains in the high-complexity image, and by detecting the outline of the center object, the center object of the image to be converted into the tactile information can be extracted.

In the noise removal and simplification step (b3) of the high-complexity image, in order to convert the high-complexity image into tactile information suitable for the resolution of the haptic display, binarization and labeling are first performed to calculate the size of the label for the contours constituting the image . When the ratio of width and height is calculated by comparing the resolution of the image and the resolution of the haptic display through Equation (8), and the labels are reduced in accordance with the resolution of the haptic display, when the label is converted into the haptic display resolution The size of the pixel S _i of the haptic display is reduced to one cell or less, it is determined that the importance of the image information of the corresponding label is low, and the image information is classified into noise and removed.

In addition, complicated contours in high complexity images make it difficult for the visually impaired to recognize the information. Therefore, in the noise removal and simplification step (b3) of the high-complexity image, the corner points of the contour lines are detected in order to lower the complexity of the contour lines, and a connection line is generated using the detected corner points. When the intersection angle between the connecting lines is more than the critical angle (150 degrees), the corner point shared by the connecting line is removed and a new connecting line is formed by the remaining two points. At this time, the set value for the critical angle (150 degrees) is the optimum value measured by performing experiments on various images having high contour complexity. As a result, corner points are removed based on a specific value to generate a new contour line for the image information, thereby reducing the complexity of the contour of the image, thereby helping the visually impaired to know the information.

The image / character region extraction unit 110 extracts a text region and an image region for each information to convert the text information and the visual information mixed in the general education data input through the training data input unit 10 into tactile information (A), the general education data input step (a1), the image area extraction step (a2), and the general education data input step ), And a character extraction step (a3) in the video region are sequentially executed.

Finally, the character recognizing and translating unit 130 applies optical character recognition (OCR) to the extracted character area (a character area other than the video area and a character area in the video area) (C100), which converts the text information into braille information that can be recognized by the visually impaired person. The character recognition and braille conversion step (c100) can be performed through the tactile information display unit (20) In the braille conversion step (c100), the character recognizing step (c1) outside the image area, the character recognizing step (c2) within the image area, and the braille rendering step (c3) of the recognized character are sequentially performed.

In the character recognizing step (c1) outside the video area, optical character recognition is performed on a character area such as a paragraph or a paragraph except for the visual data in the general education data, and the computer recognizes character information from the video information on the character area .

In the character recognition step c2 in the video region, optical character recognition is performed on the characters included in the visual data in the general education data as described above to generate computer-recognizable character information from the character region in the video region do.

In step (c3), the character information generated by the optical character recognition in the character area other than the video area and the character area in the video area is classified into 'Korean Braille Regulation' and 'English Braille Regulation' And braille information which can be perceived by the users of the visually impaired, and the connection words, abbreviations, basic arithmetic operation symbols and general symbols used by the visually handicapped persons are also brought together.

On the other hand, in general, a braille character is expressed by a unit of a character so that its length is longer than that of a character. Therefore, in this braille expression step (c3), braille information is calculated by calculating the length of the braille and converting the braille information to the width of the haptic display resolution in order to express the braille information from the character information outside the image area in the haptic display do.

Also, when braille is expressed in the image area, there is a risk that the braille overlaps with the image information, thereby lowering the perception rate of the blind. In order to solve this problem, in this step (c3), braille information, which is obtained from the character information in the image area, is placed in the image area, and then it is checked whether the braille information overlaps with surrounding image information. And outputs braille information along with the index to the outside of the image area.

FIGS. 7 to 9 are reference views for explaining the image region and character extraction step in the general education data. The visual information display method based on the haptic display for the visually impaired according to the present invention is applied to the general education data, As shown in Fig. In this experiment, the performance of the image region and character region extraction step (a), the image information hapticization step (b), and the character recognition and braille transformation step (c) To do this, we conducted three types of general education materials, which are composed of letters, visual materials (graphs, maps, photographs) and letters in visual materials, as shown in Figures 7 to 9 (a). (B) to (e) illustrate sequentially the steps (see Equations 1 to 4) for classifying the image region and the character region of the inputted training data, and (f) The results are illustrated. In this case, the blue label and the green label in the image represent a character region other than the image region and a character region included in the image region, respectively, and a red label represents an image region for visual data. The accuracy ( p ) and the recall rate (r) of the classification for the character region and the image region can be calculated by the following Equation (9) based on the extracted character.

In Equation (9), NDT (Number of Detected Texts) represents the total number of labels judged as characters, and TNCT represents the number of labels of correctly extracted characters in NDT . And TNT (Total Number of Texts) represents the total number of actual text objects in the data. Through this equation, the accuracy of the characters including graph, map, and photograph were 100%, 97.2%, and 90.9%, respectively, and the recall rate was 100% for all characters included in the label.

10 is a reference view illustrating the haptic step of each complexity image, FIG. 11 is a reference diagram illustrating a noise cancellation and simplification step of a high-complexity image, and FIG. a) shows a brightness image for a graph, a map and a photograph, respectively, and (b) illustrates a cumulative histogram of the brightness for the graph, the map and the photograph. The histogram of (c) can be generated by removing the average cumulative brightness by applying the conditional equations 1 and 2, and the change rate of the brightness value with respect to the remaining interval is 82.5%, 58.4%, and 0.4%, respectively appear. By substituting the calculated rate of change into Equation 5, the graph can be classified into a low-complexity image, and the map and photograph can be classified into a high-complexity image. (D) illustrate the result of transforming the classified image into tactile information. In this case, in the case of maps and photographs classified into high-complexity images, unlike the low complexity graph, Noise and simplification processes (Equations (6) to (8)) and Conditional Equation (3) are additionally performed as illustrated in Figs. 11A to 11D in order to transmit the image information.

12 is a reference view for explaining a character recognition step excluding an image area in a character recognition and extraction step, and FIG. 13 is a flowchart illustrating an example of a character recognition and extraction step in a character recognition and extraction step, 12 and 13 illustrate a result obtained by extracting computer-recognizable information by applying optical character recognition to extracted character regions, converting the information into braille information that can be perceived by a visually impaired person, have. 13, if braille characters do not overlap with the image information, braille in the image is represented. If the braille information overlaps with the image information, an index is displayed at the position of the braille, and an index And braille information separately.

Figure 14 illustrates the end result of transforming general education materials, including graphs, maps and photographs, into tactile information. This arrangement of tactile information prioritizes visual data, and if there is a character and an index in the visual data, related information is then output. The blue line in the resulting image is a boundary line indicating a page according to the resolution of the haptic display. As a result, the main information in the general education data is extracted through the image region and character region extraction step (a100), the complexity-based image information tactilation step (b100), and the character recognition and braille conversion step (c100) By converting the tactile information into tactile information that can be perceived by the visually impaired, it is possible to save a lot of resources required to generate the existing tactile and easily generate and provide a large amount of data to the visually impaired.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Modification is possible. Accordingly, it is intended that the scope of the invention be defined by the claims appended hereto, and that all equivalent or equivalent variations thereof fall within the scope of the present invention.

10: training data input unit 20: tactile information display unit
100: tactile information conversion system 110: image / character area extraction unit
120: an image hunting unit 130 for each complexity 130:

Claims (23)

(a100) extracting a character region and an image region for each information mixed in the general education data;
(b100) analyzing the extracted image region to distinguish a low-complexity image including a figure and a graph, a high-complexity image including an illustration and a photograph, and dividing each of the classified images into tactile A tactile sense step of converting the image information into information; And
(c100) analyzing character information in the extracted character area and character information in the video area, and converting the character information into braille information that can be recognized by a visually impaired person; and A haptic display based visual information display method. The method of claim 1, wherein the step (a100)
Wherein each piece of information in the general education data is classified into a character area other than the video area, a character area in the video area, and an image area, and the haptic display based visual information display method for the visually impaired. The method of claim 1, wherein the step (a100)
(a1) a general education data input step of generating brightness information for a character and visual characteristic of general education data and base information about a label;
(a2) an image region extracting step of extracting an image region by classifying a character region and an image region, respectively, based on characteristics of characters and visual data appearing in the generated brightness image; And
(a3) extracting a character region in the image region within the classified image region; and (h3) extracting a character region in the image region in the classified image region. 4. The method of claim 3, wherein the step (a1)
(a11) performing gray scale processing on all pixels of the scanned general education data to generate base information for the brightness image;
(a12) generating base information on the label through binarization and labeling of the base information on the generated brightness image, and displaying the base information on the haptic display based on the haptic display. 5. The method of claim 4, wherein the step (a12)
A binarization processing step of reconstructing the generated brightness image into only black pixels and white pixels;
Checking the connection state between adjacent black pixels of the binarized brightness image, determining the pixels connected to each other as one entity, determining at least characters that are classified into entities of at least a character (minimum unit) And a labeling processing step of identifying visual data classified into one or more objects according to the haptic display information. 4. The method of claim 3, wherein the extracting of the image region (a2)
(a21) calculating a change amount (G) with respect to a brightness value between neighboring pixels of the character and the visual data in the brightness image of the input general education data through Equation (1);
(a22) generating a change amount difference (GD) image by only the amount of change (G) with respect to the brightness value between neighboring pixels of the calculated character and the visual data;
(a23) In order to determine the candidate region of the character region expected in the generated GD image, the label for the generated visual data is excluded by Outlier, and the average width ( LW _average ) Using Equation (2);
(a24) reconstructing a maximum change amount difference (MGD) image by applying Equation (3) to a GD image based on an average width ( LW _average ) of the calculated characters of the label;
(a25) For the reconstructed MGD image, binarization is performed by determining the boundary value Tb through Equation (4), and the width of white pixels connected to each line is determined so that the width ( 2 LW _average ) Determining a candidate region of a character region expected in the MGD image by removing a white pixel of a width;
(a26) A morphology operation of dilate and erode is performed on the MGD image in which the candidate region of the predicted character region is determined, so that the character region scattered by the glyph and spacing in the MGD image Merging; And
(a27) It is checked whether or not the labels of the data calculated in the input step of the general education data are included in the character area of the candidate group, and the characters and the visual data in the general education data are classified into the character area and the image area And classifying the plurality of pixels,
Equation 1

Equation 2

Equation 3

Equation 4

In Equation 1 to Equation 4, x, y are the location of the rows and columns of pixels, w, h are the width and height, (x, y) B of the image is the brightness value of the corresponding pixel locations, G (x, y ) Is the variation of neighboring pixels, Q ₁ and Q ₃ are the first quartile and third quartile for the width of the label, LW and NL are the label width and the total number of labels, respectively, and T _b is binarization A visual information display method based on haptic display for a visually impaired person. 7. The method of claim 6, wherein the step (a27)
If the label of the data calculated in the inputting step of the general education data is included in the character area of the candidate group, it is determined as a label for the character, and if the label is not included in the candidate group, Wherein the character and visual data are classified into a character area and an image area, respectively, and a haptic display based visual information display method for a visually impaired person. 4. The method of claim 3, wherein the step (a3)
The method comprising the steps of: determining whether a character exists in an image region determined in the image region extraction step, and extracting the character as a character region when there is a character, Filter), the search range is limited to only the image region, and as a result, the character region in the image region is extracted. The haptic display-based visual information display method for a visually impaired person. The method according to claim 1, wherein the step (b100)
(b1) an image classification step of classifying the image into a low-complexity image and a high-complexity image according to the complexity by analyzing an accumulated histogram of the brightness of the calculated image region;
(b2) a step of discriminating and extracting a center object of a high-complexity image for determining whether the classified high-complexity image includes a main object and extracting the center object; And
(b3) removing and simplifying the noise of the high-complexity image, which simplifies and simplifies the complexity of the image to help the visually impaired to recognize the haptic display. The method according to claim 9, wherein the step (b1)
(b11) calculating an average cumulative value of brightness for the image through a conditional expression (1);
(b12) removing the phenomenon that the cumulative value of the brightness in the low-complexity image appears in a continuous interval by subtracting the average cumulative value of the calculated brightness from each cumulative brightness value through the conditional expression (2); And a control unit,
The conditional expression (1)
for (i = 0, cnt = 0, sum = 0; i <256; i ++) {if ((Q 1 -1.5 (Q 3 - Q 1) <= NH i _{A,; - <= (Q 3} +1.5 (Q 3 Q 1)) then cnt ++, sum + = NH i;} NH average = sum / cnt
The conditional expression (2)
for (i = 0; i <256; i ++) NH _i - = NH _average ;
In the above equations, NH _i is an accumulated value for the corresponding brightness, NH _average is an average cumulative value of brightness for an image, Q ₁ and Q ₃ are a first and a third quartile of cumulative brightness values, A haptic display based visual information display method. 10. The method of claim 9, wherein the step (b2)
(b21) measuring color similarity of pixels constituting the high-complexity image, and converting similar colors into representative colors through quantization;
(b22) determining a presence or absence of a center object by measuring a distribution of colors corresponding to a center object area in the image converted into the representative color; And
(b23) extracting a contour of the center object when the determination result center object exists, and displaying the contour of the center object when the determination result center object exists. 12. The method of claim 11, wherein transforming (b21) similar colors to representative colors comprises:
Determining a quantization level using a Peer Group Filtering (PGF) technique;
And converting the similar colors into representative colors through quantization according to the determined level to reduce complexity with respect to colors of the high-complexity image. 12. The method as claimed in claim 11, wherein the step (b22)
Calculating an intra-image distribution by calculating a variance with respect to a vertical direction and a horizontal direction of the image, assuming that a color having the highest frequency among the central regions of the image defined in Equation (6) is a representative color of the central object;
And determining whether or not the center object exists according to the calculated distribution map.
Equation 6

Where w and h are the width and height of the high complexity image, respectively, and AreaX _obj and AreaY _obj are the ranges of the x and y axes of the central region, respectively. [14] The method of claim 13,
It is determined that the color representing the central object is horizontally wide or the distribution is low through the conditional expression (3) as an image without the center object, and in other cases, it is determined that the center object exists ,
Conditional expression 3
s _max = max ( s _verticality , s _{horizontality} )
if (( s _max == s _{horizontality} ) && ( s _max > T ₁ )) ( s _max < T ₂ )
then I is a non-object image
otherwise I is an object image
( T ₁ = 0.06, T ₂ = 0.01)
In the conditional expression 3, s _verticality And s _{horizontality} is the maximum vertical and maximum horizontal variance, s _max for the representative color of the central object in the high complexity image ( I ) s _verticality and s _{horizontality} , the threshold values T ₁ and T _{2, which} are compared with the maximum variance and the maximum variance, are constants, which are the optimal test values obtained through repeated experiments on various high- Display based visual information display method. 12. The method according to claim 11, wherein the step (b23)
Extracting representative colors of a background region for four corners of the image defined by Equation (7) in the quantized high-complexity image, removing the extracted colors from the entire image, and leaving only information about the center object in the image ;
Performing binarization and labeling on the high-complexity image to remove color information appearing in areas other than the center area, and generating labels for the representative color of the center object; And
Extracting a center object of an image to be converted into tactile information by detecting a contour of a center object left in a high-complexity image by checking whether a position for each of the generated labels is included in the center area, ; &Lt; / RTI &gt;
Equation 7

In the equation 7 AreaX AreaY _n and _n is the background area of the four corners of each image, and w and h are the width and height of the image complexity, the haptic display based on the time information display method for the visually impaired. 10. The method of claim 9, wherein the step (b3)
(b31) calculating a size of a label for contours constituting an image by performing binarization and labeling processing on a high-complexity image;
(b32) When the ratio of the width and the height is calculated by comparing the resolution of the image and the resolution of the haptic display, and the respective labels are reduced according to the resolution of the haptic display through Equation (8) And classifying and removing the image information of the label which is reduced to one cell or less into low-importance noise,
Equation 8

In Equation (8), H _width and H _height are the _width and _height of the haptic display, I _width and I _height are the width and _height of the image, and T _w and T _h are the width and height of the image Wherein the reduction ratio, LW , LH, and n are the width, height, and total number of labels, and S is the size of the pixel when the label is converted to the haptic display resolution. The method of claim 16, wherein the removing and simplifying the noise of the high complexity image (b3)
(b33) detecting a corner point of the contours to lower the complexity of the detected contours, and creating a connecting line using the corner points;
(b34) checking the intersection angle between the connecting lines, and if the intersection angle is greater than or equal to the critical angle, removing a corner point shared by the connecting line and creating a new connecting line at the remaining two points to lower the complexity with respect to the contour of the image &Lt; / RTI &gt;
Wherein the threshold angle is an optimal test value measured by performing an experiment on various images having a high complexity of a contour line, and a haptic display based visual information display method for a visually impaired person. The method as claimed in claim 1, wherein the step (c100)
Converting the extracted character area into information recognizable by a computer by applying optical character recognition (OCR) to the character area, and converting the converted character information into a braille type recognizable by a visually impaired person. A haptic display based visual information display method. The method as claimed in claim 1, wherein the step (c100)
(c1) character recognizing step outside the image area for performing optical character recognition on a character area of a paragraph or a paragraph excluding visual data in the general education data to generate computer-recognizable character information from the image information on the character area; ;
(c2) a character recognition step in the image area for performing optical character recognition on the characters included in the visual information in the general education data and generating computer-recognizable character information from the character area in the image area; And
(c3) braille character recognition step of displaying character information outside the generated image area and character information in the image area as braille information that can be recognized by the blind people through the braille expression algorithm Wherein the visual information is displayed on the haptic display. 20. The method of claim 19, wherein the step (c3)
(c31) calculating the length of the braille to display braille information from the character information outside the image area in the haptic display, and outputting braille information by performing line wrapping according to the width of the haptic display resolution And displaying the haptic display based visual information. 20. The method of claim 19, wherein the step (c3)
(c32) After placing braille information in the video area from the character information in the video area, it is checked whether the braille information overlaps with the surrounding video information. If overlapping, the index corresponding to the braille information is placed in the video area And outputting braille information to the outside of the image area together with the corresponding index. The haptic display-based visual information display method for a visually impaired person according to claim 1, 20. The method of claim 19, wherein the step (c3)
A braille character, a braille character, an arithmetic operation symbol, and a general symbol. The haptic display-based visual information display method for a visually impaired person, comprising: 1. A visual information display system for a visually impaired person having an educational data input unit (10) for inputting data and a tactile information display unit (20) representing braille characters,
A video / character region extraction unit 110 for extracting a character region and an image region for each piece of information mixed in the general education data;
An image hunting unit 120 for each complexity for converting the low-complexity image and the high-complexity image into haptic information by analyzing the extracted image region; And
And a character recognition and a gamut unit 130 for analyzing the extracted text information in the text area and the text information in the image area and converting the text information into braille information that can be recognized by the visually impaired. Haptic display based visual information display system.

Images