KR102483860B1 - Cognitive Assistance System and Method for Visually Impaired using Tactile Display Tablet based on Artificial Intelligent - Google Patents

Abstract

The present invention relates to a cognitive assistance system and method for the visually impaired using a tactile tablet based on artificial intelligence. It relates to a cognitive assistance system and method for the visually impaired using an artificial intelligence-based tactile tablet capable of recognizing an object and providing the shape of the recognized object as braille output information that the blind can tactually recognize. .

Description

Cognitive Assistance System and Method for Visually Impaired using Tactile Display Tablet based on Artificial Intelligent}

The present invention relates to a cognitive assistance system and method for the visually impaired using a tactile tablet based on artificial intelligence. It relates to a cognitive assistance system and method for the visually impaired using an artificial intelligence-based tactile tablet capable of recognizing an object and providing the shape of the recognized object as braille output information that the blind can tactually recognize. .

Due to the research of many technologies to enable the visually impaired to recognize objects more easily, the related wearable market is expanding, and the information provided to the visually impaired is also diversifying. However, since most of these technologies or information require smart devices that are difficult for visually impaired people to use, most of the information provided to visually impaired people through smart devices is currently provided through voice.
Korean Patent Publication No. 10-2012-0063982, a prior invention, relates to an interface device and method for visually impaired users, and relates to a touch recognition unit that recognizes a user's touch and the name of a menu or application service corresponding to the location of the touch. Including a voice notification unit that notifies the user by voice, when a visually impaired user wants to use a portable terminal in the form of a touch screen such as a smartphone, guides voice information by linking touch and voice guidance or text input and voice guidance. are starting
In such an environment, there is a problem in that it is difficult for the visually impaired to intuitively understand that the form of a specific object is conveyed only as voice information to the blind, and the visually impaired user directly touches and recognizes all objects. Doing so has practical limits. In addition, it is difficult to help the visually impaired to intuitively understand only specific concepts through Braille.
In order to solve this problem, it is possible to photograph and transmit information about the surrounding environment of the visually impaired, and analyze it to provide braille output information that directly displays the shape of a specific object so that the visually impaired can acquire various image information, It is necessary to provide a service that can improve the daily life of the visually impaired as a differentiated application technology for their daily life, educational environment, safety, and quality of life by enabling them to recognize and utilize it, but there is no such conventional technology. .

Republic of Korea Patent Publication No. 10-2012-0063982 (2012.06.18.)

The present invention is to photograph an object that the visually impaired person wants to recognize through smart glasses, recognize the object appearing in the photographed image as an object in the image, and output braille that allows the visually impaired person to tactilely recognize the shape of the recognized object. Its purpose is to provide a cognitive assistance system and method for the visually impaired using a tactile tablet based on artificial intelligence that can be provided as information.

In order to solve the above problems, in the present invention, as a cognitive assistance system for the blind using a tactile tablet based on artificial intelligence, including a shooting module and a communication module, in one or more preset shooting modes according to a user's input a smart glass that can be operated and transmits information about an image captured by the photographing module and an operated photographing mode to a user terminal; a tactile tablet capable of providing tactile information to a user by outputting braille based on data received from a user terminal; and a service server that transmits/receives data with the user terminal, wherein the service server includes: an object detection unit that detects an object in a corresponding image based on an image received from the smart glasses; an outline extraction unit which performs pre-processing on the image of the detected object and extracts a shape outline of the object based on the image of the object on which the pre-processing has been performed; and a step-by-step braille output unit for deriving one or more pieces of outline braille output information having different exposure degrees of shape outlines of the object to be output from the tactile tablet, wherein the one or more outline braille output information is sequentially outputted by the tactile tablet. It provides a cognitive assistance system for the visually impaired using a tactile tablet based on artificial intelligence.

In one embodiment of the present invention, the photographing mode operated by the smart glasses may be operated as a first mode, a second mode, and a third mode according to a user's selection input.

In one embodiment of the present invention, the object detection unit may include a first artificial neural network module for detecting an object in the image captured in the first mode; a second artificial neural network module for detecting an object in the image captured in the second mode; and a third artificial neural network module for detecting an object of an image captured in the third mode.

In one embodiment of the present invention, the outline extraction unit, performing contrast processing of the image of the object detected by the object detection unit; A noise removal step of removing noise from the contrast-processed image of the object; and an outline extraction step of extracting a shape outline of the object from the image of the object from which noise has been removed.

In one embodiment of the present invention, in the step of extracting the outlines, first outline information, second outline information, and third outline information for which the degree of exposure of the shape outline of the object is determined according to a predetermined threshold criterion from the image of the object from which noise has been removed. A shape outline of an object including outline information may be extracted.

In one embodiment of the present invention, the step-by-step braille output unit is configured to generate information of an object including first outline information, second outline information, and third outline information of an object extracted according to a predetermined threshold criterion by the outline extraction unit. converting a shape outline into the outline braille output information including first outline braille output information, second outline braille output information, and third outline braille output information to be output from the tactile tablet; and controlling the converted first outline braille output information, the second outline braille output information, and the third outline braille output information to be sequentially output to the tactile tablet.

In one embodiment of the present invention, the screen of the tactile tablet provided by the step-by-step braille output unit includes first outline braille output information and second outline braille output information in which the degree of exposure of the shape outline of the object is determined according to a preset standard. outline braille output information including information and third outline braille output information is sequentially displayed, the output area of the outline braille output information in the tactile tablet is adjusted according to the user's input on the tactile tablet, and the outline braille output information The resolution of the output information may be determined and displayed.

In order to solve the above problems, in one embodiment of the present invention, as a method of providing a cognitive assistance service for the visually impaired implemented in a cognitive assistance system for the visually impaired using an artificial intelligence-based tactile tablet, the cognitive assistance for the visually impaired The system includes a photographing module and a communication module, can be operated in one or more predetermined photographing modes according to a user's input, and transmits information about an image captured by the photographing module and the operated photographing mode to a user terminal. a smart glass; a tactile tablet capable of providing tactile information to a user by performing braille output based on data received from a user terminal; and a service server that transmits and receives data with the user terminal, wherein the service server includes: an object detection step of detecting an object in a corresponding image based on an image received from the smart glasses; An outline extraction step of performing pre-processing on the image of the detected object and extracting a shape outline of the object based on the image of the object on which the pre-processing has been performed; and a step-by-step braille output step of deriving one or more outline braille output information having different exposure degrees of the shape outline of the object to be output from the tactile tablet, wherein the one or more outline braille output information is sequentially output in the tactile tablet. Provides a method for providing a cognitive assistance service for the visually impaired using a tactile tablet based on artificial intelligence, which is displayed as

According to an embodiment of the present invention, the shape of an object photographed by a user through smart glasses is provided as braille output information that a visually impaired person can tactilely recognize, so that a user with a visual impairment can see the shape of a specific object. It can exert an effect that can be perceived intuitively.

According to an embodiment of the present invention, by analyzing images acquired through smart glasses and providing information on the surrounding environment as braille information that the blind can more accurately understand, the daily life of the visually impaired who cannot see the surroundings directly It can exert an effect that can improve the discomfort in

According to an embodiment of the present invention, the update time of the learning model can be reduced by classifying the captured images according to the shooting mode in the smart glasses and detecting and recognizing objects by each learned artificial neural network. effect can be exerted.

According to an embodiment of the present invention, by performing pre-processing of the image of the detected object, noise can be removed, a more accurate shape of the object can be extracted, and an outline of the shape of the object can be derived.

According to an embodiment of the present invention, one or more outline embossed output information having different degrees of exposure are provided through the tactile tablet, and the output area and resolution are changed according to input from the tactile tablet including user's enlargement or reduction. By providing the adjusted outline braille information, the user can more intuitively grasp the shape of the object captured by the user.

1 schematically illustrates the overall form of a cognitive assistance system for the visually impaired using a tactile tablet based on artificial intelligence according to an embodiment of the present invention.
2 schematically illustrates the shape of a smart glass according to an embodiment of the present invention.
3 is a diagram illustrating an operation mode of a smart glass according to an embodiment of the present invention.
4 schematically illustrates the internal configuration of an object detection unit according to an embodiment of the present invention.
5 schematically illustrates the execution step of the outline extraction unit according to an embodiment of the present invention.
6 schematically illustrates the shape of an object extracted according to the operation of the outline extractor according to an embodiment of the present invention.
FIG. 7 schematically illustrates the shape outline of an object and the output information of braille outlines for the shape outline according to an embodiment of the present invention.
8 schematically shows a screen of a tactile tablet provided by a step-by-step outline extraction unit according to an embodiment of the present invention.
9 illustratively illustrates a computing device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a certain component is said to "include", this means that it may further include other components without excluding other components unless otherwise stated.

In addition, terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

In this specification, a "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware. On the other hand, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Therefore, as an example, '~unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and '~units' may be combined into smaller numbers of components and '~units' or further separated into additional components and '~units'. In addition, components and '~units' may be implemented to play one or more CPUs in a device or a secure multimedia card.

A “user terminal” referred to below may be implemented as a computer or portable terminal capable of accessing a server or other terminals through a network. Here, the computer includes, for example, a laptop, desktop, laptop, etc. equipped with a web browser, and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , PCS(Personal Communication System), GSM(Global System for Mobile communications), PDC(Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, and the like. In addition, a "network" is a wired network such as a local area network (LAN), a wide area network (WAN) or a value added network (VAN), a mobile radio communication network, or a satellite It can be implemented in all kinds of wireless networks such as communication networks.

1 schematically illustrates the overall form of a cognitive assistance system for the visually impaired using a tactile tablet based on artificial intelligence according to an embodiment of the present invention.

The cognitive assistance system for the visually impaired using the artificial intelligence-based tactile tablet 3000 of the present invention includes smart glasses 2000, a service server 1000, and a tactile tablet 3000. The smart glasses 2000 may transmit and receive data through a user terminal, and the user terminal may transmit data received from the smart glasses 2000 to the service server 1000. The user terminal may correspond to the aforementioned user terminal, and the service server 1000 corresponds to a computing device including one or more processors and one or more memories.

The user terminal communicates with the smart glasses 2000, the service server 1000, and the tactile tablet 3000, transmits image information or image information captured by the smart glasses 2000 to the server, and receives the image service server ( In step 1000, information on an image analyzed based on artificial intelligence technology is received, and outline dotted line output information of an object representing the shape of a recognized object is derived, and transmitted to the tactile tablet 3000 so that it can be output.

The smart glasses 2000 may include at least one of a photographing module 2100, a speaker, a communication module 2300, and a button unit 2400. The user wears the smart glasses 2000 in the same way as wearing glasses, and inputs to the provided button unit 2400 through the photographing module 2100 built in the smart glasses 2000 according to the user's point of view. External environments, objects, and people can be photographed. In addition, the captured image is transmitted to the user terminal through the communication module 2300. The speaker unit 2200 may output audio output information received from the user terminal or preset audio output information set to be provided according to the operation of the smart glasses 2000 . The button unit 2400 includes one or more buttons, and a preset function is set to each button so that the corresponding function can be operated according to a user's input.

Meanwhile, the service server 1000 includes an object detection unit 1100 that detects an object of a corresponding image based on an image received from the smart glasses 2000; an outline extraction unit 1100 which performs pre-processing on the image of the detected object and extracts a shape outline of the object based on the pre-processed image of the object; and a step-by-step braille output unit 1300 for deriving at least one outline braille output information having different exposure degrees of the shape outline of the object to be output from the tactile tablet 3000.

The object detection unit 1100 detects an object from an image received from the smart glasses 2000. Preferably, the object detection unit 1100 includes a first artificial neural network module for detecting an object for each image captured in the first mode, the second mode, and the third mode of the smart glasses 2000; An artificial neural network module, and a third artificial neural network module. The object detection unit 1100 classifies the photographed image according to the photographing mode operated by the smart glasses 2000, inputs it to each of a plurality of learned artificial neural network modules, and receives the received images by each learned artificial neural network module. Objects included in images can be recognized.

The outline extraction unit 1100 performs pre-processing of the image of the detected object, and extracts a shape outline of the object based on the pre-processed image of the object. The outline extraction unit 1100 performs contrast processing on the image of the object detected by the object detection unit 1100 and removes noise from the image of the contrast processed object to extract a more distinct shape outline of the object. there is. Preferably, the shape outline of the object includes first outline information, second outline information, and third outline information for which the degree of exposure of the shape outline of the object is determined according to a preset threshold criterion. The shape outline of the object including the first outline information, the second outline information, and the third outline information becomes a basis for deriving the outline braille output information generated by the step-by-step braille output unit 1300 thereafter.

The step-by-step braille output unit 1300 derives one or more pieces of outline braille output information, each of which has a different degree of exposure of the shape outline of an object to be output from the tactile tablet 3000. In order for the visually impaired user of the present invention to more accurately grasp the shape of the object, outline dotted output information with different exposure levels of the shape of the object is derived, and one or more derived dotted outline dotted line output information is sequentially tactile It can be provided to the user through the tablet 3000. For example, first, on the tactile tablet 3000, first outline braille output information indicating outlines representing only some shapes of objects is displayed, and then outlines of detailed and specific object shapes rather than first outline braille output information are displayed. The second outline braille output information indicating the outline is displayed, and then the third outline braille output information indicating the outline of all shapes of the object is displayed. In this way, by providing one or more pieces of outline braille output information having different degrees of exposure step by step through the tactile tablet 3000, the user can more intuitively grasp the shape of the object he or she has photographed.

Meanwhile, the tactile tablet 3000 may output braille based on data received from the service server 1000 through the user terminal and provide tactile information to the user. Any interface device capable of providing tactile graphics, a tactile interface device capable of interacting with a user, and a computing device capable of outputting tactile graphics may all be included.

The cognitive assistance system for the visually impaired using the tactile tablet 3000 based on artificial intelligence shown in FIG. Only the minimum components to explain are shown.

Hereinafter, specific operations of the smart glasses 2000 and the service server 1000 according to the present invention will be described in more detail.

FIG. 2 schematically illustrates a shape of a smart glass 2000 according to an embodiment of the present invention, and FIG. 3 is a diagram explaining an operation mode of the smart glass 2000 according to an embodiment of the present invention.

As shown in FIG. 2, the smart glasses 2000 are configured in a similar shape to glasses, so that the user wears the smart glasses 2000 in front of the eyes and receives a built-in shooting according to the user's input through the provided button unit 2400. External environments, objects, and people may be photographed by the module 2100 . The smart glasses 2000 can be operated in one or more preset shooting modes according to a user's input, and transmits an image captured by the shooting module 2100 and information on the operated shooting mode to the user terminal. .

The button unit 2400 may control the operation of the smart glasses 2000 according to a user's input. As shown in FIG. 2 , the button unit 2400 may include one or more buttons, and each button may control the operation of the smart glasses 2000 according to a preset function of the corresponding button. According to an embodiment of the present invention, the button unit 2400 is one of a shooting button for executing shooting through the shooting module 2100 according to a user's input and a mode setting button for setting the shooting mode of the smart glasses 2000. Including the above, but not limited thereto, the function of the button to be operated by the user can be set in advance according to the situation.

In FIG. 2, the shape and components of the smart glass 2000 are shown, but the shapes or locations where components are arranged are not limited to the shapes and locations shown in FIG. It can be placed anywhere on the smart glass (2000).

Preferably, the photographing mode operated in the smart glasses 2000 may be operated as a first mode, a second mode, and a third mode according to a user's selection input. In one embodiment of the present invention, the first mode, the second mode, and the third mode may be classified according to the size of an object photographed by a visually impaired user of the present invention. As shown in FIG. 3, the first mode can be set when shooting large objects such as cars, buses, and beds, and the second mode is for shooting medium-sized objects such as radios, chairs, and laptops. case, and the third mode can be set when shooting small objects such as shampoo, soap, and glasses. A user of the smart glasses 2000 may select a specific mode through an input through the button unit 2400 and take a photograph of a corresponding object through the smart glasses 2000 . After that, the smart glasses 2000 may transmit the captured image and information about the photographing mode of the smart glasses 2000 operated when the image is photographed to the user terminal.

In this way, based on the image taken by the visually impaired user through the smart glasses 2000, an object is detected from the image, and an object appearing in the image in which the detected object is taken is recognized as an object in the image, By providing the shape of the recognized object as braille output information that the visually impaired can tactilely recognize, it is possible to exert an effect of allowing the visually impaired user to recognize the shape of a specific object more intuitively.

4 schematically illustrates the internal configuration of the object detection unit 1100 according to an embodiment of the present invention.

The object detection unit 1100 according to an embodiment of the present invention includes a first artificial neural network module for detecting an object in an image captured in the first mode; a second artificial neural network module for detecting an object in the image captured in the second mode; and a third artificial neural network module for detecting an object in the image captured in the third mode.

As described above, the image captured by the smart glasses 2000 is transmitted to the service server 1000 through the user terminal. The smart glasses 2000 may be operated in a shooting mode including a first mode, a second mode, and a third mode according to a user's selection input. In one embodiment of the present invention, the first mode is a shooting mode for a large object, the second mode is a shooting mode for a medium-sized object, and the third mode is a shooting mode for a small object. The object detection unit 1100 includes a first artificial neural network module for detecting an object in an image captured in a first mode, a second artificial neural network module for detecting an object in an image captured in a second mode, and a third mode. Including the third artificial neural network module for detecting objects in the captured image, information on the captured image and the operated shooting mode is received from the smart glasses 2000, and each image is classified to perform appropriate learning input to the artificial neural network module and perform recognition on it. Although three learned artificial neural network modules are shown in FIG. 4, it is not limited thereto, and classification of images is performed according to a shooting mode operated in the smart glasses 2000 or a predetermined criterion of images received from the smart glasses 2000. and input the classified image to each suitable artificial neural network module.

As such, in one embodiment of the present invention, the object detection unit 1100 classifies the captured image according to the shooting mode in the smart glasses 2000, and detects and recognizes the object by each learned artificial neural network. By doing so, it is possible to exhibit an effect of shortening the update time of the learning model.

Figure 5 schematically shows the steps performed by the outline extractor 1100 according to an embodiment of the present invention, and Figure 6 shows an object extracted according to the operation of the outline extractor 1100 according to an embodiment of the present invention. The shape of is shown schematically.

The outline extraction unit 1100 according to an embodiment of the present invention, as shown in FIG. 5 , performs contrast processing on the image of the object detected by the object detection unit 1100 (S100); A noise removal step (S110) of removing noise from the contrast-processed image of the object; and an outline extraction step (S120) of extracting a shape outline of the object from the image of the object from which noise has been removed.

Specifically, in step S100, the outline extraction unit 1100 performs contrast processing on the image of the object detected by the object detection unit 1100. In an embodiment of the present invention, the outline extraction unit 1100 may distinguish an object and a background from an image using a GrabCut algorithm. As shown in FIG. 6, if the GrabCut algorithm is used, the background can be removed from the image and only the shape part of the object to be extracted can be derived, and a mask for the background can be created and created by performing marking to distinguish the object from the background. By applying a mask and an image of an object to an algorithm, a more elaborate shape part of an object can be derived. Also, in step S100, as shown in FIG. 6, the size of the shape part of the extracted object may be adjusted.

In step S110, the noise of the image of the object extracted through contrast processing is removed. In an embodiment of the present invention, noise of an image may be removed by performing GaussianBlur using a Gaosian filter in order to extract an outline of an accurate shape. If the noise of the image is not removed, since it may be difficult to accurately detect the outline of the object, in the present invention, the noise of the image may be removed by performing step S110. In one embodiment of the present invention, it is not limited to blurring using a Gaosian filter, and other methods of removing noise from an image may be used.

In step S120, the shape outline of the object is extracted from the image of the object from which noise has been removed. In one embodiment of the present invention, the shape outline of an object may be extracted based on the Canny algorithm. The Canny algorithm is a multi-step algorithm that detects image information or edge information from image information. It derives a gradient vector based on an image from which noise has been removed and adjusts the gradient vector according to a preset standard. The shape outline of an object can be extracted. The Canny algorithm may also include removing the noise (S110). Preferably, the outline extraction unit 1100 includes first outline information, second outline information, and third outline information for which the degree of exposure of the shape outline of the object is determined according to a preset threshold criterion from the image of the object from which noise has been removed. Extracts the shape outline of an object that includes information. As shown in FIG. 6, first outline information, second outline information, and third outline information having different degrees of exposure are derived, and thereafter, each outline information is converted to outline braille output information by the step-by-step braille output unit 1300. can be converted

In this way, in one embodiment of the present invention, the effect of removing noise by performing pre-processing of the image of the detected object, extracting a more accurate shape of the object, and deriving the outline for the shape of the object can exert

FIG. 7 schematically illustrates the shape outline of an object and the output information of braille outlines for the shape outline according to an embodiment of the present invention.

As described above, the outline extraction unit 1100 determines the degree of exposure of the shape outline of the object according to a preset threshold criterion from the image of the object from which noise has been removed, the first outline information, the second outline information, and the third outline information. extracts a shape outline of an object including, and the step-by-step braille output unit 1300 according to an embodiment of the present invention has a first step of the object extracted by the outline extractor 1100 according to a preset threshold criterion. First outline braille output information, second outline braille output information, and third outline braille output information to output the shape outline of the object including outline information, second outline information, and third outline information in the tactile tablet 3000 The step of converting to the outline braille output information including is performed. Thereafter, the converted first outline braille output information, the second outline braille output information, and the third outline braille output information can be controlled to be sequentially output to the tactile tablet 3000 . Each of the left screens in (a), (b), and (c) of FIG. 7 shows the outline information derived by the outline extraction unit 1100, and each right screen shows the outline braille output information converted from the outline information. show In one embodiment of the present invention, the first outline braille output information indicating the outline representing only some shapes of a rough object is provided through the tactile tablet 3000, and then the shape of the object in detail and specific rather than the first outline braille output information Second outline braille output information indicating the outline of the object may be provided, and then third outline braille output information indicating outlines of all shapes of the object may be provided. In this way, by providing one or more pieces of outline braille output information having different degrees of exposure step by step through the tactile tablet 3000, the user can more intuitively grasp the shape of the object he or she has photographed. Preferably, the screen of the tactile tablet 3000 provided by the step-by-step braille output unit 1300 includes first outline braille output information derived according to a preset first threshold value standard and a preset second threshold value. The second outline braille output information derived according to the criterion and the third outline dotted line output information derived according to the predetermined third threshold criterion are sequentially displayed.

8 schematically shows a screen of the tactile tablet 3000 provided by the step-by-step outline extraction unit 1100 according to an embodiment of the present invention.

According to an embodiment of the present invention, the screen of the tactile tablet 3000 provided by the step-by-step braille output unit 1300 has a first outline braille output in which the degree of exposure of the shape outline of an object is determined according to a preset standard. Outline braille output information including information, second outline braille output information, and third outline braille output information is sequentially displayed, and the outline in the tactile tablet 3000 according to the user's input in the tactile tablet 3000 The output area of the braille output information is adjusted, and the resolution of the outline braille output information is determined and displayed. The outline dotted character output information shown in (a) and (b) of FIG. 8 can be sequentially provided to the user through the tactile tablet 3000, and the user who is provided with the screen shown in FIG. 8 can more specifically determine the shape of the object. In order to confirm, the output area of the outline braille output information can be adjusted by performing an input such as enlargement or reduction. As shown in FIG. 8, a specific area can be enlarged or reduced according to a user's input, and when the outline dot output information is enlarged, higher resolution outline dot output information is provided so that the user can recognize the shape of a detailed object. can do. In this way, one or more outline dotted line output information having different degrees of exposure are provided through the tactile tablet 3000 step by step, and the output area and By providing the resolution-adjusted outline braille information, the user can more intuitively grasp the shape of the object captured by the user.

9 illustratively illustrates the internal configuration of a computing device according to an embodiment of the present invention.

As shown in FIG. 9, a computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, an input/output subsystem ( I/O subsystem 11400, a power circuit 11500, and a communication circuit 11600 may be included at least. At this time, the computing device 11000 may correspond to the smart glasses 2000, the user terminal, the service server 1000, and the tactile tablet 3000.

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. there is. The memory 11200 may include various other data included in a software module, a command set, or a learned embedding model necessary for the operation of the computing device 11000 .

In this case, access to the memory 11200 from other components, such as the processor 11100 or the peripheral device interface 11300, may be controlled by the processor 11100.

Peripheral interface 11300 may couple input and/or output peripherals of computing device 11000 to processor 11100 and memory 11200 . The processor 11100 may execute various functions for the computing device 11000 and process data by executing software modules or command sets stored in the memory 11200 .

Input/output subsystem 11400 can couple various input/output peripherals to peripheral interface 11300. For example, the input/output subsystem 11400 may include a controller for coupling a peripheral device such as a monitor, keyboard, mouse, printer, or touch screen or sensor to the peripheral interface 11300 as needed. According to another aspect, input/output peripherals may be coupled to the peripheral interface 11300 without going through the input/output subsystem 11400.

The power circuit 11500 may supply power to all or some of the terminal's components. For example, power circuit 11500 may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator or power It may contain any other components for creation, management and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, the communication circuit 11600 may include an RF circuit and transmit/receive an RF signal, also known as an electromagnetic signal, to enable communication with another computing device.

The embodiment of FIG. 9 is only an example of the computing device 11000, and the computing device 11000 may omit some of the components shown in FIG. 9, further include additional components not shown in FIG. It may have a configuration or arrangement combining two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor in addition to the components shown in FIG. , Bluetooth, NFC, Zigbee, etc.) may include a circuit for RF communication. Components that may be included in the computing device 11000 may be implemented as hardware including one or more signal processing or application-specific integrated circuits, software, or a combination of both hardware and software.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in computer readable media. In particular, the program according to the present embodiment may be composed of a PC-based program or a mobile terminal-specific application. An application to which the present invention is applied may be installed in a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request of a user terminal.

The device described above may be implemented as a hardware component, a software component, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computing devices and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (8)

As a cognitive assistance system for the visually impaired using a tactile tablet based on artificial intelligence,
A smart glass that includes a photographing module and a communication module, can be operated in one or more predetermined photographing modes according to a user's input, and transmits information about an image captured by the photographing module and the operated photographing mode to a user terminal. ;
a tactile tablet capable of providing tactile information to a user by outputting braille based on data received from a user terminal; and
Including; service server for transmitting and receiving data with the user terminal;
The service server,
an object detection unit for detecting an object in a corresponding image based on the image received from the smart glasses;
an outline extraction unit which performs pre-processing on the image of the detected object and extracts a shape outline of the object based on the image of the object on which the pre-processing has been performed; and
A step-by-step braille output unit for deriving at least one outline braille output information having different exposure degrees of the shape outline of the object to be output from the tactile tablet;
In the tactile tablet, the one or more outline braille output information is sequentially displayed, a cognitive assistance system for the visually impaired using a tactile tablet based on artificial intelligence.
delete delete The method of claim 1,
The outline extraction unit,
performing contrast processing on the image of the object detected by the object detection unit;
A noise removal step of removing noise from the contrast-processed image of the object; and
A cognitive assistance system for the visually impaired using a tactile tablet based on artificial intelligence, performing the outline extraction step of extracting the shape outline of the object from the image of the object from which noise has been removed.
delete delete delete As a method of providing a cognitive assistance service for the visually impaired implemented in a cognitive assistance system for the visually impaired using an artificial intelligence-based tactile tablet,
The cognitive assistance system for the visually impaired,
A smart glass that includes a photographing module and a communication module, can be operated in one or more predetermined photographing modes according to a user's input, and transmits information about an image captured by the photographing module and the operated photographing mode to a user terminal. ;
a tactile tablet capable of providing tactile information to a user by outputting braille based on data received from a user terminal; and
Including; service server for transmitting and receiving data with the user terminal;
The service server,
an object detection step of detecting an object in a corresponding image based on the image received from the smart glasses;
An outline extraction step of performing pre-processing on the image of the detected object and extracting a shape outline of the object based on the image of the object on which the pre-processing has been performed; and
Performing a step-by-step braille output step of deriving at least one outline braille output information having different exposure degrees of the shape outline of the object to be output from the tactile tablet;
A method for providing a cognitive assistance service for the visually impaired using a tactile tablet based on artificial intelligence, wherein the one or more outline braille output information is sequentially displayed in the tactile tablet.

Images