WO2020116674A1 - Intelligent method and device for enhancing visual information for mobility-impaired person - Google Patents

Abstract

Provided is a method for increasing the utilization and accuracy of an audio-visual information enhancement device by eliminating errors that occur due to the use of refined information when utilizing conventional technology (sensor or imaging device) deemed a limitation of audio-visual information enhancement devices, and by delivering received information as-is. An intelligent information enhancement system according to an embodiment of the present invention includes: a mobility-impaired person's terminal which generates a video, transmits the video to a monitoring personnel terminal, and enhances guide information received from the monitoring personnel terminal; and the monitoring personnel terminal which transmits, to the mobility-impaired person's terminal, guide information generated by monitoring the video received from the mobility-impaired person's terminal. Accordingly, an information enhancement service may be provided which transmits, at high quality and in real time, various information (e.g., directions to hospitals, vehicle approach warnings, traffic signal change notifications, or the like) that may enable socially vulnerable groups or the audio-visually and mobility-impaired to cope with various situations.

Description

Intelligent visual information augmentation method and device for the weak

The present invention relates to a visual assistance tool, and more particularly, to a method and apparatus for augmenting visual information for a weak person in traffic.

The existing audio-visual information enhancement device was mainly used as a device for the disabled (visually impaired and visually impaired). As shown in FIG. 1, an information augmentation device similar to a smart glasses is mainly used, and the desired information is provided to a visually impaired person through the camera 112 in an appropriate form (mainly a reduced form of an image or macroscopic information such as a direction). Convert and provide. The smart glasses include a gyroscope 138 that can measure a direction, and thus serve to provide location information in a situation that is difficult to provide as an image.

The existing audio-visual information enhancement device for the traffic weak using smart glasses uses a camera sensor to identify letters, numbers, and symbols, and informs them by voice, or provides information using a rather frivolous device such as an LED array. Stopped. In addition, in identifying letters/numbers/symbols, it can be considered that the accuracy of the image is very poor because there was a limitation with the amount of existing information.

In addition, as an ICT linking technology for the hearing impaired, a device capable of making a telephone call or information indication similar to that of a normal person has been developed using bone hearing without using an external aid such as a hearing aid using bone vibration.

However, all of these devices are devices that transmit audio-visual stimulation through a device or sensor to the audio-visual weak, and when errors or problems occur when delivering information obtained from them rather than the limits of the sensors or devices themselves Occupy the majority. For example, artificial intelligence technology has been greatly developed at present, and it is possible to recognize a certain amount of video or voice. However, when using refined information using existing technology, significant errors may occur.

The present invention has been devised to solve the problems as described above, and the object of the present invention is to occur when using refined information by utilizing existing technology (sensors or imaging devices) that were pointed out as limitations in the audio-visual information enhancement device. An object of the present invention is to provide a method of removing errors and improving the utilization and accuracy of the audio-visual information enhancement device by transmitting the transmitted information as it is.

According to an embodiment of the present invention for achieving the above object, the intelligent information augmentation system generates a video and transmits it to a monitor agent terminal, a traffic weak terminal that augments the guide information received from the monitor agent terminal; And a monitor agent terminal transmitting the guide information generated by monitoring the video received from the traffic weak terminal to the traffic weak terminal.

The guide information may include at least one of geographic guide information, road guide information, vehicle access guide information, and traffic signal guide information.

The traffic weak terminal and the monitor agent terminal may be matched one-to-one.

The traffic weak terminal may output guide information as auditory information.

The traffic weak terminal may output guide information in Braille.

The traffic weak terminal may generate a video using a 360 camera or a wide viewing angle camera.

The video generated by the traffic weak terminal and the guide information generated by the monitor agent terminal may be stored in the cloud and used to train an AI model for service provision.

According to another aspect of the present invention, the traffic weak terminal, generating a video and transmitting it to the monitor agent terminal; A monitor agent terminal transmitting the guide information generated by monitoring the video received from the traffic weak terminal to the traffic weak terminal; It includes; the step of the traffic weak terminal, augmenting the guide information received from the monitor terminal.

As described above, according to embodiments of the present invention, a variety of information (e.g., directions to hospitals, vehicle access warnings, traffic signal change guidance) that can respond to various situations targeting the socially vulnerable or the visually impaired Etc.), it is possible to provide an information augmentation service that delivers high quality in real time.

1 is a view showing a conventional smart glasses,

Figure 2 is a conceptual diagram of an intelligent visual information augmentation service for the disabled according to an embodiment of the present invention,

3 is a flow chart of an intelligent visual information augmentation service for the disabled according to an embodiment of the present invention,

4 and 5 are diagrams illustrating a haptic device using a braille modulator and braille output using the same,

6 is a diagram showing an intelligent information augmentation system for a weak person according to an embodiment of the present invention;

7 is a detailed block diagram of the intelligent information augmentation device shown in FIG. 6,

8 is a detailed block diagram of the monitor agent terminal illustrated in FIG. 6.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

2 is a conceptual diagram of an intelligent visual information augmentation service for a weak person according to an embodiment of the present invention.

In an embodiment of the present invention, as shown in FIG. 2, in a state in which a so-called traffic vulnerable person such as a blind person, a hearing impaired person, an elderly person, an infant, etc. is equipped with an image acquisition device for real-time monitoring, surrounding environment, motion, voice information, etc. Get information.

The acquired information delivers the necessary information by using a high-speed network for real-time feedback monitoring. At this time, the real-time feedback monitoring is provided by the person directly.

Therefore, a method for properly analyzing surrounding information, motion, and voice information and providing necessary information is provided by a selected person through a certain level of education, and the proposed system also provides real-time monitoring and battery performance of the image acquisition device. It becomes an essential device to operate.

3 is a flow chart of an intelligent visual information augmentation service for the disabled in accordance with an embodiment of the present invention.

As shown in FIG. 3, when a weak handicapped person acquires the surrounding environment, motion, and voice information using an image acquisition device capable of real-time monitoring (eg, a 360 camera capable of recognizing the surrounding environment, a wide viewing angle camera, etc.), It is delivered in real time through a high-speed network.

The delivered information delivers the selected information to the traffic weak by trained and trained people. At this time, the selected information is specific to geographic guidance, road guidance, vehicle access information, traffic signal change guidance, etc. It will mainly convey non-detailed information.

When transmitting, specific information is mainly delivered to the visually-impaired person using a wearable information-enhancing device (e.g., smart glasses, haptic devices, headsets, etc.).For the hearing-impaired person, a haptic device or a smartphone using a braille modulator is used. Deliver specific information.

The haptic device usable in this case is illustrated in FIG. 4. 4 is a view showing a haptic device using a braille modulator. And, FIG. 5 shows a state in which the haptic device displays guide information for guiding in the left direction in Braille.

The information obtained from the image acquisition device can be stored in a real-time feedback monitoring service and a cloud platform for driving large-capacity video/audio information, and the stored information is AI-based environment recognition and speech conversion technology to utilize video/audio data for each situation. It is easy to utilize accumulated data applied to.

6 is a diagram illustrating an intelligent information augmentation system for a weak person in accordance with an embodiment of the present invention.

Intelligent information augmentation system according to an embodiment of the present invention, as shown in Figure 6, intelligent information augmentation devices (100-1, 100-2, ..., 100-n) and the monitor agent terminals 200 -1, 200-2, ..., 200-n) are connected and built to enable mutual communication through a high-speed network.

Intelligent information enhancers (100-1, 100-2, ..., 100-n) and monitor agent terminals (200-1, 200-2, ..., 200-n) are matched one-to-one . In other words, one monitor agent is assigned to one weak person.

The intelligent information augmentation device (100-1, 100-2, ..., 100-n) monitors the video generated by using the camera agent terminal (200-1, 200-2, ..., 200-n) Transfer to.

Monitor agent terminal (200-1, 200-2, ..., 200-n) is generated by analyzing the video received from the intelligent information enhancement device (100-1, 100-2, ..., 100-n) The information is fed back to the intelligent information augmentation devices (100-1, 100-2, ..., 100-n).

In the guide information fed back to the intelligent information enhancer (100-1, 100-2, ..., 100-n) by the monitor agent terminal (200-1, 200-2, ..., 200-n), Geographical guidance information, road guidance information, vehicle access guidance information, traffic signal guidance information, and the like may be included.

Then, the intelligent information augmentation device (100-1, 100-2, ..., 100-n) auditory information from the monitor agent terminal (200-1, 200-2, ..., 200-n) / Visually output and augment.

7 is a detailed block diagram of the intelligent information augmentation devices 100-1, 100-2, ..., 100-n shown in FIG. 6.

Intelligent information augmentation device (100-1, 100-2, ..., 100-n) is a device worn on the weak, as shown in Figure 7, the camera 110, the communication unit 120, the processor ( 130) and the augmentation unit 140.

The camera 110 generates a video by photographing a scene viewed by the traffic weak or around the point where the traffic weak is located. Since it is a video, it contains video and audio.

The communication unit 120 transmits the video generated by the camera 110 to the monitor agent terminals 200-1, 200-2, ..., 200-n through a high-speed network, and the monitor agent terminal 200-1 , 200-2, ..., 200-n).

The processor 130 augments the guide information received from the monitor agent terminals 200-1, 200-2, ..., 200-n through the communication unit 120 through the auditory information enhancement unit 140.

The augmentation unit 140 includes a speaker and a headset for aurally outputting guide information, a display for visually outputting guide information, a haptic device for tactilely outputting guide information, and the like.

8 is a detailed block diagram of the monitor agent terminals 200-1, 200-2, ..., 200-n shown in FIG.

Monitor agent terminal (200-1, 200-2, ..., 200-n), as shown in Figure 8, the communication unit 210, the output unit 220, the processor 230, the input unit 240 and It may be implemented as a computing system including a storage unit 250.

The communication unit 210 receives a video from the intelligent information augmentation devices 100-1, 100-2, ..., 100-n through a high-speed network, and receives guide information to be described later by the intelligent information augmentation device 100-1, 100-2, ..., 100-n).

The output unit 220 is a display unit for outputting the received video, and the input unit 240 is an input unit for receiving guidance information from a monitor agent.

The processor 230 outputs a video received from the intelligent information enhancement device 100-1, 100-2, ..., 100-n through the communication unit 210 through the output unit 220, and input unit 240 ) And transmits the guide information input through the communication unit 210 to the intelligent information enhancement devices 100-1, 100-2, ..., 100-n.

The storage unit 250 provides necessary storage space for the processor 230 to operate and function.

So far, a preferred embodiment has been described in detail for an intelligent visual information augmentation method and apparatus for the weak of traffic.

In an embodiment of the present invention, a real-time information augmentation service is provided by providing high-quality real-time information augmentation service through real-time monitoring and feedback of a user's surrounding environment and motion, and voice information, and data accumulated through contextual video/audio data storage and management is AI-based A system that can be used as an environment recognition and speech conversion technology was proposed.

Information enhancement service that delivers various information (e.g., directions to hospitals, vehicle access warnings, traffic signal change guides, etc.) in real time in high quality, targeting socially vulnerable groups or people with limited audiovisual traffic. It is possible to provide.

Meanwhile, it goes without saying that the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program that performs functions of the apparatus and method according to the present embodiment. Further, the technical idea according to various embodiments of the present invention may be implemented in the form of computer-readable codes recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and stores data. Of course, the computer-readable recording medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, or the like. In addition, computer-readable codes or programs stored in a computer-readable recording medium may be transmitted through a network connected between computers.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the claims. In addition, various modifications can be implemented by those skilled in the art, and these modifications should not be individually understood from the technical idea or prospect of the present invention.

Claims (8)

1. A traffic weak terminal for generating a video and transmitting it to a monitor agent terminal, and augmenting the guide information received from the monitor agent terminal; And

   An intelligent information augmentation system comprising: a monitor agent terminal that transmits guide information generated by monitoring a video received from a traffic weak terminal to a traffic weak terminal.
2. The method according to claim 1,

   Guidance information,

   Intelligent information augmentation system, characterized in that it comprises at least one of geographic guidance information, road guidance information, vehicle access guidance information, traffic signal guidance information.
3. The method according to claim 1,

   Traffic weak terminal and monitor agent terminal,

   An intelligent information augmentation system characterized by being matched one-to-one.
4. The method according to claim 1,

   Traffic weak terminal,

   Intelligent information augmentation system, characterized in that for outputting the guide information as auditory information.
5. The method according to claim 1,

   Traffic weak terminal,

   Intelligent information augmentation system, characterized in that outputting the guide information in Braille.
6. The method according to claim 1,

   Traffic weak terminal,

   An intelligent information augmentation system characterized by generating a video using a 360 camera or a wide viewing angle camera.
7. The method according to claim 1,

   The video generated by the traffic weak terminal and the guide information generated by the monitor agent terminal include:

   Intelligent information augmentation system, characterized in that it is stored in the cloud and used for learning AI models for service provision.
8. A traffic weak terminal, generating a video and transmitting it to a monitor agent terminal;

   A monitor agent terminal transmitting the guide information generated by monitoring the video received from the traffic weak terminal to the traffic weak terminal;

   A method for augmenting intelligent information comprising a step of augmenting guide information received from a terminal of a monitor agent by a traffic weak terminal.

Images