RU198852U1 - PORTABLE VISUAL CAPTURE AND RECOGNITION DEVICE - Google Patents

Abstract

The utility model relates to the field of typhlotechnical means and can be used by visually impaired, blind and deaf-blind people in order to compensate for impaired visual functions, contributing to the active adaptation of a person to the environment. The technical result consists in increasing the degree of user adaptation to the environment. The device contains a compact case with a control board, a single-board computer, video cameras and a rangefinder. 2 wp f-ly, 5 dwg

Description

Technology area

The utility model relates to the field of typhlotechnical means and can be used by visually impaired, blind and deaf-blind people in order to compensate for impaired visual functions, contributing to the active adaptation of a person to the environment.

State of the art

Known cane-scanner for people with visual impairments, which allows you to inform the user about the presence and distance to obstacles along their path, both outdoors and indoors (see RU 141620 U1, publ. 06/10/2014, A61F 9/08). The known device is a tactile cane, in the rod and handle of which an electronic information processing unit controlled by the keyboard is mounted, which receives energy from the power module built into the cane and contains a non-volatile memory for storing digitized audio messages, connected via a digital bus to a microprocessor, to which they are also connected via the matching module three ultrasonic transceivers mounted on the cane in such a way that the projection of the solid angles of their echolocation zones forms on a vertical plane perpendicular to the direction of movement of the user holding the scanner cane in his hand, the total working area, the shape and dimensions of which make it possible to determine the presence of those in front of the cane - a scanner of objects by the height and width of a person's full-length figure and their removal with the ability to notify about them through the information output device.

The scanner cane automatically notifies the user of obstacles located at different heights within the human height. The disadvantages of this known means include the functional limitations of the device.

Also known is an electronic cane for the blind with motion feedback (CN 108670741, published on October 19, 2018), consisting of a distance measurement module, an interval motion module, a device control unit, and a battery. Additionally, the reed can be equipped with a sensing unit to provide better tactile feedback. The interval motion module reads linear or rotary motion as it travels a certain distance and has a servo drive, a defined range of motion and is equipped with a positioning device. The disadvantages of the known solution can also be attributed to its functional limitations, since it does not allow providing the user with identifying information about the detected object.

The present invention is aimed at overcoming the indicated drawbacks of the prior art and solving the technical problem of expanding the functionality of the portable device for users with vision problems.

Disclosure of a utility model

As will be shown below, the proposed utility model, when used, makes it possible to achieve a technical result, which consists in increasing the degree of user adaptation to the environment. This technical result is a consequence of the implementation in the device of new functions that were not previously presented in the prior art, as well as a consequence of the improvement of the process implemented in the device for informing the user about the results obtained during the interaction of the technical means included in the device.

So, due to the essential features of the formula, through which the recognition of objects is realized, as well as informing the user about such objects (in particular, by means of sound notification and / or text on the display using a raised-point braille font), the proposed device can be used not only blind, but also blind-deaf users, which contributes to their adaptation to social life in society, due to the receipt of more detailed and complete information about the environment.

Accordingly, to achieve the specified technical result, a Portable device for capturing and recognizing visual images is proposed, which is enclosed and fixed in a single housing equipped with a radiator: a power and control board based on a microcontroller (PPU) connected to a single-board computer (OC), associated with a PPU vibration motor and at least one power source, as well as located in the front part of the housing and associated with the PPU: two video cameras, an ultrasonic or laser range finder, a microphone and a light source; control and indication elements located in the upper part of the body and connected with the PPU, and the PPU is equipped with a charge controller connected to the charging connector, an on / off button of the device located on the case and at least one power source and is made with the possibility of command-telemetric exchange data from OK, and OK is equipped with audio, WiFi and Bluetooth interfaces designed to output information to external devices, an interface for connecting an information carrier and is configured to execute program modules: a module for exchanging information with a PPU, a module for detecting objects and calculating a distance, a module for recognizing objects and outputting recognition results; and a module for communicating with external devices, while at least one board from the PPU and OK is equipped with a USB connector.

This embodiment is presented as a preferred embodiment of the utility model. Further in the description, variants are also presented that are intended to develop or clarify the essence of the utility model, but at the same time do not pursue the goal of replacing or excluding essential features. Some of these options are reflected in the form of signs of dependent claims.

Implementation of the utility model

This section of the description provides more detailed information showing the causal relationship between the essential features of the utility model and the above technical result, as well as confirming the possibility of its implementation on the basis of means and methods known from the prior art.

The modern development of the electronic element base makes it possible to implement the joint implementation of many functions in devices of a compact form factor. So, in the prior art, so-called single-board computers have found their widespread use, for example, single-board computers "Raspberry Pi" from the Raspberry Pi Foundation. These devices have relatively miniature dimensions and are used in various fields, including entertainment (for example, displaying network content), and telemetry, Smart Home technology, and robotics. Thus, one of the motives for creating the proposed utility model was the desire to develop a compact device that can be used by a person with disabilities.

In essence, a single-board computer is a basic module equipped with the necessary in modern conditions interfaces for communication with external devices, as well as the ability to program certain functions. Within the framework of this application, a detailed disclosure of the structural and functional diagrams of a single-board computer is not intended, since it is assumed that its execution from the prior art is known to a specialist.

At the same time, it should be noted that the presence of a single-board computer in itself does not allow providing the functions intended by the developer. This requires the combined use of a single-board computer and selected specialized tools depending on the task at hand, loading software modules intended for the interaction of such tools, processing information obtained as a result of their operation, and also issuing information to the consumer based on the results of such processing.

Within the framework of this application, the specified problem is solved by combining a single-board computer and the following specialized tools in a compact case, as well as ensuring the interaction of such tools.

For a more complete understanding of the essence of the utility model, the application materials include explanatory drawings, a brief description of which is given below.

FIG. 1 is a general view of the assembled device.

FIG. 2 is a general view of the device from above.

FIG. 3 is a diagram of the interaction of boards and modules.

FIG. 4 is an explosion diagram of the composition of the device.

FIG. 5 is a view of the arrangement of indicators and buttons of the module for capturing and recognizing visual images.

As shown in FIG. 1-2, the proposed device looks like a compact body, which can be held with one hand. For added convenience, the body is equipped with strap clips, as is usually done in portable video cameras.

Referring to FIG. 3, the proposed solution contains a power and control board (PPU) 1 based on a microcontroller (3), as well as a single-board computer (OC) connected to the PPU 1. As a single-board computer, in terms of the total technical equipment, it is advisable to use Raspberry Pi computers, in particular since the latest versions of these computers are equipped with built-in WiFi and Bluetooth modules.

An ultrasonic or laser range finder 6, a microphone 7, also connected to the OK 2 through the audio capture board 9. Controls 11.1 and indications 11.2 (buttons and LED indicators, respectively) are connected to the PPU 1, in an embodiment, are located on the board 11 connected to the PPU 1. Device also equipped with a vibration motor 12 and a light source (preferably based on LEDs), an on / off button 14 and a power source 15. One or more batteries can be used as a power source. When using several accumulators in the PPU, the balancing board associated with accumulators can be additionally included. From the front side of the case, video cameras 4 and 5 are installed in the device. The video cameras are connected through a camera switching subsystem with OK 2. The switching subsystem can be made in the form of a separate board located in the case or located on the PPU. The device contains a charging connector, a charge controller 16 associated with an on / off button 14 and a power source 15.

The design features of the proposed device are more clearly shown in FIG. 4 (the so-called explosion-diagram), according to which, under the corresponding positions are presented: 1. Power and control board. 2. Single-board computer board 3. Button and indication board 4. Internal radiator. 5. Cover for gasket buttons. 6. Gasket for FLEX buttons. 7. Buttons. 8. Light guides for indicating device status. 9. Housing. 10. Bottom cover. 11. External radiator (back cover). 12. Eyelet for attaching a strap. 13. Protective sapphire crystal for cameras. 14. Protective sapphire crystal for the LED. 15. Light source board (flashlight). 16. Front cover. 17. Rangefinder. 18. Cameras. 19. Camera mounting bar. 20. Power button (on / off). 21. Strap. 22. Battery. 23. Battery clamp. 24. MicroSD card. 25. Vibration motor. 26. Vibrating motor clamp. 27. Plug for connectors. 28. Microphone.

It is also preferable that at least one of the PPU and OK boards (or both boards) be equipped with a USB connector (USB, microUSB or Type-C) or several connectors at once (for example, USB and microUSB or Type-C). The PPU board can also be equipped with a connector. Designed for programming the wet controller. USB connectors can be used, for example, to connect internal devices equipped with a USB interface or to charge the device (for example, using microUSB or Type-C). To connect devices with a wireless interface, WiFi and Bluetooth interfaces, implemented as part of the proposed device, can be used.

FIG. 5 shows a variant of the placement of control buttons and indication means, as well as their functional purpose under the corresponding positions: object recognition button 4, volume up button 6, volume down button 5, flashlight on and off button 7, button for adding new faces 8, programmable button 9 , rangefinder button 10, as well as indication LEDs: Wi-Fi connection indication 1, battery charge status 2, Bluetooth connection status 3.

According to an embodiment, the device provides a dual functionality of the control buttons, in particular:

Object recognition button 4:

- short (less than 2 s) pressing the button recognizes found objects, direction and distance to them;

- long (more than 2 s) pressing and holding the button down - switching on the mode of continuous object recognition; releasing the button - exit the continuous object recognition mode.

Programmable Button 9:

- short (less than 2 s) pressing the button turns on the selected operating mode;

- long-term (more than 2 s) pressing the button allows you to turn off the selected operating mode.

Rangefinder on / off button 10:

- short (less than 2 s) pressing the button turns on the rangefinder mode, in which the vibration level of the body is inversely proportional to the distance to the obstacle;

- long (more than 2 s) pressing the button allows you to turn off the search mode for the connected mobile phone, which allows you to find the phone connected to the device.

The device works as follows.

The power and control board based on the microcontroller carries out constant command-telemetric data exchange between the connected dependent means and switches the connection of the single-board computer, thereby regulating their operation and realizing the purpose of the device - automatic capture and recognition of objects, their identification with subsequent data output: to the display of the mobile phone or braille display, or an audio headset via the corresponding audio output on the single board computer. The device generates data in such a way that the data is displayed on the braille display in the form of tactile text, which informs the user about all the objects around him. In addition, the device has a function to control the device itself via a connected braille display. In general, data and control information are exchanged between the device and the connected device.

Among the dependent boards, according to the order of data transmission and processing, one can distinguish a single-board computer with a multi-core processor, a flashlight board and an ultrasonic or laser rangefinder.

The power and control board is preferably subdivided into switching subsystems for cameras, a subsystem for receiving and processing data from a rangefinder, a subsystem for capturing sound from a microphone, a subsystem of a charger-discharge device, the information from which is transmitted to a single-board computer.

Structurally, the software running on a single-board computer has a modular design and consists of several software modules: the main control module for monitoring the exchange with the power and control board; a module for detecting objects and calculating their distance; face recognition module; module for communication with other assistive devices using the Bluetooth protocol; audio output module.

Each module is designed to perform its function: the module of communication of assistive devices via Bluetooth - performs command-telemetric exchange with other devices, receives and issues commands for them, and also transmits telemetry. Face recognition module - allows the user to add new faces, as well as find and report found matches by comparing information with the database. The module for detecting objects and calculating their distance - solves the problem of detecting a predetermined list of objects using a neural network, followed by building a depth map using a stereopair of the image and determining the distances to objects in the frame.

To obtain a stereo pair of images, the device provides two video cameras, a switching subsystem and a corresponding software module of a single-board computer. In an embodiment, instead of two video cameras, one stereo camera may be installed in the device.

The head module for control, monitoring and exchange is responsible for uninterrupted interaction with the power and control board, launching basic scripts at user commands and transferring execution commands from other devices.

When the device operates in rangefinder mode, telemetric information about the distance is converted into tactile feedback by means of a vibration motor. In this mode, the vibration level of the body can be inversely proportional to the distance to the obstacle.

For example, a visually impaired user turns on the device and plugs in headphones. With the help of buttons located in the upper part of the body, it learns about the presence, location and distance of surrounding objects, adds and recognizes faces, learns about the distance to obstacles using tactile communication. Also, the device automatically notifies the user about what objects and people surround him, reporting this through the audio headset.

A user with both hearing and vision disabilities connects a braille display to the device wirelessly using Bluetooth technology. When moving, all recognized objects and people, as well as road signs will be processed into text and transferred to the Braille display, thereby allowing the user to freely navigate and perceive information from people and objects around him within a radius of 10 meters. Information about the distance to obstacles is transmitted using tactile feedback.

A database containing information about objects can be loaded into the device by placing a storage medium in it - a MicroSD card or by updating via the Internet. This medium can also be loaded with information about the recognized objects or the correlation of information with objects entered by the user (for example, information about the correspondence of a new recognized person with a person). The database can be periodically updated, and the device makes it easy to download updated information. Therefore, in the object recognition mode, the user will have access to information about objects at a distance, he will not need to use the services of third parties, approach objects for tactile identification, which will increase the level of space perception, and will also ensure his safe movement around the surrounding space (for example, when looking for or crossing pedestrian crossings). The face recognition function will allow the user to increase their involvement in the environment.

It should be noted that the achievement of the above in the description of the technical result is a consequence of the cumulative interaction of the means provided in the proposed solution. The combination of the means indicated in the formula in a compact body provides a synergy of the technical effect, which is not manifested when the means are used separately or in another combination.

Claims (4)

1. A portable device for capturing and recognizing visual images, which is enclosed and fixed in a single case equipped with a radiator: a power and control board based on a microcontroller (PPU) connected to a single-board computer (OC), associated with a PPU, a vibration motor and at least one power source, as well as located in the front part of the case and connected to the PPU: two video cameras, a rangefinder, a microphone and a light source; control and indication elements located in the upper part of the body and connected with the PPU, and the PPU is equipped with a charge controller connected to the charging connector, an on / off button of the device located on the case and at least one power source and is made with the possibility of command-telemetric exchange data from OK, and OK is equipped with audio, WiFi and Bluetooth interfaces designed to output information to external devices, an interface for connecting an information carrier and is configured to execute program modules: a module for exchanging information with a PPU, a module for detecting objects and calculating a distance, a module for recognizing objects and outputting recognition results; and a module for communicating with external devices, while at least one board from the PPU and OK is equipped with a USB connector. 2. A portable device according to claim 1, characterized in that the body contains a strap mount, and the cameras and the light source are protected by sapphire crystals. 3. A portable device according to claim 1, characterized in that it is capable of exchanging data with a connected braille display and / or with a mobile phone. 4. A portable device according to claim 1, characterized in that the range finder is ultrasonic or laser.

Images