JP2023049253A - Visually impaired person walk support device, visually impaired person walk support terminal, visually impaired person walk support method and program - Google Patents

Abstract

To provide a visually impaired person walk support device which can support walk of a visually impaired person.SOLUTION: A visually impaired person walk support device according to the present invention comprises: an image acquisition unit which acquires image information including a ground surface; a braille block information extraction unit which analyzes the image information to extract braille block information including information on at least one of the position and size of a braille block; a distance calculation unit which calculates the direction and distance in which the braille block exists on the basis of at least one of the position and size of the braille block; an initial guide information notification unit which notifies a user of initial guide information that notifies of the direction and distance in which the braille block exists; a virtual walk area creation unit which creates a virtual walk area that is sectioned by two line segments extending from the lower end of the image toward the depth direction in the image; a braille block determination unit which determines the type of the extracted braille block; and a guide information notification unit which notifies of guide information on the basis of at least one of the type of the braille block and the virtual walk area.SELECTED DRAWING: Figure 1

Description

The present invention relates to a walking assistance device for the visually impaired, a walking assistance terminal for the visually impaired, a walking assistance method for the visually impaired, and a program.

In order to ensure the safe walking of the visually impaired, guidance blocks for the visually impaired (hereinafter referred to as Braille blocks) are installed in various locations such as sidewalks, stations, and public facilities. A visually impaired person determines the course by reading the shape of the projections of the Braille blocks by feeling the soles of the feet touching the Braille blocks or by using a white cane.

In addition, a guide helper is placed to accompany the visually impaired person, and a service is also provided that reliably supports the walking of the visually impaired person. There is also known a technique for reproducing audio data corresponding to the walking position of a visually impaired person by associating the audio data with the audio data (Patent Document 1).

JP 2014-137752 A

However, since the technique of Patent Document 1 is a technique for reproducing audio data corresponding to the walking position in advance, it is used when a visually impaired person or the like moves to a completely new place because there is no corresponding audio data. Can not.

In addition, when a visually impaired person is in a place where no tactile blocks are installed nearby, there is a problem that it is difficult to move to a place where the tactile blocks are installed.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a walking support device for the visually impaired to assist the visually impaired in walking.

To achieve the above object, the walking support device for the visually impaired of the present invention comprises an image acquisition unit, a braille block information extraction unit, a distance calculation unit, an initial guidance information notification unit, a virtual walking area creation unit, a braille block determination unit, including a guidance information notification unit,
The image acquisition unit acquires image information,
The image information is image information including the ground,
The braille block information extraction unit analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
The distance calculation unit calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image,
The initial guidance information notification unit notifies the user of initial guidance information that notifies the direction and distance of the braille block,
The virtual walking area creating unit creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination unit determines the type of the extracted braille block,
The guidance information notification unit notifies the user of guidance information based on at least one of the type of the braille block and the virtual walking area.

A walking support terminal for the visually impaired of the present invention includes an imaging unit, a communication unit, and an audio output unit,
The imaging unit is capable of capturing an image including the ground,
The communication unit is capable of communicating with the walking support device for the visually impaired of the present invention,
transmitting image information including the image to the walking support device for the visually impaired;
receiving at least one of the initial guidance information and guidance information;
The audio output unit outputs the initial guidance information or the guidance information as audio information.

A walking support method for the visually impaired of the present invention includes an image acquisition step, a braille block information extraction step, a distance calculation step, an initial guidance information notification step, a virtual walking area creation step, a braille block determination step, and a guidance information notification step,
The image acquisition step acquires image information,
The image information is image information including the ground,
The braille block information extraction step analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
the distance calculating step calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image;
The initial guidance information notifying step notifies the user of initial guidance information that notifies the direction and distance of the braille block,
The virtual walking area creating step creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination step determines the type of the extracted braille block,
The guidance information notifying step notifies the user of guidance information based on at least one of the type of the braille block and the virtual walking area.

The program of the present invention is
A program for causing a computer to execute an image acquisition procedure, a braille block information extraction procedure, a distance calculation procedure, an initial guidance information notification procedure, a virtual walking area creation procedure, a braille block determination procedure, and a guidance information notification procedure,
The image acquisition step acquires image information,
The image information is image information including the ground,
The braille block information extraction procedure analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
the distance calculation step calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image;
The initial guidance information notification procedure notifies the user of initial guidance information that notifies the user of the direction and distance of the braille block,
The virtual walking area creating procedure creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination procedure determines the type of the extracted braille block,
The guidance information notification procedure is a program that notifies the user of guidance information based on at least one of the type of the braille block and the virtual walking area.

ADVANTAGE OF THE INVENTION According to this invention, a visually impaired person's walking can be assisted.

FIG. 1 is a block diagram showing an example configuration of a walking support device for the visually impaired according to Embodiment 1. As shown in FIG. FIG. 2 is a block diagram showing an example of the hardware configuration of the walking assistance device for the visually impaired according to the first embodiment. FIG. 3 is a block diagram showing an example configuration of a walking support system for the visually impaired including the walking support device for the visually impaired according to the first embodiment. FIG. 4 is a block diagram showing an example of the hardware configuration of the walking support terminal for the visually impaired according to the first embodiment; FIG. 5 is a flow chart showing an example of processing in the walking support device for the visually impaired according to the first embodiment. FIG. 6 is a schematic diagram for explaining a method of creating a virtual walking area in the walking support device for the visually impaired according to the first embodiment. FIG. 7 is a schematic diagram for explaining an example of guidance information notification in the walking support device for the visually impaired according to the first embodiment. FIG. 8 is a block diagram showing the configuration of an example of the walking support device for the visually impaired according to the second embodiment. FIG. 9 is a flow chart showing an example of processing in the walking support device for the visually impaired according to the second embodiment. FIG. 10 is a block diagram showing the configuration of an example of the walking support device for the visually impaired according to the third embodiment. FIG. 11 is a flow chart showing an example of processing in the walking support device for the visually impaired according to the third embodiment. FIG. 12 is a schematic diagram for explaining a method of determining an avoidable area in the walking support device for the visually impaired according to the third embodiment. FIG. 13 is a block diagram showing an example configuration of a walking support system for the visually impaired including the walking support device for the visually impaired according to the fourth embodiment. FIG. 14 is a flow chart showing an example of processing in the walking support device for the visually impaired according to the fourth embodiment. FIG. 15 is a schematic diagram for explaining a direction correction method in the walking support device for the visually impaired according to the fourth embodiment.

In the present invention, "visually handicapped person" means a person whose visual function is insufficient, and for example, the function and degree thereof are not particularly limited. A visually impaired person is also referred to as, for example, a visually impaired person, a visually impaired person, a blind person, a weakly sighted person, a low vision person, or the like. INDUSTRIAL APPLICABILITY The present invention is particularly useful, for example, in assisting the walking of blind or partially sighted persons who need a white cane (safety cane for the blind) when walking.

In the present invention, the tactile block is a block with protrusions on the surface, which is laid on the ground or floor surface to guide the visually impaired person safely, and is also called a visually impaired person guidance block. The shape and dimensions of the braille block are defined by, for example, the JIS standard (JIS T 9251:2014), and a dot-like block (also referred to as a warning block) in which dot-like projections for calling attention are arranged, A linear block (also referred to as a guiding block) in which linear protrusions are arranged to indicate the direction of walking or to display the inside of the platform. In addition, in the guide block, the longitudinal direction of the linear protrusion means the traveling direction for guiding the visually impaired person.

Next, an embodiment of the present invention will be described with reference to the drawings. The invention is not limited to the following embodiments. In each figure below, the same reference numerals are given to the same parts. In addition, the description of each embodiment can be used with reference to each other's description unless otherwise specified, and the configurations of the respective embodiments can be combined unless otherwise specified.

[Embodiment 1]
A walking support device for the visually impaired according to this embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing an example configuration of a walking support device 10 for the visually impaired according to this embodiment. As shown in FIG. 1, a walking support device 10 for the visually impaired (hereinafter also referred to as "this device 10") includes an image acquisition unit 11, a braille block information extraction unit 12, a distance calculation unit 13, and an initial guidance information notification unit 14. , a virtual walking area creation unit 15 , a braille block determination unit 16 , and a guidance information notification unit 17 . Moreover, although not shown, the device 10 may include, for example, a storage unit.

The device 10 may be, for example, a single device including each of the above units, or may be a device in which each of the above units can be connected via a communication network. Further, the device 10 can be connected to an external device, which will be described later, via a communication network. The communication network is not particularly limited, and a known network can be used, for example, it may be wired or wireless. The communication line network includes, for example, the Internet line, WWW (World Wide Web), telephone line, LAN (Local Area Network), SAN (Storage Area Network), DTN (Delay Tolerant Networking), LPWA (Low Power Wide Area), L5G (Local 5G), etc. Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), local 5G, and LPWA. The wireless communication may be direct communication between devices (Ad Hoc communication), infrastructure communication, or indirect communication via an access point. The device 10 may be incorporated in a server as a system, for example. Further, the device 10 may be, for example, a personal computer (PC, for example, desktop type, notebook type), a smart phone, a tablet terminal, etc. in which the program of the present invention is installed. Furthermore, the device 10 may be in a form of cloud computing, edge computing, or the like, in which at least one of the units is on the server and the other units are on the terminal, for example.

FIG. 2 illustrates a block diagram of the hardware configuration of the device 10. As shown in FIG. The device 10 includes, for example, a CPU 101, a memory 102, a bus 103, a storage device 104, an input device 106, a display device 107, a communication device (communication section) 108, and the like. Each unit of the device 10 is interconnected via a bus 103 by each interface (I/F).

For example, the CPU 101 cooperates with other components by means of a controller (system controller, I/O controller, etc.) and takes charge of overall control of the apparatus 10 . In the device 10, the CPU 101 executes, for example, the program 105 of the present invention and other programs, and reads and writes various information. Specifically, for example, the CPU 101 includes an image acquisition unit 11, a braille block information extraction unit 12, a distance calculation unit 13, an initial guidance information notification unit 14, a virtual walking area creation unit 15, a braille block determination unit 16, and a guidance information notification unit. It functions as part 17 . The device 10 includes a CPU as an arithmetic device, but may be provided with other arithmetic devices such as a GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), etc., or may be provided with a combination of the CPU and these. good.

Bus 103 can also be connected to external devices, for example. The external device includes, for example, a walking support terminal for the visually impaired, an external storage device (external database, etc.), a printer, an external input device, an external display device, an audio output device such as a speaker, an external imaging device such as a camera, and Examples include various sensors such as an acceleration sensor, a geomagnetic sensor, and a direction sensor. The apparatus 10 can be connected to an external network (the communication line network) by, for example, a communication device 108 connected to the bus 103, and can be connected to other apparatuses such as the walking support terminal for the visually impaired via the external network. You can also

The memory 102 is, for example, a main memory (main storage device). When the CPU 101 performs processing, for example, the memory 102 reads various operating programs such as the program 105 of the present invention stored in the storage device 104, which will be described later, and the CPU 101 receives data from the memory 102, Run the program. The main memory is, for example, RAM (random access memory). Also, the memory 102 may be, for example, a ROM (read only memory).

The storage device 104 is also called a so-called auxiliary storage device, for example, in contrast to the main memory (main storage device). As described above, the storage device 104 stores the operating program 105 including the program of the present invention. Storage device 104 may be, for example, a combination of a recording medium and a drive that reads from and writes to the recording medium. The recording medium is not particularly limited, and may be, for example, a built-in type or an external type, and includes HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, memory card, and the like. be done. The storage device 104 may be, for example, a hard disk drive (HDD) in which a recording medium and drive are integrated, and a solid state drive (SSD). For example, when the device 10 includes the storage section, the storage device 104 functions as the storage section.

In the device 10, the memory 102 and the storage device 104 store log information, information acquired from an external database (not shown) or an external device, information generated by the device 10, and data stored when the device 10 executes processing. It is also possible to store various information such as information to be used. At least part of the information may be stored in an external server other than the memory 102 and the storage device 104, or may be distributed and stored in a plurality of terminals using blockchain technology or the like. .

The device 10 further comprises an input device 106 and a display 107, for example. The input device 106 includes, for example, a touch panel, a track pad, a pointing device such as a mouse; a keyboard; an imaging means such as a camera or scanner; a card reader such as an IC card reader or a magnetic card reader; be done. The display 107 is, for example, a display device such as an LED display or a liquid crystal display. In Embodiment 1, the input device 106 and the display 107 are configured separately, but the input device 106 and the display 107 may be configured integrally like a touch panel display.

The device 10 may be capable of communicating with the walking support terminal 20 for the visually impaired, for example. In this case, for example, the entire system including the walking assistance device 10 for the visually impaired and the walking assistance terminal 20 for the visually impaired is also referred to as a walking assistance system for the visually impaired 100 .

A walking support system device 100 for the visually impaired person including the walking support device 10 for the visually impaired person and the walking support terminal 20 for the visually impaired person of the present invention will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the configuration of a walking support system for the visually impaired 100 including the walking support device 10 for the visually impaired and the walking support terminal 20 for the visually impaired according to the present invention. As shown in FIG. 3 , the walking support device 10 for the visually impaired and the walking support terminal 20 for the visually impaired can be connected to each other via a communication network 30 . The communication network 30 is as described above, and the connection between the walking support device 10 for the visually impaired and the walking support terminal 20 for the visually impaired may be wired or wireless, for example. In the walking support system device 100 for the visually impaired, the walking support device 10 for the visually impaired may be, for example, a small device that can be carried by the visually impaired user, or a large device such as a server PC. may In the latter case, for example, the walking support system device 100 for the visually impaired person includes a plurality of walking support terminals 20 for the visually impaired person. After processing, the initial guidance information or the guidance information may be notified (transmitted) to each walking support terminal 20 for the visually impaired.

The visually impaired walking support terminal 20 (hereinafter also referred to as "terminal 20") includes, for example, an imaging unit, a communication unit, and an audio output unit. The imaging unit is capable of capturing an image including the ground. The communication unit is capable of communicating with the walking support device for the visually impaired 10, transmits image information including the image to the walking support device for the visually impaired, and receives at least one of the initial guidance information and the guidance information. The audio output unit outputs the initial guidance information or the guidance information to the user as audio information. The walking support terminal 20 for the visually impaired is not particularly limited as long as it has these configurations, and specific examples include mobile terminals such as mobile phones, smartphones, and tablet terminals; smart watches, smart glasses, and wearable terminals; It may be in the form of a white cane equipped with an imaging device such as a camera, a communication device, a speaker, an earphone, or an audio output device such as a headphone. The walking support terminal 20 for the visually impaired is preferably a glasses-type wearable device from the viewpoint of portability. In addition, when the walking support terminal 20 for the visually impaired is a wearable device, the image captured by the walking support terminal 20 for the visually impaired can capture an area that is substantially the same as the visual field of the visually impaired user. For example, it is possible to more accurately assist the visually impaired in walking.

FIG. 4 illustrates a block diagram of the hardware configuration of the walking support terminal 20 for the visually impaired. As shown in FIG. 4, the visually impaired walking support terminal 20 includes, for example, a CPU 201, a memory 202, a bus 203, a storage device 204, an input device 206, a display 207, a communication device 208, an imaging device 209, an audio output device 210, and the like. Prepare. The imaging device 208 is, for example, an imaging device such as a camera, and the audio output device 210 is, for example, an audio output device such as a speaker, earphone, or headphone. The description of each configuration of the walking assistance device 10 for the visually impaired can be used for the description of each configuration of the walking assistance terminal 20 for the visually impaired. Each unit of the walking support terminal 20 for the visually impaired is connected via a bus 203 by each interface (I/F). In the walking support terminal 20 for the visually impaired, a CPU 201 controls a communication device 208, an imaging device 209, and an audio output device 210 connected via a bus 203, and functions as an imaging unit 21, a communication unit 22, and an audio output unit 23. do. The walking support terminal 20 for the visually impaired may include various sensors capable of detecting the user's movement and movement direction, such as an acceleration sensor, a geomagnetic sensor, and a direction sensor.

Next, an example of the walking support method for the visually impaired according to this embodiment will be described based on the flowchart of FIG. The walking assistance method for the visually impaired according to the present embodiment is implemented as follows, for example, using the walking assistance device 10 for the visually impaired shown in FIGS. The walking support method for the visually impaired according to this embodiment is not limited to the use of the walking support device 10 for the visually impaired shown in FIGS. 1 and 2 . Further, in the following description, a visually impaired person who is the user of the walking assistance device 10 for the visually impaired wears the walking assistance terminal 20 for the visually impaired, which is an eyeglass-type wearable device, on the head as an example. Although illustrated, the present invention is not limited to the following examples.

First, the image acquisition unit 11 acquires image information (S1, image acquisition step). The image information is information of an image including the ground, and is preferably an image including the ground in front of the visually impaired user, for example. The image information may be, for example, a still image or a moving image, but is preferably a moving image from the viewpoint of real-time guidance according to the user's walking. Further, the image acquisition unit 11 may acquire images continuously or intermittently, for example. It is preferable to acquire the images over time, and more preferably to acquire the images over time. The image acquisition unit 11 may, for example, acquire image information captured by an external imaging device connected to the bus 103, or acquire image information captured by a terminal owned by the user via the communication network. It may be acquired by communication. The terminal possessed by the user is, for example, the walking support terminal 20 for the visually impaired.

Next, the braille block information extraction unit 12 analyzes the image information and extracts braille block information (S2, braille block information extraction step). The braille block information includes information on at least one of the position and size of the braille block in the image. The braille block information extraction unit 12 can extract the braille block information by, for example, a general image analysis method for extracting an object from an image. , Braille blocks can be extracted from the image information. As described above, the size and other specifications of the braille blocks are stipulated by the JIS standard, and there are recommended installation standards for the color of the braille blocks and the luminance ratio between the braille blocks and the surroundings. . Therefore, for example, the braille block information can be extracted from the image based on these information of the braille blocks. If the device 10 includes the storage unit, the learned model may be stored in the storage unit or in a database outside the device. In the latter case, the braille block information extraction unit 12 can acquire and use the learned model via the communication network, for example.

Next, the distance calculator 13 calculates the direction and distance of the braille block based on at least one of the position and size of the braille block included in the image information (S4, distance calculation step). For example, the distance calculation unit 13 uses distance conversion information that associates the actual distance with the number of pixels in the image for each angle and position (height) at which the image is captured, so that the braille block can be calculated. For example, the distance conversion information may be stored in the storage unit, or may be stored in an external database. The direction in which the braille block exists is, for example, based on the midpoint of the lower end of the image information, and the pixels from which the braille block is extracted are in front, right front, right, left front, and left of the midpoint. It can be determined by determining where it is located.

Next, the initial guidance information notifying unit 14 notifies the user of initial guidance information indicating the direction and distance of the Braille block (S4, initial guidance information notification step). Specific examples of the initial guidance information include directions and distances in which the Braille blocks are present, such as "There is a Braille block 10 steps or more ahead" and "There is a Braille block 5 meters ahead on the right". Information to notify will be given. The information for notifying the distance may be represented by, for example, the number of steps, or may be represented by the unit of distance. Preferably. Also, the initial guidance information may include, for example, information indicating that the braille block has been extracted (detected). A specific example is information such as "detection of braille block, start guidance". For example, the initial guidance information notification unit 14 notifies (transmits) the initial guidance information to the terminal 20 possessed by the user by communication via the communication network, and the voice output device provided in the terminal 20 causes the terminal 20 to The received initial guidance information is notified (output) by voice. Further, the present invention is not limited to this, and for example, an audio output device such as a speaker connected to the bus 103 of the device 10 may notify (output) the initial guidance information by voice.

For example, when the braille block is not extracted from the image information, the initial guidance information notification unit 14 may notify the user of information instructing the user to search for the braille block. The information instructing the user to search for the braille block is, for example, information prompting the user to take an image of the user's surroundings, such as "Turn the camera left and right." Further, when the terminal of the user is a terminal that is attached to a part of the body such as a wearable device such as glasses, the initial guidance information notification unit 14 may say, "Please shake your head left and right for a while." You may notify the information which instruct|indicates to move the mounted|worn site|part like.

Next, the virtual walking area creating unit 15 creates a virtual walking area in the image (S5, virtual walking area creating step). The virtual walking area is a region in the image that extends in the depth direction from the lower end of the image and is divided by two line segments, and is also called a trace area, for example.

An example of a method for creating the virtual walking area by the virtual walking area creating unit 15 will be described with reference to FIG. FIG. 6 is a photograph showing an example of the image information acquired in S1. As shown in FIG. 6, the virtual walking area creating unit 15 creates, for example, a first line segment 151 and a second line segment 152 extending in the depth direction from the lower end of the image. A region 153 in the central portion of the screen partitioned by a first line segment 151 and a second line segment 152 indicated by oblique lines in FIG. 6 becomes the virtual walking area (trace area). The virtual walking area is not limited to that shown in FIG. 6, and for example, the first line segment 151 and the second line segment 152 may be parallel. Also, the virtual walking area creating unit 15 may create, for example, a line segment 154 indicating distance in the image. Specifically, the virtual walking area creating unit 15 can create a line segment 154 indicating the distance based on the distance conversion information, for example. For example, a plurality of line segments 154 indicating the distance may be created, and the virtual walking area creating unit 15 can create the line segments 154 at predetermined intervals (for example, every 1 m) from the lower end of the image.

Next, the braille block determination unit 16 determines the type of the extracted braille block (S6, braille block determination step). Specifically, for example, the braille block determination unit 16 determines whether the type of the extracted braille block corresponds to a guidance block, a warning block, or an unknown classification. As described above, the Braille blocks can be broadly classified into warning blocks having dot-like projections arranged on their surfaces and guiding blocks having linear projections arranged on their surfaces. The size, height, number, etc. of the are defined by JIS standards. For this reason, the braille block determining unit 16 determines whether the projected shape of the surface of the extracted braille block is either a point-like projection or a linear projection. If it is determined to be a linear projection, it is classified to be a guidance block. In addition, if the braille block determination unit 16 cannot determine the projection shape of the surface of the braille block due to factors such as the braille block existing far away, the braille block being damaged, or the image being unclear, Can be classified as unclassifiable.

Next, the guidance information notification unit 17 notifies guidance information based on at least one of the type of the braille block and the virtual walking area (S7, guidance information notification step). For example, when the extracted Braille block is a guidance block, the guidance information notification unit 17 determines that the guidance block is included in the virtual walking area, and that the longitudinal direction of the linear projection of the guidance block and the movement direction of the user match. to match the direction of movement of the user. Further, for example, when the extracted braille block is at least one of a warning block and an unknown classification, the guidance information notification unit 17 notifies guidance information that notifies the direction and distance of the braille block. For example, the guidance information notification unit 17 notifies (transmits) the guidance information to the terminal 20 possessed by the user by communication via the communication network, and the voice output device provided in the terminal 20 processed by the user The guidance information received by the terminal 20 is notified (output) by voice. Further, the present invention is not limited to this, and for example, an audio output device such as a speaker connected to the bus 103 of the device 10 may notify (output) the guidance information by voice.

A specific example of the processing of S7 will be described with reference to FIG. FIG. 7 is a schematic diagram showing a situation in which the user walks on the guidance block from point A where the warning block is laid, passes point B, and moves to point C where the warning block is laid. In FIG. 7, the white arrow indicated by reference numeral 20 means the user's traveling direction, the area 153 surrounded by the dashed line means the virtual walking area 153, the reference numeral 40A means the warning block, and the reference numeral 40B. means an induction block. First, the user 20 reaches point A shown in FIG. 7 according to the initial guidance information notified in the initial guidance information notification step of S4. Next, the guidance information notifying unit 17 provides guidance information for notifying the direction and distance of the warning block, such as "forward, one step ahead, warning block", for the warning block existing in front of the user 20. Notice. Next, the guidance information about the guidance block 40B existing on the right front of the user 20 at the point A is notified. The guide block 40B is included in the virtual walking area 153, but the traveling direction of the user 20 is forward (upward in the drawing), whereas the longitudinal direction of the linear protrusion of the guide block 40B, that is, the guide block Since the direction guided by 40B is right forward (upper right direction on the drawing), they do not match. Therefore, at the point A, the guidance information notifying unit 17 provides the user with guidance information such as "front right, guidance block" so that the longitudinal direction of the linear projection of the guidance block 40B and the direction of movement of the user match. Notice. According to the notification, the user 20 proceeds right forward on the guidance block 40B and reaches the point B. FIG. At the point B, the guidance block 40B does not exist in front of the user 20, and the guidance block 40B exists in front of the user 20 to the left. At point B, the guiding block 40B is included in the virtual walking area 153, but the traveling direction of the user 20 is forward (upper right direction in the drawing), whereas the linear projection of the guiding block 40B , i.e., the direction guided by the guide block 40B is the left front (upward direction in the drawing), so they do not match. Therefore, at point B, the guidance information notifying unit 17 provides the user with guidance information such as "Left front, guidance block" so that the longitudinal direction of the linear projection of the guidance block 40B and the direction of movement of the user match. Notice. According to the notification, the user 20 proceeds to the left front on the guidance block 40B and reaches the point C. As shown in FIG. Then, the guidance information notifying unit 17 notifies the user 20 of the direction and distance in which the warning block exists, such as "forward, four steps ahead, warning block". do.

For example, when the guidance block moves out of the virtual walking area while the user is walking, the guidance information notification unit 17 corrects the traveling direction of the user so that the guidance block is included in the virtual direction area again. may be notified of guidance information specifying the direction and distance of movement of the user. For example, as a result of the user moving in a direction that deviates to the right from the direction in which the user should normally proceed, when the guidance block moves out of the virtual area, the guidance information notification unit 17 outputs "Left front, guidance block" or " You are deviating to the right. Please take a half step to the left."

Further, the guidance information notification unit 17 may notify the guidance information about a braille block existing at a predetermined distance, for example. The previously specified distance is also called a guidance notification distance, for example, and can be arbitrarily set. One distance may be set, or two or more distances may be set. Specifically, for example, the guidance notification distance is set to 5 m (10 steps ahead), and the guidance block is located 10 m ahead of the user, and the warning block is located 5 m ahead of the user. I will list and explain. In this case, the guidance information notifying unit 17 notifies the Braille block existing 5 m ahead of the user within the guidance notification distance as "forward, 10 steps ahead, warning block", and the user moves forward, When the guidance block is 5 meters ahead of the user, the message "forward, 10 steps ahead, guidance block" is notified. Further, when the guidance notification distance is set to 1 m (2 steps ahead) and 5 m (10 steps ahead), for example, the guidance information notification unit 17 follows the guidance notification distance within the guidance notification distance 5 m ahead of the user. The previously existing braille block is notified as "forward, 10 steps ahead, warning block", and when the user moves forward and the warning block is 1 m ahead of the user, the previously notified warning block is displayed. , again, "Forward, two steps ahead, warning block". By setting the guidance notification distance in this manner, the guidance information can be notified at any timing for each user, and the user can be guided more accurately.

According to the walking support device for the visually impaired according to the present embodiment, the initial guidance information notification unit can guide the visually impaired person to the surrounding braille blocks, and the guidance information notification unit enables the visually impaired person to walk without deviating from the guidance block. It is possible to guide the direction of movement of visually impaired people.

[Embodiment 2]
Embodiment 2 is another example of the walking support device for the visually impaired according to the present invention.

The walking support device for the visually impaired according to the present embodiment has the same configuration as the walking support device for the visually impaired 10 according to the first embodiment except that it includes an obstacle determination unit. , and the description thereof can be used. The walking support device 10A for the visually impaired according to the present embodiment includes, for example, an obstacle determination unit. The obstacle determination unit analyzes the image information to determine whether there is an obstacle on the braille block, The guidance information notification unit notifies the user of the presence of the obstacle as the guidance information when it is determined that there is an obstacle on the braille block.

FIG. 8 is a block diagram showing an example configuration of the walking support device 10A for the visually impaired according to this embodiment. As shown in FIG. 8, the walking support device for the visually impaired 10A includes an obstacle determination unit 18 in addition to the configuration of the walking support device for the visually impaired 10 of the first embodiment. The hardware configuration of the walking support device 10A for the visually impaired person is the hardware configuration of the walking support device 10 for the visually impaired person shown in FIG. 1 is the same except that it has the configuration of the walking support device 10A for the visually impaired.

Next, the walking support method for the visually impaired according to this embodiment will be described with reference to the flowchart of FIG. The walking assistance method for the visually impaired according to the present embodiment can be implemented using, for example, the walking assistance device for the visually impaired according to the embodiment shown in FIG. 8 . The walking support method for the visually impaired according to the present invention is not limited to the use of the walking support device for the visually impaired 10A.

First, S1 to S6 are performed in the same manner as S1 to S6 in the walking support method for the visually impaired according to the first embodiment.

Next, the obstacle determination unit 18 analyzes the image information and determines whether or not there is an obstacle on the braille block (S11, obstacle determination step). For example, when an object other than the braille block is extracted on the braille block extracted in step S2 by a general image analysis method for extracting an object from an image, the obstacle determination unit 18 detects an obstacle on the braille block. determine that something exists. Further, when the obstacle determination unit 18 determines that there is an obstacle on the braille block, the obstacle determination unit 18 may, for example, further classify the type of the obstacle. The classification of the types of obstacles is, for example, whether they are humans (e.g., pedestrians, people standing on Braille blocks, people in wheelchairs, people on bicycles, etc.) or other obstacles. is given.

Next, the guidance information notification unit 17 notifies guidance information (S7, guidance information notification step). For example, when it is determined in S11 that there is no obstacle (S11, No), the guidance information notification unit 17 notifies the user of the guidance information in the same manner as in S7 of the first embodiment. On the other hand, if it is determined in S11 that there is an obstacle (S11, Yes), the guidance information notifying unit 17 provides the guidance information to the user, such as "There is an obstacle ahead." The existence of an obstacle is notified (S7A). Further, when the types of obstacles are classified in S11, the types of obstacles may also be notified to the user, for example, "There is a pedestrian in front". Further, the guidance information notification unit 17 may notify the distance and direction of the obstacle together, for example, based on the distance and direction of the braille block on which the obstacle is superimposed.

In this embodiment, S11 is performed after S6, but the walking assistance method for the visually impaired according to the present invention is not limited to this. For example, S11 may be performed in a process downstream of S2, may be performed upstream or downstream of S3 to S7, or may be performed in parallel with S3 to S7.

According to the walking support device for the visually impaired of the present embodiment, for example, the obstacle determination unit can determine the presence or absence of an obstacle on the braille block, and if the obstacle exists, the user can be notified of the presence of the obstacle. . Therefore, according to the walking support device for the visually impaired according to the present embodiment, the user can know in advance the presence of an obstacle on the tactile block, and for example, the user is prevented from colliding with the obstacle on the tactile block. can.

[Embodiment 3]
Embodiment 3 is another example of the walking support device for the visually impaired according to the present invention.

The walking support device for the visually impaired according to the present embodiment has the configuration of the walking support device for the visually impaired 10 according to Embodiment 1, except that it includes an obstacle determination section and an avoidance area determination section. It is the same as the pedestrian walking support device 10, and the description thereof can be used. The visually impaired walking support device 10B of the present embodiment includes, for example, an obstacle determination unit and an avoidance area determination unit,
The obstacle determination unit analyzes the image information to determine the presence or absence of an obstacle on the braille block,
When it is determined that there is an obstacle on the braille block, the avoidance area determination unit analyzes the image information to determine whether or not there is an avoidable area in which the user can avoid the obstacle,
The guidance information notification unit
if it is determined that there is no avoidable area, notifying the user that an obstacle has been extracted;
If it is determined that there is the avoidable area,
Notifying the user of avoidance guidance information designating the movement direction and movement distance of the user so that the user advances into the avoidable area;
When the obstacle is no longer extracted, guidance information specifying the moving direction and moving distance of the user is notified so that the guidance block is included in the virtual walking area.

FIG. 10 is a block diagram showing an example configuration of a walking support device 10B for the visually impaired according to this embodiment. As shown in FIG. 10, the visually impaired walking support device 10B includes an obstacle determination unit 18 and an avoidance area determination unit 19 in addition to the configuration of the visually impaired walking support device 10 of the first embodiment. The hardware configuration of the walking assistance device 10B for the visually impaired person is that in the hardware configuration of the walking assistance device 10 for the visually impaired person shown in FIG. 1, except that it has the configuration of the visually impaired walking support device 10B.

Next, the walking support method for the visually impaired according to this embodiment will be described with reference to the flowchart of FIG. 11 . The walking assistance method for the visually impaired according to the present embodiment can be carried out using, for example, the walking assistance device 10B for the visually impaired according to the embodiment shown in FIG. The walking support method for the visually impaired according to the present invention is not limited to the use of the walking support device 10B for the visually impaired.

First, S1 to S6 are performed in the same manner as S1 to S6 in the walking support method for the visually impaired according to the first embodiment.

Next, the obstacle determination unit 18 analyzes the image information and determines whether or not there is an obstacle on the braille block (S11, obstacle determination step). For example, when an object other than the braille block is extracted on the braille block extracted in step S2 by a general image analysis method for extracting an object from an image, the obstacle determination unit 18 detects an obstacle on the braille block. determine that something exists. Further, when the obstacle determination unit 18 determines that there is an obstacle on the braille block, the obstacle determination unit 18 may, for example, further classify the type of the obstacle. Classification of the types of obstacles is, for example, pedestrians or other obstacles.

For example, when it is determined in S11 that there is no obstacle (S11, No), the guidance information notification unit 17 notifies the user of the guidance information in the same manner as in S7 of the first embodiment (S7, guidance information notification process).

When it is determined that there is an obstacle on the braille block (S11, Yes), the avoidance area determination unit 19 analyzes the image information and determines whether or not there is an avoidable area in which the user can avoid the obstacle. Judgment (S21, avoidance region judgment step). Specifically, the avoidance region determination unit 19 detects, for example, a region where the user can walk, such as a boundary portion of a sidewalk, from the image information. The boundary portion of the sidewalk can be detected using, for example, a known edge detection method or a learned model created by machine learning. Next, for example, the avoidance region determination unit 19 measures the distance between the obstacle object boundary extracted in S11 and the walking boundary portion, and if the distance is equal to or greater than a predetermined value, the avoidable region is determined. If it is determined that there is, and the distance is less than a predetermined value, it can be determined that there is no avoidable area. An arbitrary value can be set as the predetermined value according to conditions such as the user's physique, stride length, and surrounding conditions, for example. Further, for example, when there are a plurality of avoidable areas, the avoidable area determination unit 19 may determine a wider area as the avoidable area, or determine the roadway by image analysis, and may be determined to be the avoidable area. For example, when the detection of the boundary portion of the sidewalk fails, or when the measurement of the distance between the boundary of the obstacle object and the boundary portion of the walking fails, the avoidance area determination unit 19 determines the avoidance area. It may be determined that the presence/absence of the available area cannot be determined.

The processing of S21 will be described more specifically with reference to FIG. FIGS. 12A and 12B are photographs showing a case where a bicycle as an obstacle is extracted on the braille block in the image information. First, the case of FIG. 12A will be described. The avoidance area determination unit 19 performs edge detection on the image information, and detects a boundary portion 191 of the sidewalk as indicated by a white dashed line 191 . Next, as indicated by the white arrow 192 in FIG. 12A, the distance from the boundary area 193 of the obstacle (bicycle) indicated by the solid white line 193 to the boundary portion 191 of the sidewalk is measured. It is determined whether or not it is greater than or equal to the value, and the area between the boundary area 193 of the obstacle and the boundary portion 191 of the sidewalk is determined to be the avoidable area. Next, the case of FIG. 12B will be described. The avoidance area determination unit 19 performs edge detection on the image information, and detects a boundary portion 191 of the sidewalk as indicated by a white dashed line 191 . In the image information shown in FIG. 12B, the edge of the sidewalk on the left side of the drawing is cut off, and the distance from the boundary area 193 of the obstacle (bicycle) indicated by the solid white line to the boundary portion 191 of the sidewalk cannot be measured. Therefore, the avoidance area determination unit 19 determines that the existence of the avoidable area cannot be determined.

If it is determined in S21 that there is no avoidable area (S21, No), the guidance information notification unit 17, for example, provides the guidance information such as "There is an obstacle ahead" to inform the user of the obstacle. Notify the existence of an object (S7BA. If the type of obstacle is classified in S11, the type of obstacle is also notified to the user, for example, "There is a pedestrian in front of you." The guidance information notification unit 17 may also notify the distance and direction of the obstacle based on the distance and direction of the braille block on which the obstacle overlaps, for example.

Further, if it is determined in S21 that there is no avoidable area, or if the presence or absence of the avoidable area cannot be determined, the guidance information notification unit 17 may, for example, prompt the user to search for an avoidable area. You may notify the information to instruct. The instruction to the user to search for the avoidable area is, for example, information prompting the user to take an image of the surroundings of the user, such as "Turn the camera left and right." Further, when the terminal of the user is a terminal that is worn on a part of the body such as a wearable device such as glasses, the guidance information notification unit 17 may display a message such as "please shake your face left and right for a while". Information instructing to move the attached part may be notified. As a result, since image information around the extracted obstacle can be obtained, for example, S21 can be performed again to determine whether or not there is an avoidable area.

When it is determined in S21 that there is an avoidable area (S21, Yes), the guidance information notification unit 17 notifies the user of the avoidance guidance information, for example (S7B). The avoidance guidance information includes, for example, information specifying a moving direction of the user so that the user advances into the avoidable area, and information specifying the user's moving direction when the obstacle is no longer extracted. and information specifying the user's direction of movement and distance of movement as included in the . Specifically, for example, as shown in S12(A), the case where it is determined that the avoidable area is located on the left front of the user will be described as an example. First, the guidance information notification unit 17 notifies the user of the traveling direction so that the user can proceed to the avoidable area (for example, "Look forward left"). At this time, the guidance information notification unit 17 may also notify the presence or absence of an obstacle, for example (for example, "There is an obstacle ahead. Please look to the left front"). Next, the guidance information notification unit 17 notifies the user of information that guides the user to proceed forward (for example, "Please proceed forward"). Then, the guidance information notifying unit 17 provides information specifying the user's movement direction and movement distance so that the guidance block is included in the virtual walking area when the obstacle is no longer extracted from the image information. (eg, "Right forward, 4 steps, guidance block"). At this time, the guidance information notification unit 17 may also notify, for example, that the obstacle has been successfully avoided (for example, "Obstacle avoided. Forward right, 4 steps, guidance block").

According to the walking support device for the visually impaired according to the present embodiment, for example, the obstacle determination unit determines the presence or absence of an obstacle on the braille block, and if an obstacle exists, the avoidable area is determined by the avoidance area determination unit. The presence or absence can be determined, and the user can be notified of the avoidance guidance information. Therefore, according to the walking support device for the visually impaired of this embodiment, the walking of the user can be supported so that the user can avoid obstacles on the tactile blocks without colliding with them. Avoid colliding with obstacles above.

[Embodiment 4]
Embodiment 4 is another example of the walking support system for the visually impaired according to the present invention.

A walking support system device 100C for the visually impaired according to this embodiment will be described with reference to FIG. FIG. 13 is a block diagram showing an example configuration of a walking support system device for the visually impaired person 100C including the walking support device for the visually impaired person 10C and the walking assistance terminal for the visually impaired person 20A.

The walking support device for the visually impaired according to the present embodiment is the same as the walking support device for the visually impaired according to the first embodiment, except that in addition to the configuration of the walking support device for the visually impaired 10 according to the first embodiment, a direction correction information acquisition unit is included. 10, and the description thereof can be used. The visually impaired walking support device 10C of the present embodiment includes, for example, a direction correction information acquisition unit,
The direction correction information acquisition unit acquires direction correction information,
The direction correction information includes imaging direction information and traveling direction information,
The imaging direction information is information of an imaging direction in which the user captured an image,
The traveling direction information is information on the direction in which the user travels,
The initial guidance information notifying unit corrects the direction in which the braille block exists in the initial guidance information based on the direction correction information,
The guidance information notifying unit corrects at least one of a direction in which the braille block exists and a moving direction of the user in the guidance information based on the direction correction information.

As shown in FIG. 13, the walking support device for the visually impaired 10C includes a direction correction information acquisition unit 111 in addition to the configuration of the walking support device for the visually impaired 10 of the first embodiment. The hardware configuration of the walking assistance device 10C for the visually impaired person is the hardware configuration of the walking assistance device 10 for the visually impaired person shown in FIG. , except that it has the configuration of the walking support device for the visually impaired 10C.

The walking support terminal for the visually impaired according to this embodiment includes, in addition to the configuration of the walking support terminal for the visually impaired 20 according to Embodiment 1, an imaging direction measuring unit and a traveling direction measuring unit,
The imaging direction measurement unit measures imaging direction information,
The imaging direction information is information of an imaging direction in which the image is captured by the imaging unit,
The traveling direction measurement unit measures traveling direction information of the user,
The traveling direction information is information on the direction in which the user travels,
The communication unit is capable of communicating with the visually impaired walking support device 10C,
The imaging direction information and traveling direction information are transmitted to the walking support device for the visually impaired.

As shown in FIG. 13, the walking support terminal 20A for the visually impaired includes an imaging direction measurement unit 24 and a traveling direction measurement unit 25 in addition to the configuration of the walking support terminal 20 for the visually impaired according to the first embodiment. The hardware configuration of the walking support terminal 20A for the visually impaired person is the hardware configuration of the walking support terminal 20 for the visually impaired person shown in FIG. , except that it has the configuration of the walking support terminal 20A for the visually impaired. The imaging direction measuring unit and traveling direction measuring unit can use known direction measuring means capable of measuring the imaging direction and traveling direction of an image, such as the above-described geomagnetic sensor and direction sensor.

Next, the walking assistance method for the visually impaired according to this embodiment will be described with reference to the flowchart of FIG. 14 . The walking support method for the visually impaired according to the present embodiment can be implemented using, for example, the walking support system for the visually impaired according to the present embodiment 100C shown in FIG. The walking support method for the visually impaired according to the present invention is not limited to the use of the walking support system for the visually impaired 100C. In the following description, a visually impaired user wears a glasses-type wearable device (visually impaired person walking support terminal 20A) including a first geomagnetic sensor as the image pickup direction measurement unit 24 on the head, and wears it on the waist. , the case where the second geomagnetic sensor is mounted as the traveling direction measuring unit 25 will be described as an example, but the present invention is not limited to the following example.

First, S1 to S3 are performed in the same manner as S1 to S3 in the walking support method for the visually impaired according to the first embodiment.

Next, the direction correction information acquisition unit 111 acquires direction correction information (S41, direction correction information acquisition step). The direction correction information includes, for example, imaging direction information and traveling direction information. The imaging direction information is, for example, information about the imaging direction in which the user has captured an image, and is also referred to as the user's line-of-sight direction. Further, the traveling direction information is, for example, information about the direction in which the user travels, and is also called the direction of the user's body. In the present embodiment, since the visually impaired person walking support terminal 20A having the imaging device is attached to the user's head, the direction correction information acquisition unit 111 may be obtained by, for example, the terminal 20A attached to the user's head. from the first geomagnetic sensor 24 of the horizontal direction is acquired as the imaging direction information.
In addition, since the second geomagnetic sensor 25 is attached to the waist of the user as the traveling direction measuring unit 25, the direction correction information acquisition unit 111 can detect, for example, the second geomagnetic sensor 25 attached to the waist of the user. , the horizontal direction information is acquired as the traveling direction information. The direction correction information acquisition unit 111 may acquire the direction correction information, for example, when the braille block is extracted from the image information.

The initial guidance information notifying unit 14 corrects the direction in which the braille block exists in the initial guidance information based on the direction correction information, and notifies the initial guidance information (S4, initial guidance information notification step). Specifically, the initial guidance information notification unit 14, for example, determines whether or not the imaging direction information included in the direction correction information matches the traveling direction information. Correct the direction in which the braille block exists. A specific example of the correction will be described later.

Next, S5 to S6 are carried out in the same manner as S5 to S6 in the walking assistance method for the visually impaired according to the first embodiment.

Next, based on the direction correction information, the guidance information notification unit 17 corrects at least one of the direction in which the braille block exists and the direction in which the user moves in the guidance information, and notifies the guidance information (S7, guidance information notification process). Specifically, for example, the guidance information notification unit 17 determines whether or not the imaging direction information included in the direction correction information matches the traveling direction information. and/or the direction of movement of the user. The correction will be described later.

A specific example of direction correction in S4 and S7 will be described in detail with reference to FIG. FIGS. 15A to 15C are schematic diagrams showing a state of looking down on a user who uses the walking assistance method for the visually impaired according to the present invention. FIG. 15(A) shows a state in which a braille block 40 exists in front of the user's traveling direction (upward in FIG. 15). In this case, for example, the initial guidance information notification unit 14 and the guidance information notification unit 17 do not correct the direction because the imaging direction information and the traveling direction information match, and do not correct the direction. ” or other initial guidance information or guidance information. On the other hand, FIG. 15(B) shows a state in which the user's traveling direction is forward (upward in FIG. 15) and the braille block 40 is present right in front of the user (upper right in FIG. 15). In this case, as shown in FIG. 15(B), the imaging direction in which an image from which the braille block 40 can be detected is right front of the user, and thus the imaging direction does not match the user's traveling direction. When the direction in which the braille block 40 exists is notified with reference to the imaging direction, it is "forward", but the braille block actually exists in the right front of the user. Therefore, the initial guidance information notification unit 14 and the guidance information notification unit 17 correct the direction in which the braille block 40 exists in the initial guidance information or the guidance information, and use the user's traveling direction instead of the imaging direction as a reference, Notifies initial guidance information or guidance information such as "There is a braille block on the right front". This allows the user to more intuitively understand the direction in which the braille blocks are present, as shown in FIG. 15(C).

In the present embodiment, the case where the walking support terminal 20A for the visually impaired has the first and second geomagnetic sensors as the imaging direction measuring section and the traveling direction measuring section has been described as an example. The configuration of the visually impaired walking support system device 100C is not limited to this. The walking support terminal 10C for the visually impaired according to the present embodiment only needs to acquire the imaging direction information and the traveling direction information as the direction correction information. The imaging direction information and the traveling direction information may be acquired, or the walking support device for the visually impaired person 10C of the present invention includes a direction sensor, and the direction sensor measures and acquires the imaging direction information and the traveling direction information. may

When the user walks, the imaging direction of the image (the direction of the face) does not always match the direction of travel (the direction of the body). According to the walking support system device for the visually impaired person 100C of the present embodiment, the direction in which the user is guided can be corrected based on the imaging direction of the image acquired by the direction correction information acquisition unit and the traveling direction. You can more intuitively understand the direction in which the tactile blocks exist.

[Embodiment 5]
The program of the present embodiment is a program for causing a computer to execute each step of the walking assistance method for the visually impaired. Specifically, the program of the present embodiment causes a computer to perform an image acquisition procedure, a braille block information extraction procedure, a distance calculation procedure, an initial guidance information notification procedure, a virtual walking area creation procedure, a braille block determination procedure, and a guidance information notification procedure. It is a program to run.

The image acquisition step acquires image information,
The image information is image information including the ground,
The braille block information extraction procedure analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
the distance calculation step calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image;
The initial guidance information notification procedure notifies the user of initial guidance information that notifies the user of the direction and distance of the braille block,
The virtual walking area creating procedure creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination procedure determines whether the type of the extracted braille block corresponds to a guidance block, a warning block, or an unknown classification,
The guidance information notification procedure includes:
When the extracted braille block is a guiding block, the guiding block is included in the virtual walking area, and the movement of the user is performed so that the longitudinal direction of the linear projection of the guiding block matches the direction of movement of the user. Notify guidance information specifying the direction,
If the extracted braille block is at least one of a warning block and an unknown classification, guidance information is provided to notify the direction and distance of the braille block.

Further, the program of the present embodiment causes the computer to function as an image acquisition procedure, a braille block information extraction procedure, a distance calculation procedure, an initial guidance information notification procedure, a virtual walking area creation procedure, a braille block determination procedure, and a guidance information notification procedure. It can also be called a program.

The program of the present embodiment can use the description of the walking support device for the visually impaired and the walking support method for the visually impaired of the present invention. For each procedure described above, for example, "procedure" can be read as "processing". Moreover, the program of this embodiment may be recorded on a computer-readable recording medium, for example. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples thereof include random access memory (RAM), read-only memory (ROM), hard disk (HD), optical disk, floppy (registered trademark) disk (FD), and the like.

Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes can be made to the configuration and details of the present invention within the scope of the present invention that can be understood by those skilled in the art.

<Appendix>
Some or all of the above-described embodiments can be described as in the following appendices, but are not limited to the following.
(Appendix 1)
including an image acquisition unit, a braille block information extraction unit, a distance calculation unit, an initial guidance information notification unit, a virtual walking area creation unit, a braille block determination unit, and a guidance information notification unit;
The image acquisition unit acquires image information,
The image information is image information including the ground,
The braille block information extraction unit analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
The distance calculation unit calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image,
The initial guidance information notification unit notifies the user of initial guidance information that notifies the direction and distance of the braille block,
The virtual walking area creating unit creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination unit determines the type of the extracted braille block,
The walking support device for the visually impaired, wherein the guidance information notification unit notifies the user of guidance information based on at least one of the type of the braille block and the virtual walking area.
(Appendix 2)
The braille block determination unit determines whether the type of the extracted braille block corresponds to a guidance block, a warning block, or an unknown classification,
The guidance information notification unit
When the extracted braille block is a guiding block, the guiding block is included in the virtual walking area, and the movement of the user is performed so that the longitudinal direction of the linear projection of the guiding block matches the direction of movement of the user. Notify guidance information specifying the direction,
1. The walking support device for the visually impaired according to appendix 1, wherein when the extracted braille block is at least one of a warning block and an unknown classification, guidance information that notifies the direction and distance of the presence of the braille block is provided.
(Appendix 3)
3. The walking support device for the visually impaired according to appendix 1 or 2, wherein the initial guidance information notifying unit notifies the user of information instructing the user to search for the braille block when the braille block is not extracted from the image information.
(Appendix 4)
4. The walking support device for the visually impaired according to any one of Appendices 1 to 3, wherein the guidance information notification unit notifies the guidance information for a Braille block existing at a predetermined distance.
(Appendix 5)
including an obstacle determination unit,
The obstacle determination unit analyzes the image information to determine the presence or absence of an obstacle on the braille block,
5. The visually impaired person according to any one of attachments 1 to 4, wherein, when it is determined that there is an obstacle on the braille block, the guidance information notification unit notifies the user of the existence of the obstacle as the guidance information. Walking support device.
(Appendix 6)
including an obstacle determination unit and an avoidance area determination unit,
The obstacle determination unit analyzes the image information to determine the presence or absence of an obstacle on the braille block,
When it is determined that there is an obstacle on the braille block, the avoidance area determination unit analyzes the image information to determine whether or not there is an avoidable area in which the user can avoid the obstacle,
The guidance information notification unit
if it is determined that there is no avoidable area, notifying the user that an obstacle has been extracted;
If it is determined that the avoidable area exists, notifying the user of avoidance guidance information;
The avoidance guidance information includes information designating a moving direction of the user so that the user advances into the avoidable area, and information specifying the movement direction of the user when the obstacle is no longer extracted, and the guidance block is included in the virtual walking area. 6. The walking support device for the visually impaired according to any one of Appendices 1 to 5, including information designating the user's movement direction and movement distance so that the user's movement direction and movement distance are specified.
(Appendix 7)
When the avoidable area determination unit determines that the avoidable area does not exist or cannot determine whether or not the avoidable area exists, the guidance information notification unit instructs the user to search for the avoidable area. The walking support device for the visually impaired according to appendix 6, which notifies information.
(Appendix 8)
including a direction correction information acquisition unit,
The direction correction information acquisition unit acquires direction correction information,
The direction correction information includes imaging direction information and traveling direction information,
The imaging direction information is information of an imaging direction in which the user captured an image,
The traveling direction information is information on the direction in which the user travels,
The initial guidance information notifying unit corrects the direction in which the braille block exists in the initial guidance information based on the direction correction information,
8. The walking for the visually impaired according to any one of Appendices 1 to 7, wherein the guidance information notifying unit corrects at least one of a direction in which the braille block exists and a moving direction of the user in the guidance information based on the direction correction information. support equipment.
(Appendix 9)
including an imaging unit, a communication unit, and an audio output unit,
The imaging unit is capable of capturing an image including the ground,
The communication unit is capable of communicating with the walking support device for the visually impaired according to any one of Appendices 1 to 8,
transmitting image information including the image to the walking support device for the visually impaired;
receiving at least one of the initial guidance information and guidance information;
A walking support terminal for the visually impaired, wherein the audio output unit outputs the initial guidance information or the guidance information as audio information.
(Appendix 10)
including an imaging direction measuring unit and a traveling direction measuring unit,
The imaging direction measurement unit measures imaging direction information,
The imaging direction information is information of an imaging direction in which the image is captured by the imaging unit,
The traveling direction measurement unit measures traveling direction information of the user,
The traveling direction information is information on the direction in which the user travels,
The communication unit is capable of communicating with the walking support device for the visually impaired according to Supplementary Note 8,
The walking support terminal for the visually impaired according to appendix 9, which transmits the imaging direction information and the traveling direction information to the walking support device for the visually impaired.
(Appendix 11)
including a server and a terminal,
the server and the terminal are capable of communicating with each other;
The server is the walking support device for the visually impaired according to any one of Appendices 1 to 8,
A walking support system for the visually impaired, wherein the terminal is the walking support terminal for the visually impaired according to Appendix 9 or 10.
(Appendix 12)
including an image acquisition process, a braille block information extraction process, a distance calculation process, an initial guidance information notification process, a virtual walking area creation process, a braille block determination process, and a guidance information notification process,
The image acquisition step acquires image information,
The image information is image information including the ground,
The braille block information extraction step analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
the distance calculating step calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image;
The initial guidance information notifying step notifies the user of initial guidance information that notifies the direction and distance of the braille block,
The virtual walking area creating step creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination step determines the type of the extracted braille block,
The guidance information notifying step notifies the user of the guidance information based on at least one of the type of the braille block and the virtual walking area.
(Appendix 13)
The braille block determination step determines whether the type of the extracted braille block corresponds to a guidance block, a warning block, or an unknown classification,
The guidance information notification step includes:
When the extracted braille block is a guiding block, the guiding block is included in the virtual walking area, and the movement of the user is performed so that the longitudinal direction of the linear projection of the guiding block matches the direction of movement of the user. Notify guidance information specifying the direction,
13. The walking support method for the visually impaired according to appendix 12, wherein, when the extracted braille block is at least one of a warning block and an unknown classification, guidance information that notifies the direction and distance of the presence of the braille block is provided.
(Appendix 14)
14. The walking support method for the visually impaired according to appendix 12 or 13, wherein the initial guidance information notification step notifies information instructing the user to search for the braille block if the braille block is not extracted from the image information.
(Appendix 15)
15. The walking support method for the visually impaired according to any one of Appendices 12 to 14, wherein the guidance information notifying step notifies the guidance information for a Braille block existing at a predetermined distance.
(Appendix 16)
including an obstacle determination step;
The obstacle determination step analyzes the image information to determine the presence or absence of an obstacle on the braille block,
16. The visually impaired person according to any one of Appendices 12 to 15, wherein, in the guidance information notification step, when it is determined that there is an obstacle on the Braille block, the presence of the obstacle is notified to the user as the guidance information. Walking assistance method.
(Appendix 17)
including an obstacle determination step and an avoidance area determination step;
The obstacle determination step analyzes the image information to determine the presence or absence of an obstacle on the braille block,
In the avoidance area determination step, when it is determined that there is an obstacle on the braille block, the image information is analyzed to determine whether or not there is an avoidable area where the user can avoid the obstacle,
The guidance information notification step includes:
if it is determined that there is no avoidable area, notifying the user that an obstacle has been extracted;
If it is determined that the avoidable area exists, notifying the user of avoidance guidance information;
The avoidance guidance information includes information designating a moving direction of the user so that the user advances into the avoidable area, and information specifying the movement direction of the user when the obstacle is no longer extracted, and the guidance block is included in the virtual walking area. 17. The walking support method for the visually impaired according to any one of Appendices 12 to 16, including information designating the user's movement direction and movement distance so that the user's movement direction and movement distance are specified.
(Appendix 18)
When the avoidable area determination step determines that the avoidable area does not exist or the presence or absence of the avoidable area cannot be determined, the guidance information notification step instructs the user to search for the avoidable area. 18. The walking assistance method for the visually impaired according to appendix 17, wherein information is notified.
(Appendix 19)
including a direction correction information acquisition step;
The direction correction information acquisition step acquires direction correction information,
The direction correction information includes imaging direction information and traveling direction information,
The imaging direction information is information of an imaging direction in which the user captured an image,
The traveling direction information is information on the direction in which the user travels,
The initial guidance information notifying step corrects the direction in which the braille block exists in the initial guidance information based on the direction correction information,
19. The visually impaired walking according to any one of Appendices 12 to 18, wherein the guidance information notification step corrects at least one of a direction in which the braille blocks exist in the guidance information and a moving direction of the user based on the direction correction information. how to help.
(Appendix 20)
A program for causing a computer to execute an image acquisition procedure, a braille block information extraction procedure, a distance calculation procedure, an initial guidance information notification procedure, a virtual walking area creation procedure, a braille block determination procedure, and a guidance information notification procedure,
The image acquisition step acquires image information,
The image information is image information including the ground,
The braille block information extraction procedure analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
the distance calculation step calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image;
The initial guidance information notification procedure notifies the user of initial guidance information that notifies the user of the direction and distance of the braille block,
The virtual walking area creating procedure creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination procedure determines the type of the extracted braille block,
The program, wherein the guidance information notification procedure notifies the user of the guidance information based on at least one of the type of the braille block and the virtual walking area.
(Appendix 21)
The Braille block determination procedure determines whether it corresponds to a guidance block, a warning block, or an unknown classification,
The guidance information notification procedure includes:
When the extracted braille block is a guiding block, the guiding block is included in the virtual walking area, and the movement of the user is performed so that the longitudinal direction of the linear projection of the guiding block matches the direction of movement of the user. Notify guidance information specifying the direction,
21. The program according to appendix 20, wherein if the extracted braille block is at least one of a warning block and an unknown classification, the program notifies guidance information that notifies the direction and distance of the braille block.
(Appendix 22)
22. The program according to appendix 20 or 21, wherein the initial guidance information notification procedure notifies information instructing the user to search for the braille block if the braille block is not extracted from the image information.
(Appendix 23)
23. The program according to any one of Appendices 20 to 22, wherein the guidance information notification procedure notifies the guidance information for a Braille block existing at a predetermined distance.
(Appendix 24)
including an obstacle determination procedure;
The obstacle determination procedure analyzes the image information to determine the presence or absence of an obstacle on the braille block,
24. The program according to any one of appendices 20 to 23, wherein the guide information notification procedure notifies the user of the existence of the obstacle as the guide information when it is determined that there is an obstacle on the braille block.
(Appendix 25)
Including an obstacle determination procedure and an avoidance area determination procedure,
The obstacle determination procedure analyzes the image information to determine the presence or absence of an obstacle on the braille block,
The avoidance area determination procedure includes, when it is determined that an obstacle exists on the braille block, analyzing the image information to determine whether or not there is an avoidable area in which the user can avoid the obstacle,
The guidance information notification procedure includes:
if it is determined that there is no avoidable area, notifying the user that an obstacle has been extracted;
If it is determined that the avoidable area exists, notifying the user of avoidance guidance information;
The avoidance guidance information includes information designating a moving direction of the user so that the user advances into the avoidable area, and information specifying the movement direction of the user when the obstacle is no longer extracted, and the guidance block is included in the virtual walking area. 25. The program according to any one of the appendices 20 to 24, including information specifying the user's movement direction and distance to be moved.
(Appendix 26)
When the avoidable area determination procedure determines that the avoidable area does not exist or the presence or absence of the avoidable area cannot be determined, the guidance information notification procedure instructs the user to search for the avoidable area. 26. The program of Appendix 25, which notifies information.
(Appendix 27)
including a direction correction information acquisition procedure,
The direction correction information acquisition step acquires direction correction information,
The direction correction information includes imaging direction information and traveling direction information,
The imaging direction information is information of an imaging direction in which the user captured an image,
The traveling direction information is information on the direction in which the user travels,
The initial guidance information notification procedure corrects the direction in which the braille block exists in the initial guidance information based on the direction correction information,
27. The program according to any one of attachments 20 to 26, wherein the guidance information notification procedure corrects at least one of a direction in which the braille blocks exist in the guidance information and a moving direction of the user based on the direction correction information.
(Appendix 28)
A computer-readable recording medium recording the program according to any one of appendices 20 to 27.

According to the present invention, the initial guidance information notification unit can guide the visually impaired person to the surrounding Braille blocks, and the guidance information notification unit enables the visually impaired person to walk without deviating from the guidance block. can guide the direction. Therefore, the present invention is useful in the field of walking assistance for people with walking disabilities such as visually impaired people.

10, 10A, 10B, 10C Walking support device for the visually impaired 11 Image acquisition unit 12 Braille block information extraction unit 13 Distance calculation unit 14 Initial guidance information notification unit 15 Virtual walking area creation unit 16 Braille block determination unit 17 Guidance information notification unit 18 Obstacle determination unit 19 Avoidance area determination unit 111 Direction correction information acquisition unit 101, 201 Central processing unit 102, 202 Memory 103, 203 Bus 104, 204 Storage device 105, 205 Program 106, 206 Input device 107, 207 Display device 108, 208 Communication device 100, 100C Walking support system device for the visually impaired 20 Walking support terminal for the visually impaired 21 Imaging unit 22 Communication unit 23 Audio output unit 24 Imaging direction measuring unit 25 Traveling direction measuring unit

Claims (10)

including an image acquisition unit, a braille block information extraction unit, a distance calculation unit, an initial guidance information notification unit, a virtual walking area creation unit, a braille block determination unit, and a guidance information notification unit;
The image acquisition unit acquires image information,
The image information is image information including the ground,
The braille block information extraction unit analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
The distance calculation unit calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image,
The initial guidance information notification unit notifies the user of initial guidance information that notifies the direction and distance of the braille block,
The virtual walking area creating unit creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination unit determines the type of the extracted braille block,
The walking support device for the visually impaired, wherein the guidance information notification unit notifies the user of guidance information based on at least one of the type of the braille block and the virtual walking area. The braille block determination unit determines whether the type of the extracted braille block corresponds to a guidance block, a warning block, or an unknown classification,
The guidance information notification unit
When the extracted braille block is a guiding block, the guiding block is included in the virtual walking area, and the movement of the user is performed so that the longitudinal direction of the linear projection of the guiding block matches the direction of movement of the user. Notify guidance information specifying the direction,
2. The walking support device for the visually impaired according to claim 1, wherein when the extracted braille block is at least one of a warning block and an unknown classification, guidance information is provided to notify the direction and distance of the presence of the braille block. 3. The walking support device for the visually impaired according to claim 1, wherein said initial guidance information notifying unit notifies the user of information instructing him/her to search for the braille block when the braille block is not extracted from the image information. including an obstacle determination unit,
The obstacle determination unit analyzes the image information to determine the presence or absence of an obstacle on the braille block,
4. The guide information notification unit according to any one of claims 1 to 3, wherein when it is determined that an obstacle exists on the braille block, the guide information notification unit notifies the user of the existence of the obstacle as the guide information. A walking support device for the visually impaired. including an obstacle determination unit and an avoidance area determination unit,
The obstacle determination unit analyzes the image information to determine the presence or absence of an obstacle on the braille block,
When it is determined that there is an obstacle on the braille block, the avoidance area determination unit analyzes the image information to determine whether or not there is an avoidable area in which the user can avoid the obstacle,
The guidance information notification unit
if it is determined that there is no avoidable area, notifying the user that an obstacle has been extracted;
If it is determined that the avoidable area exists, notifying the user of avoidance guidance information;
The avoidance guidance information includes information designating a moving direction of the user so that the user advances into the avoidable area, and information specifying the movement direction of the user when the obstacle is no longer extracted, and the guidance block is included in the virtual walking area. 5. The walking support device for the visually impaired according to any one of claims 1 to 4, further comprising information designating a direction of movement and a distance of movement of the user. When the avoidable area determination unit determines that the avoidable area does not exist or cannot determine whether or not the avoidable area exists, the guidance information notification unit instructs the user to search for the avoidable area. 6. The walking support device for the visually impaired according to claim 5, which notifies information. including a direction correction information acquisition unit,
The direction correction information acquisition unit acquires direction correction information,
The direction correction information includes imaging direction information and traveling direction information,
The imaging direction information is information of an imaging direction in which the user captured an image,
The traveling direction information is information on the direction in which the user travels,
The initial guidance information notifying unit corrects the direction in which the braille block exists in the initial guidance information based on the direction correction information,
7. The vision system according to any one of claims 1 to 6, wherein the guidance information notifying unit corrects at least one of a direction in which the braille blocks exist in the guidance information and a moving direction of the user based on the direction correction information. Disabled person walking support device. including an imaging unit, a communication unit, and an audio output unit,
The imaging unit is capable of capturing an image including the ground,
The communication unit is capable of communicating with the walking support device for the visually impaired according to any one of claims 1 to 7,
transmitting image information including the image to the walking support device for the visually impaired;
receiving at least one of the initial guidance information and guidance information;
A walking support terminal for the visually impaired, wherein the audio output unit outputs the initial guidance information or the guidance information as audio information. including an image acquisition process, a braille block information extraction process, a distance calculation process, an initial guidance information notification process, a virtual walking area creation process, a braille block determination process, and a guidance information notification process,
The image acquisition step acquires image information,
The image information is image information including the ground,
The braille block information extraction step analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
the distance calculating step calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image;
The initial guidance information notifying step notifies the user of initial guidance information that notifies the direction and distance of the braille block,
The virtual walking area creating step creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination step determines the type of the extracted braille block,
The guidance information notifying step notifies the user of the guidance information based on at least one of the type of the braille block and the virtual walking area. A program for causing a computer to execute an image acquisition procedure, a braille block information extraction procedure, a distance calculation procedure, an initial guidance information notification procedure, a virtual walking area creation procedure, a braille block determination procedure, and a guidance information notification procedure,
The image acquisition step acquires image information,
The image information is image information including the ground,
The braille block information extraction procedure analyzes the image information to extract braille block information,
The braille block information includes information on at least one of the position and size of the braille block in the image,
the distance calculation step calculates the direction and distance in which the braille block exists based on at least one of the position and size of the braille block in the image;
The initial guidance information notification procedure notifies the user of initial guidance information that notifies the user of the direction and distance of the braille block,
The virtual walking area creating procedure creates a virtual walking area in the image,
The virtual walking area is an area in the image that is delimited by two line segments extending in the depth direction from the bottom end of the image,
The braille block determination procedure determines the type of the extracted braille block,
The program, wherein the guidance information notification procedure notifies the user of the guidance information based on at least one of the type of the braille block and the virtual walking area.

Images