JP2021153814A - Control device, walking aid device, walking aid system, control method and program - Google Patents

Abstract

To provide a control device that can aid walking even on a passage in which any special mark is not installed, and to provide a walking aid device, a walking aid system, a control method and a program.SOLUTION: A control device 101 includes: analysis means 101a for analyzing an image captured by imaging means 104; acquisition means 101b for acquiring location information relating to a location of a specific object in the image; and control means 101c for controlling notification operation by notification means using an analysis result by the analysis means and the location information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a control device, a walking assist device, a walking assist system, a control method, and a program for assisting walking of a visually impaired person.

In general, a visually impaired person swings a white cane left and right in front of himself and walks on the road while detecting an obstacle. However, although it is possible to prevent hitting an obstacle by using a white cane, it is difficult to walk straight along the road.

As a technique for assisting a visually impaired person to walk along a road, Patent Document 1 provides a sensor for detecting the magnetism of a magnetic tape installed along a passage at the lower end of a white cane according to the detection result. A system for transmitting vibration and the like to a user is disclosed.

Further, in Patent Document 2, an irradiation unit that irradiates the guidance line with white light and an identification unit that identifies the light reflected from the guidance line are provided at the tip of the white cane, and vibration or sound is provided to the user according to the identified color. The communication system is disclosed.

Japanese Unexamined Patent Publication No. 2003-131556 Japanese Unexamined Patent Publication No. 2005-342218

However, since the systems of Patent Documents 1 and 2 need to detect the magnetic tape or the induction line, they can operate properly only in the passage where the magnetic tape or the induction line is installed in advance.

Further, in the systems of Patent Documents 1 and 2, since the sensor is installed at the tip of the white cane, there is a possibility that the sensor may break down due to an obstacle.

An object of the present invention is to provide a control device, a walking assist device, a walking assist system, a control method, and a program capable of assisting walking even in a passage not provided with a special mark.

The control device as one aspect of the present invention includes an analysis means for analyzing an image captured by the imaging means, an acquisition means for acquiring position information regarding the position of a specific object in the image, an analysis result by the analysis means, and a position. It is characterized by having a control means for controlling a notification operation by the notification means using information.

According to the present invention, it is possible to provide a control device, a walking assist device, a walking assist system, a control method, and a program capable of assisting walking even in a passage not provided with a special mark.

It is a block diagram which shows the outline of the structure of the walking assist device which concerns on embodiment of this invention. It is a flowchart which shows the notification processing of the walking assist device of Example 1. It is a schematic diagram which shows the example of the result of image analysis. It is a schematic diagram which shows the example of the walking guide line. It is a schematic diagram showing a white cane. It is a schematic diagram which shows the history of the position of a white cane. It is a flowchart which shows the notification processing of the walking assist device of Example 2. It is a block diagram which shows the outline of the structure of the walking assist system of Example 3. It is a flowchart which shows the processing of an image pickup device. It is a flowchart which shows the processing of a vibrating device.

Hereinafter, examples of the present invention will be described in detail with reference to the drawings. In each figure, the same member is given the same reference number, and duplicate description is omitted.

FIG. 1A is a block diagram showing an outline of the configuration of the walking assist device 100 according to the embodiment of the present invention. The walking assist device 100 includes a control unit 101, a memory 102, a non-volatile memory 103, an imaging unit (imaging means) 104, an image processing unit 105, and a vibration unit (notifying means) 106. Each unit is connected to the internal bus 110 and is configured so that data can be exchanged with each other via the internal bus 110.

The memory 102 includes, for example, a RAM (Random Access Memory).

The non-volatile memory 103 is composed of, for example, a hard disk (HD), a ROM (Read Only Memory), or the like, and stores image data, audio data, other data, various programs for operating the control unit 101, and the like.

The control unit 101 controls each part of the walking assist device 100 by using the memory 102 as a work memory according to a program stored in the non-volatile memory 103, for example. As shown in FIG. 1B, the control unit 101 includes an analysis means 101a, an acquisition means 101b, and a control means 101c. In the present embodiment, the control unit 101 is mounted inside the walking assist device 100, but may be configured as a control device different from the walking assist device 100.

The image pickup unit 104 includes a lens and an optical sensor that converts light from the lens into an electric signal, captures an image under the control of the control unit 101, and stores the image in the memory 102. As the optical sensor, a CMOS (Complementary Metall-Oxide-Semiconductor) image sensor, a CCD (Charge-Coupled Device) image sensor, or the like can be used.

Based on the control of the control unit 101, the image processing unit 105 performs various image processing such as resizing processing, trimming processing, edge detection processing, movement vector calculation processing, and matrix calculation processing for the image data stored in the memory 102. To give. The image processing unit 105 may be composed of a dedicated circuit block for performing specific image processing. Further, depending on the type of image processing, the control unit 101 may perform image processing according to a program.

The vibrating unit 106 is a vibrating device such as a haptic device, and vibrates based on the control from the control unit 101. The user can know that the vehicle is moving in the correct direction by using the vibration transmitted from the vibration unit 106. In the present embodiment, the vibration of the vibrating unit 106 is used to execute the notification operation to the user, but the present invention is not limited to this as long as the device can execute the notification operation to the user. Further, in the present embodiment, the vibrating unit 106 is provided in the walking assist device 100, but may be provided in another device. In any case, it is sufficient that the control unit 101 and the vibration unit 106 are connected by wire or wirelessly, and the control unit 101 can transmit a signal instructing the vibration unit 106 to execute the notification operation. ..

Hereinafter, the operation of each embodiment of the present invention will be described.

In this embodiment, the white cane, which is a walking assisting cane, is a virtual line (walking guide line) that serves as a walking guide set by using information about a specific object that serves as a walking mark acquired from an image. ), It is possible to guide walking by notifying the user.

FIG. 2 is a flowchart showing a notification process of the walking assist device 100 of the first embodiment. Each process in the flowchart of FIG. 2 is realized by the control unit 101 expanding the program stored in the non-volatile memory 103 into the memory 102 and executing the program.

In step S201, the control unit 101 acquires the image captured by the image capturing unit 104.

In step S202, the control unit 101 (analysis means 101a) acquires information on a specific object that serves as a walking mark by analyzing the image captured in step S201. As a method of analyzing an image and acquiring information about an object, there are a method of using the result of edge detection and the like, and a method of using machine learning such as Deep Learning. Further, the image may be analyzed by using the image processing unit 105. In this embodiment, information about a specific object that serves as a walking mark is acquired, but as described later, if it is information about a specific object that can set a predetermined area for assisting walking, it is a walking mark. It is not limited to information about a specific object. Further, although the information about the specific object is acquired by the control unit 101 (analysis means 101a) analyzing the image in this embodiment, it may be acquired by using the position information based on the signal from the artificial satellite. ..

FIG. 3 is a schematic diagram showing an example of the analysis result of the image in step S202. In FIG. 3, the road 301 and the white line 302 on the road are acquired as the analysis result of the image.

In step S203, the control unit 101 (analysis means 101a) sets a predetermined area on the image to assist walking based on the analysis result of the image. In this embodiment, the control unit 101 (analysis means 101a) sets a virtual line (walking guide line) indicating a walkable area using information about a specific object as an example of a predetermined area. The predetermined area also includes, for example, a space in which walking (passage) should not be permitted. Further, the control unit 101 (analysis means 101a) may set a predetermined region by using the position information based on the signal from the artificial satellite.

FIG. 4 is a schematic diagram showing an example of a walking guide line. In FIG. 4, a walking guide line 401 is drawn inside the white line 302 along the white line.

In step S204, the control unit 101 (acquisition means 101b) acquires position information regarding the position of the white cane (specific object) in the image obtained by analyzing the image acquired in step S201.

FIG. 5 is a schematic view showing a white cane. 501, 502, and 503 represent the grip, handle, and tip of the white cane, respectively. 504 represents a painted portion painted in red of about 20 cm near the tip of the white cane.

In step S204, the control unit 101 acquires a white position by detecting the painted unit 504. As a method of detecting a portion having a specific color from an image, there are a method of using a color histogram for each divided block, a method of using machine learning such as Deep Learning, and the like as in step S202.

In the present embodiment, the position of the white cane is acquired by analyzing the image, but a transmitter for specifying the position is installed in the white cane, and the position of the white cane is acquired by the transmitted signal. May be good.

In step S205, the control unit 101 determines whether or not the position information regarding the position of the specific object is located within the predetermined area set in step S203. In this embodiment, it is determined whether or not the white cane intersects the walking guide line (crosses the walking guide line). If it is determined that the white cane intersects the walking guide line, the process proceeds to step S206, and if not, this flow ends.

FIG. 6 is a schematic diagram showing the history of the position of the white cane. In FIG. 6, the white circle represents the position of the white cane, and the position of the white cane changes in the order of (a) to (d). In FIGS. 6A to 6C, the white cane is located on the left side of the walking guide line, and the control unit 101 determines that the white cane does not intersect the walking guide line. The white cane located on the left side of the walking guide line in FIG. 6 (c) has moved to the right side of the walking guide line in FIG. 6 (d). The control unit 101 determines that the white cane intersects the walking guide line at the time of FIG. 6D.

In step S206, the control unit 101 (control means 101c) notifies the user that the white cane has crossed the walking guide line by vibrating the vibrating unit 106.

As described above, according to the configuration of the present embodiment, it is possible to notify the user that the white cane has crossed the walking guide line by using vibration. As a result, walking can be assisted even in a passage where no special mark is installed.

In the configuration of the first embodiment, since the user is notified each time the white cane crosses the walking guide line, the number of notifications tends to increase, which may be annoying to the user. In this embodiment, the user is notified only when the white cane moves in a predetermined direction.

FIG. 7 is a flowchart showing the notification process of the walking assist device 100 of this embodiment. Each process in the flowchart of FIG. 7 is realized by the control unit 101 expanding the program stored in the non-volatile memory 103 into the memory 102 and executing the program.

Since the processes of steps S701 to S703, steps S705 to S706, and step S708 of FIG. 7 are the same as the processes of steps S201 to S206 of FIG. 2, the description thereof will be omitted.

In step S704, the control unit 101 determines the notification direction for notifying the user when the white cane is moving in any direction from right to left or left to right. For example, in FIG. 4, since the walking guide line 401 is drawn inside the white line 302, the user can easily walk inside the white line 302 by notifying the user when the white cane moves from left to right. .. Therefore, in the case of FIG. 4, the control unit 101 determines the direction in which the white cane notifies the user from left to right as the notification direction (predetermined direction).

In step S707, the control unit 101 determines whether or not the moving direction of the white cane matches the notification direction determined in step 704. If the moving direction of the white cane matches the direction determined in step 701, the process proceeds to step S708, otherwise the flow ends.

As described above, according to the configuration of the present embodiment, in addition to the effect of the first embodiment, the user is notified only when the white cane moves in a predetermined direction, so that the convenience can be improved.

In this embodiment, a case where the imaging device and the vibration device are configured as different devices will be described.

FIG. 8 is a block diagram showing an outline of the configuration of a walking assist system including an imaging device (walking assist device) and a vibration device (notifying device). The gait assist system includes an imaging device 800 and a vibration device 820. Since 102 to 106 in FIG. 8 are the same as those in 102 to 106 in FIG. 1, the description thereof will be omitted.

The image pickup device 800 includes a control unit 801 and a memory 102, a non-volatile memory 103, an image pickup unit 104, an image processing unit 105, and a communication I / F (transmission means) 802. Each unit is connected to the internal bus 810 and is configured so that data can be exchanged with each other via the internal bus 810. The control unit 801 controls each unit of the image pickup device 800 by using the memory 102 as a work memory according to a program stored in the non-volatile memory 103, for example. The communication I / F 802 is an interface for communicating with an external device, a server on the Internet, or the like to send and receive various data such as files and commands.

The vibration device 820 includes a control unit 821, a memory 102, a non-volatile memory 103, a communication I / F (reception means) 822, and a vibration unit 106. Each unit is connected to the internal bus 830 and is configured so that data can be exchanged with each other via the internal bus 830. The control unit 821 controls each unit of the vibration device 820 by using the memory 102 as a work memory according to a program stored in the non-volatile memory 103, for example.

FIG. 9 is a flowchart showing the processing of the imaging device 800. Each process in the flowchart of FIG. 9 is realized by the control unit 801 expanding the program stored in the non-volatile memory 103 into the memory 102 and executing the program.

Since the processes of steps S901 to S905 of FIG. 9 are the same as the processes of steps S201 to S205 of FIG. 2, the description thereof will be omitted.

In step S906, the control unit 801 instructs the vibration device 820 to vibrate via the communication I / F 802.

FIG. 10 is a flowchart showing the processing of the vibration device 820. Each process in the flowchart of FIG. 10 is realized by the control unit 821 expanding the program stored in the non-volatile memory 103 into the memory 102 and executing the program.

In step S1001, the control unit 821 determines whether or not a vibration instruction has been received from the image pickup device 800 via the communication I / F 822. If the vibration instruction is received, the process proceeds to step S1002, and if not, the process of this step is repeated.

In step S1002, the control unit 821 notifies the user that the white cane has crossed the walking guide line by vibrating the vibrating unit 106.

As described above, according to the configuration of the present embodiment, by mounting the vibration device 820 near the grip of the white cane, the user can receive the walking guide via the white cane.
[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof.

101 Control unit (control device)
101a Analysis means 101b Acquisition means 101c Control means 104 Imaging unit (imaging means)
106 Vibrating part (notifying means)

Claims (17)

An analysis means for analyzing an image captured by the imaging means,
An acquisition means for acquiring position information regarding the position of a specific object in the image, and
A control device including an analysis result by the analysis means and a control means for controlling a notification operation by the notification means using the position information. The control device according to claim 1, wherein the acquisition means acquires the position information obtained by analyzing the image. The analysis means sets a predetermined region on the image based on the analysis result of the image.
The control device according to claim 1 or 2, wherein the notification means performs a notification operation when the position information is located in the predetermined area. The control device according to claim 3, wherein the analysis means acquires information about a specific object in the image and sets the predetermined region using the information about the specific object. The control device according to claim 4, wherein the analysis means acquires information about the specific object by using position information based on a signal from an artificial satellite. The control device according to claim 3, wherein the analysis means sets the predetermined region using position information based on a signal from an artificial satellite. The control device according to any one of claims 3 to 6, wherein the predetermined area is a walking guide line indicating a walkable area. The control means is characterized in that the notification means causes the notification means to execute the notification operation when the specific object intersects the walking guide line and the movement direction of the specific object is a predetermined direction. 7. The control device according to 7. The control device according to any one of claims 1 to 8, wherein the control means is connected to the notification means by wire or wirelessly. The control device according to any one of claims 1 to 9, wherein the notification means is a vibration means. The control device according to any one of claims 1 to 10, wherein the specific object is a wand for walking assistance. An imaging means for capturing an image and
A walking assist device comprising the control device according to any one of claims 1 to 11. The walking assist device according to claim 12, further comprising a transmission means for transmitting a signal instructing the execution of the notification operation to the notification means for executing the notification operation. The walking assist device according to claim 12, further comprising a notification means for executing the notification operation. The walking assist device according to claim 12 or 13,
A walking assist system comprising a receiving means for receiving a signal instructing execution of a notification operation from the walking assist device, and a notification device including a notification means for executing the notification operation. Steps to analyze the image captured by the imaging means,
A step of acquiring position information regarding the position of a specific object in the image, and
A control method comprising: a step of controlling a notification operation by a notification means using the analysis result of the image and the position information. A program that causes a computer to execute the control method according to claim 16.

Images