JP6393199B2 - Walking obstacle location determination system and its self-propelled traveling device - Google Patents

Description

  The present invention relates to a walking obstacle location determination system and a self-propelled traveling device thereof.

  For humans, housing is the foundation of their lives, but it is known that the average purchase age of housing is in their 30s. However, homes purchased at an early age may become increasingly difficult to live as they grow older or as their family structure changes. For example, entrances that have no problem at the time of purchase, threshold steps, slopes of stairs, etc., increase the risk of falling as the residents become older.

  Currently, it is said that there are three types of senior citizens (seniors). They are healthy and active as well as the active generation (called active seniors), elderly people in need of care, and healthy. It is divided into elderly people (including gap seniors, including elderly people in “need to be in need of support”) who may be in a state requiring care. Of these, gap seniors are expected to account for more than 40% of the total elderly, an estimated 10 million. In particular, the cause of the deterioration of the health condition of gap seniors is most often due to falls. For example, over 500,000 elderly people who were urgently transported to hospitals for five years from 2006 to 2011. Accounting for 79% of the total.

  Therefore, for residents, especially gap seniors, where there is a risk of falling or being “smiling” in the house due to crawling, slipping, or colliding with the body, or currently there are so many problems Even if it is not necessary, it is necessary to keep track of the places where the danger may occur in the future (hereinafter referred to as “care points” or “gait obstacles”), and measures are taken in advance for those places. It is desirable to apply. Ideally, it would be possible to renovate the current residence in a barrier-free manner, but full renovation is costly.

  On the other hand, in public institutions that support the lives of the elderly, or stationary companies that propose renovation to customers, there is a possibility that the elderly, etc. of the residence where the resident currently lives, may become an obstacle during walking. It is necessary to investigate a certain place (walking obstacle part) and convey the degree of risk to residents and family watchers (hereinafter referred to as residents). However, it takes a lot of time and cost to investigate one resident residence.

  By the way, nowadays, an automatic traveling robot and an automatic cleaning robot that can travel in a room are known. For example, in Patent Document 1, a distance sensor that measures the distance between a floor surface in front of the moving body and the moving body is provided on the moving body that includes traveling wheels, and the front floor surface and the moving body that are measured by the same distance sensor are provided. The technology of an autonomous mobile robot provided with a detecting means for detecting a step on the front floor surface by comparing the distance between the predetermined distance and a predetermined distance between the floor surface and the moving main body is disclosed. This autonomous mobile robot is provided with a movement amount measuring means in the moving body, and when the deviation between the measurement distance from the front floor surface and the set distance continuously exceeds the reference amount, the movement amount measuring means performs the same reference. The movement distance after exceeding the amount is measured, and when the movement distance exceeds the reference movement amount, it is determined that traveling is impossible and the moving main body is stopped.

  If the technology disclosed in Patent Document 1 is applied, the automatic traveling robot, when detecting a step or a slope in front of the travel route, recognizes the height, width, or slope of the step, thereby reducing the level difference. Or, if the groove has a narrow width, it is possible to travel.

JP 2006-146376 A

  Using the technology of automated robots as described above, it is possible to automatically detect walking obstacles such as steps, notify residents, etc., and grasp the walking environment in the residence for the future. And can also significantly reduce the time and cost problems of the surveys described above.

  Therefore, in the present invention, in view of the above-mentioned problems, as part of services that support the life of gap seniors, etc., it automatically detects walking obstacles such as steps and slopes in the residence and notifies the resident etc. An object of the present invention is to provide a walking obstacle location determination system and a self-propelled traveling device capable of grasping the walking environment.

  In order to solve the above problems, the walking obstacle location determination system and the self-propelled traveling device of the present invention provide the following solution.

  A self-propelled traveling device that automatically travels in a room, which is a means for acquiring or generating floor plan data in the room, a position detecting means for detecting its own position in the room, and a walking obstacle in the room Obstacle detection means for detecting a possible obstacle, imaging means for picking up a surrounding image during automatic traveling, the captured image taken by the imaging means, and the obstacle detected by the obstacle detection means And obstacle information storage means for recording the object information together with the date and time information.

  In addition, the self-propelled traveling device further includes image generation means for generating an image in which the position of the obstacle is superimposed on the indoor floor plan.

  The obstacle detection means includes a friction degree sensor that measures the friction degree with the floor surface, and detects a rotational obstacle during automatic running and measures the friction degree to identify a slippery walking obstacle portion. It is characterized by that.

  The obstacle detection means includes an aerial obstacle determination means, and the aerial obstacle determination means determines an object located within a predetermined height range from an image of the object acquired during traveling as an aerial obstacle. It is characterized by doing.

  Further, the obstacle detection means includes a temperature sensor, and identifies a walking obstacle portion having a low temperature by measuring an outside air temperature during traveling. The obstacle detection means includes a contact detection sensor, and detects the air obstacle by the contact detection sensor during traveling.

  Further, the obstacle detection means searches for the same place in the room by different movements a plurality of times.

  In addition, the self-propelled traveling device includes voice dialogue means, and when the obstacle detection means finds a new obstacle, it notifies the resident or the watcher by voice.

  Further, the obstacle detection means includes obstacle determination means, the obstacle determination means acquires the resident characteristic information from the voice dialogue means, and based on the characteristic information, the obstacle determination means This is characterized in that a determination criterion is set.

  In addition, the image processing apparatus may further include an image projecting unit configured to project an image in which the position of the obstacle is superimposed on the indoor floor plan generated by the image generating unit on the indoor wall surface.

  Further, the self-propelled traveling device has obstacle confirmation means for allowing a resident or a watcher to confirm the obstacle information stored in the obstacle information storage means, and as a result of the confirmation, there is no problem. The obstacle information is deleted from the obstacle information storage means.

  In another aspect of the present invention, a walking obstacle location comprising: a self-propelled traveling device that automatically travels indoors; and a management device that communicates wirelessly with the self-propelled traveling device and notifies a walking obstacle location in the room. In the determination system, the self-propelled traveling device includes position detection means for detecting its own position in the room, obstacle detection means for detecting an obstacle that may become a walking obstacle in the room, An image pickup means for picking up an image of surroundings during automatic traveling; and an obstacle information collection means for collecting information of the obstacle detected by the obstacle detection means and a picked-up image picked up by the image pickup means. The management device generates an indoor image including a walking obstacle portion obtained by performing image processing on the captured image collected by the obstacle information collecting unit and information on the obstacle. Characterized in that it comprises a and.

  According to the present invention, a walking obstacle location determination system and a self-propelled traveling device capable of automatically detecting a walking obstacle location such as a step or an inclination and notifying a resident etc. and grasping the walking environment in the residence are provided. can do.

It is an image figure of the walking obstacle location determination system which concerns on embodiment of this invention. It is a figure which shows an example of the external appearance of a self-propelled traveling apparatus. It is a figure which shows another example of the external appearance of a self-propelled traveling apparatus. It is a functional block diagram which shows the structure of the walking disorder location determination system which concerns on embodiment of this invention. It is a figure which shows an example of the data memorize | stored in walking disorder location information DB. It is a flowchart which shows the processing operation of a self-propelled traveling device. It is a flowchart which shows the processing operation of a management apparatus. It is a figure which shows an example of the screen which displays a floor plan. It is a figure which shows the functional block of another structure of a self-propelled traveling apparatus.

  DESCRIPTION OF EMBODIMENTS Hereinafter, a mode for carrying out the present invention (hereinafter referred to as this embodiment) will be described in detail with reference to the accompanying drawings. Note that the same numbers or symbols are assigned to the same elements throughout the description of the embodiment.

  FIG. 1 is an image diagram of a walking obstacle location determination system according to the present embodiment. Hereinafter, the above system will be referred to as “the present system”. Here, the “walking obstacle part” means a part (for example, a step, an inclination, a slippery floor or staircase, a protruding furniture, etc.) that may hinder indoor walking.

  This system includes a self-propelled traveling device that automatically travels indoors, and a management device that communicates wirelessly with the self-propelled traveling device and notifies a walking obstacle location in the room. The management device (management server) communicates wirelessly with the self-propelled traveling device, receives various data detected by the self-propelled traveling device, processes and stores the data.

  FIG. 1 shows an image diagram of a state in which a self-propelled traveling device brought into a house moves around the first floor and the second floor and detects various “care points” existing in the house. “Hidden point” refers to steps and slopes on the floor, steep stairs and slippery floors, darkness, places with extremely high or very low temperature differences, and other objects and environments that may cause walking problems. Means. For example, this figure shows how a self-propelled traveling device that travels on the first floor measures the width of the passage and detects a narrow space next to the table as a “smiling point” that humans can easily hit when walking. ing. In addition, a self-propelled traveling device that travels on the second floor measures the friction level of the floor surface, and shows that the exit of the staircase is detected as a “smiling point” that is easy to slip. Data including the position information of the detected location is transmitted to the management device, processed, accumulated, and can be displayed together with the floor plan of the residence at any time. At this time, the position of the obstacle is superimposed on the floor plan, and it is desirable to further include an image such as a photograph as the detailed information of the obstacle. Alternatively, the floor plan itself may be a three-dimensional plan view generated from a 360-degree panoramic photograph including the floor surface. Even in the case of a floor plan, in particular, it is desirable to include a three-dimensional image obtained by capturing photographs of the obstacle from a plurality of locations in order to display details of the air obstacle described later. The management device may be brought into the residence together with the self-propelled traveling device, but may be provided as a management server in a remote place.

  FIG. 2 is a diagram illustrating an example of the appearance of the self-propelled traveling device. As shown in the figure, the self-propelled traveling apparatus 1a is provided with a caterpillar as a driving means 50 on the lower side of a quadrangular body, and includes a camera as an imaging means 15 in the front-rear and left-right directions of the body. Similarly, an imaging camera 15 is provided with a telescopic camera for imaging an airborne obstacle. An airborne obstacle is an object above the self-propelled traveling device, and the self-propelled traveling device can travel under it without any problem, but an object that is likely to become an obstacle when a human walks. Say. For example, a table top, a half-open drawer of a desk or a chest, an article protruding from a shelf, and the like are typical examples. On the other hand, an object that obstructs the traveling of the self-propelled traveling device itself is referred to as a floor surface obstacle. For example, steps, steps, furniture, etc. are typical examples.

  In addition, it is desirable to provide several auxiliary wheels 51 having a spherical shape inside the caterpillar on the lower side of the body in order to assist the running of the caterpillar (for example, to enable lateral movement). Further, an antenna 11a for communicating with other self-propelled traveling devices, management devices, and the like is provided on the front side of the body, and various sensors (not shown) are provided on each surface of the body. Since the self-propelled traveling devices can communicate with each other, when a plurality of self-propelled traveling devices are arranged in the residence, it is possible to control so that the search areas do not overlap each other. The travel mechanism is not limited to the caterpillar, and may be wheel drive. Moreover, the mechanism which can go up and down stairs may be sufficient.

  FIG. 3 is a diagram illustrating another example of the appearance of the self-propelled traveling device. FIG. 3A shows a shape suitable for measuring an airborne obstacle of the self-propelled traveling device. As shown in this figure, the self-propelled traveling device 1b is provided with several wheels on the lower side of a cylindrical body, and can easily rotate 360 °. In addition, a head is provided on the upper part of the body, and a camera that is the imaging means 15 is provided on the head. Since the head can be expanded and contracted up and down and can be rotated 360 °, the head and neck have a built-in contact detection sensor that detects contact with an obstacle. It has a structure suitable for detecting airborne obstacles above the self-propelled traveling device.

  Moreover, as another embodiment, the self-propelled traveling device 1c can be formed into a pet shape (for example, a cat-shaped robot). For example, as shown in FIG. 3C, the imaging means 15 is formed in the eyes of a cat, and the tail is an antenna 11a. In addition, by providing the driving means 50 capable of walking on four legs instead of the caterpillar or the wheel, it is also possible to rent or sell to the resident as a resident's pet. Pet-type robots are programmed to circulate indoors when they are not used as pets, even if they are installed in a resident's home for a long period of time. It can be carried out. Therefore, it is possible to detect and notify a walking obstacle that appears depending on the season, such as a stove, a kotatsu, and a fan.

  A self-propelled traveling device travels in a room, detects the shape (contour) of the room, and determines a reference point as a reference for the position in the room (hereinafter, room information including the indoor shape and the reference point). Called). For example, the start position when the power is turned on may be determined as the reference point, or the position where the battery of the self-propelled traveling device is charged may be used as the reference point. Then, the user travels around the room, acquires various types of data and images using sensors and cameras built in the self-propelled traveling device, and determines whether or not these data and images cause a walking obstacle. Even if it is an obstacle, an object in a size and position that is not problematic for human walking is not determined as a walking obstacle. However, since the size and position of an object that becomes a walking obstacle varies depending on the resident, the setting can be changed as a parameter. In addition, for determining a walking obstacle location, for example, indoor illuminance, temperature, level difference, slipperiness (friction level) of the floor, floor surface obstacle, presence of an airborne obstacle, and the like are determined. Then, when the self-propelled traveling device determines that the detected object is likely to have a walking disorder, the walking obstacle location information determined together with the room information is transmitted to the management device using the wireless communication means.

  When the management device receives the room information and the walking obstacle location information, the management device generates an image in which the walking obstacle location is superimposed on the floor plan of the house based on the information. In addition, when the floor plan of a house already exists, based on the information of the floor plan and the walking obstacle part information, an image in which a walking obstacle part is superimposed on the floor plan of the house is generated. The generated image is transmitted to a dedicated terminal of a public institution or real estate company and displayed on the screen. It can also be displayed on the terminal of a resident or the like (a resident or a watcher such as a family).

  In this way, it is possible to identify indoor “careful points” and automatically notify public institutions and real estate companies, etc., without having humans go to investigate, It can be used to propose reforms. In addition, by using this system by a resident or the like, “care points” can be grasped in advance, and the safety of the room can be improved before an accident occurs. “Hidden points” change continuously depending on the resident's condition, time (seasonal, secular change), weather, how to place indoor furniture and objects, etc. It is desirable to measure. In addition, in FIG. 1, although demonstrated taking as an example installing a self-propelled traveling apparatus for every floor of a residence, it is equipped with the mechanism which goes up and down stairs to the caterpillar which is the drive means 50, or a wheel, for example. Thus, it may be possible to automatically detect the walking obstacle locations on the first floor and the second floor.

(Configuration of walking obstacle location determination system)
FIG. 4 is a functional block diagram showing the configuration of the walking obstacle location determination system according to the present embodiment. As shown in FIG. 4, this system communicates wirelessly with the self-propelled traveling device 10 that automatically travels indoors and the self-propelled traveling device 10 as described above, and notifies walking obstacle location information in the room. The management device 20 is provided.

(Configuration of self-propelled traveling device)
As a configuration example, the self-propelled traveling device 10 includes a communication unit (transmission unit) 11, a position detection unit 12, an obstacle determination unit 13, an automatic traveling unit 14, an imaging unit 15, and a sensor group 16. The obstacle information collecting means 17 and the rechargeable battery 18 are included. The obstacle determination means 13 and the sensor group 16 are collectively referred to as an obstacle detection means 19.

  The communication unit 11 serves as a communication interface unit when communicating with the management device 20. Further, the communication unit 11 wirelessly transmits the captured image and the running state information collected by the obstacle information collecting unit 17 to the management device 20. In the following description, information detected by the sensor group 16 is collectively referred to as travel state information.

  The position detection means 12 detects position information in its own room. The position information detection means uses a known indoor positioning technique such as indoor GPS, and the description thereof is omitted here. The self-propelled traveling device 10 travels in the room, detects the shape of the room, and determines a reference point for position information in the dwelling of the self-propelled traveling device 10.

  The obstacle detection means 19 includes a sensor group 16 that detects surrounding information during automatic traveling, and an obstacle determination means that determines whether the obstacle is based on information obtained from the sensor group 16 and a captured image during traveling. 13. Moreover, the obstacle determination means 13 determines an obstacle that may become a walking obstacle in the room. Therefore, the obstacle determination means 13 includes a floor surface obstacle determination means 13a and an airborne obstacle determination means 13b.

  For example, the floor obstacle determination unit 13a irradiates laser light or ultrasonic waves in front, back, left, and right, and measures the distance and size from the reflected light or ultrasonic waves to the object. Determine if. The floor surface obstacle is an object or a shape of an object having a predetermined size or larger that inhibits the progress of the self-propelled traveling device 10 when moving on the floor surface. For example, walls, steps, sills, stairs, furniture, electrical appliances and the like. Actually, the self-propelled traveling device may be brought into contact with the object to be determined as a floor surface obstacle. Further, the floor surface in the traveling direction is searched, an image of an indoor object is captured, and an object at a position exceeding a predetermined height (for example, 30 cm) is determined as an air obstacle. However, even if an object is located at a position exceeding a predetermined height, an object located at a height exceeding the normal height of a person is not determined as an air obstacle. That is, an object located in a predetermined height range (for example, 30 cm to 180 cm) is determined as an air obstacle, and an object less than the predetermined height is determined as a floor obstacle. Further, the degree of slipperiness of the floor surface is detected by measuring the degree of friction (friction coefficient) of the floor surface, and a location exceeding a predetermined value is determined as a floor surface obstacle. Of course, when a contact is made with an object in the traveling direction, the contact detection sensor may be activated to determine that the object is a floor surface obstacle. The techniques for detecting and determining these obstacles on the floor are assumed to make the most of known techniques, and details thereof are omitted here.

  As described above, the air obstacle determination means 13b can be an obstacle located at a predetermined height within the vicinity of the upper part of the travel route as a walking obstacle point based on information obtained during travel. Judged as an aerial obstacle. Specifically, on the basis of information such as the distance, height, size, etc. from the indoor image (captured image) captured by the imaging unit 15 and / or the object acquired from the sensor group 16 at the captured location. It is determined whether or not the object becomes a walking obstacle. That is, the captured image is subjected to image analysis, an object in the image is extracted, and it is determined whether the object itself or a projection of the object is not at a height at which a human walks. For example, if a furniture half-open drawer is detected above the route along which the self-propelled traveling device 10 travels, it is not determined as an obstacle on the floor, but is determined as an aerial obstacle. Further, when the vehicle can travel under the table, the table top is not a floor obstruction, but may be determined as an airborne obstruction depending on how the table top protrudes. As a technique for extracting and analyzing an object in an image for this purpose, a known technique is used. In addition, the distinction between a floor surface obstacle and an airborne obstacle is a height from the floor surface. For example, the obstacle on the floor surface is set to be 30 cm or less from the floor surface, and an object located beyond the floor obstacle is an air obstacle, and can be changed depending on the setting.

  The automatic traveling means 14 includes a known configuration (inverter, motor, brake, etc.) necessary for driving electrically as the driving means 50 of the self-propelled traveling device 10 and performs automatic traveling control. Further, the vehicle detours or gets over the obstacle detected by the obstacle detection means 19 and travels. Specifically, a step is detected by the obstacle detection means 19, and the vehicle travels around or gets over depending on the height of the step.

  The imaging means 15 is a camera that captures surrounding images during automatic traveling. The imaging means 15 (camera) amplifies the optical system including a lens, an imaging device such as a CMOS sensor, a signal photoelectrically converted by the imaging device through the lens, and performs imaging after color adjustment such as luminance adjustment, white balance, and γ correction. A control system for generating an image and outputting it to the obstacle information collecting means 17 is included. Also included is a mechanical component that can move the head portion including the optical system in the vertical direction in order to photograph an airborne obstacle.

  The sensor group 16 detects various information in the traveling room and sends it to the obstacle determination means 13. Specifically, at least a step sensor 31 that detects a step, an illuminance sensor 32 that detects the intensity (illuminance) of room light, and a friction sensor that measures the friction between a running surface (floor surface) and a caterpillar or a wheel. 33, a contact detection sensor 34 for detecting contact with an obstacle, a temperature sensor 35 for detecting environmental temperature (outside air temperature) during automatic traveling, a biaxial or triaxial acceleration sensor 36 for detecting acceleration, A battery sensor 37 that measures the remaining battery level and a gyro sensor 38 that detects the rotation angle of the apparatus are included.

  The step sensor 31 measures the step (width, height, etc.) for the obstacle determination means 13 to determine the step. For example, the step sensor 31 performs an angle scan while projecting a laser beam on the detected step. And a control system that measures the distance to the object by detecting the reflected light.

  The friction degree sensor 33 detects the rotational resistance of the caterpillar during traveling by the automatic traveling means 14 and measures the friction degree (friction coefficient).

  The obstacle information collecting unit 17 collects the captured image captured by the imaging unit 15 and the running state information detected by the sensor group 16. The collection timing may be performed every time the self-propelled traveling device 10 determines an obstacle, but may be performed every certain traveling distance or traveling time.

(Management device configuration)
The management apparatus 20 typically includes a communication unit 21, an image processing unit 22, a display unit 23, an image generation unit 24, a control unit 25, a walking obstacle location information DB 100, a floor plan DB 200, Consists of

  The communication unit 21 serves as a communication interface unit that performs wireless communication with the self-propelled traveling device 10 and the management device 20. The communication between the self-propelled traveling devices 10 may be performed using normal Wi-Fi (registered trademark), but when the management device 20 is in a remote place as a management server, communication is performed via the Internet.

  The image processing unit 22 performs image processing for determining a floor obstacle or an aerial obstacle performed by the obstacle determination unit 13 of the self-propelled traveling device 10. In other words, the indoor image captured by the self-propelled traveling device 10 is acquired, and image analysis is performed using a known image analysis technique to extract an object in the image. The extracted object information is sent to the obstacle determination means 13 to determine whether the object is a walking obstacle. The image processing means 22 may be provided inside the obstacle determination means 13 of the self-propelled traveling device 10.

  The display unit 23 is configured by, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electro-Luminescence), and the like, and is a three-dimensional plan view generated from a three-dimensional plan view generated by the image generation unit 24 or a 360-degree panoramic photograph. Further, information generated by the control means 25 is displayed and output.

  The image generation unit 24 generates a room image in which floor obstacles and air obstacles are shown on the floor plan based on the walking obstacle location obtained by image processing and the corresponding room information. In addition, when the floor plan of the corresponding room is registered in the floor plan DB 200, the room image is generated in combination with the position of the obstacle using the information of the floor plan. If there is no floor plan, the travel route information obtained by the self-propelled traveling device 10 traveling in the room and the information on the places where travel is impossible are collected, and the outline of the entire region traveled is simplified as a floor plan. You may make it produce | generate. At this time, the generated simple floor plan may be corrected by a human by referring to the captured image in the room.

  The control means 25 incorporates a CPU (microprocessor) and a memory, and plays a role of controlling each functional block described above by the CPU sequentially reading and executing a program recorded in the memory.

  An example of the data structure of the walking obstacle location information DB 100 is shown in FIG. The walking obstacle location information DB 100 is a database in which the walking obstacle location information generated by the control means 25 is stored. The control means 25 is collected by the obstacle information collecting means 17 of the self-propelled traveling device 10 and communicated. 11. Based on the running state information transmitted / received by the communication means 21, the walking obstacle location information is generated.

  As shown in FIG. 5, an example of the data structure includes at least data items of a house “No”, “house name”, and “walking obstacle location information” numbered for each house. In the data item “walking obstacle location information”, the “coordinate position from the reference point of the walking obstacle location determined by each sensor”, “failure content” (reason determined to be a walking obstacle location), “ “Measured value” is registered.

  According to FIG. 5, in “Sokentarou House” numbered No. “00100”, there are six places determined as walking obstacle places. Among them, the plane coordinate positions x (120) and y (180) are relatively dark places where the illuminance is less than 10 lux, and the plane coordinate positions x (130) and y (50) have a temperature of 8 ° C. Indicates a relatively cold place. Further, the plane coordinate positions x (140) and y (170) have a step of 5 cm and there is a risk of falling, and the plane coordinate positions x (150) and y (82) have a degree of friction. 3 indicates that the place is relatively slippery and requires attention. Further, the plane coordinate position x (160), y (145) is a place where there is an obstacle with a height of 25 cm and there is a possibility of collision or falling, and the plane coordinate position x (170), y (103) indicates that there is an aerial obstacle at a height of 120 cm and there is a risk that a human may collide when walking. In the coordinate position z, the first floor is set to zero (reference), and the height value of the detection position is entered in the middle of the second floor or stairs.

  The floor plan DB 200 stores floor plans prepared in advance or information of simplified floor plans generated by the management device 20. Usually, since there is a floor plan for an apartment such as a real estate rental, a condominium, etc., the information is stored in the floor plan DB 200. In the case where there is no floor plan, for example, in an old apartment or a detached house, the self-propelled traveling device 10 detects the outline of the room and saves it in the floor plan DB 200 as described above. Is done.

  The functions and configuration of the system described above are merely examples, and one functional block (database and function processing unit) may be divided or a plurality of functional blocks may be configured as one functional block. Good. Each function processing unit reads a computer program stored in a storage device such as a ROM (Read Only Memory), flash memory, SSD (Solid State Drive), or hard disk by a CPU (Central Processing Unit) built in the device This is realized by a computer program executed by the CPU. That is, in each function processing unit, this computer program reads and writes necessary data such as a table from a database (DB; Data Base) stored in a storage device or a storage area on a memory, and may be related in some cases. This is realized by controlling hardware (for example, an input / output device, a display device, and a communication interface device). In addition, the database in the embodiment of the present invention may be a commercial database, but it simply means a collection of tables and files, and the internal structure of the database itself does not matter.

  FIG. 6 is a flowchart showing the processing operation of the self-propelled traveling device of the present system. Note that the processing described below does not necessarily have to be performed in the order shown in this flowchart, and the processing order may be changed as long as the relationship between input data and output data of each processing block is not impaired.

  As shown in FIG. 6, when the self-propelled traveling device 10 receives an activation command (step S <b> 11), it measures the shape of the room (room information) by traveling in the room. (Step S12). Specifically, the self-propelled traveling device 10 is activated by receiving an activation command from the power switch or the management device 20. It is assumed that the activated self-propelled traveling device 10 can automatically travel in the room by the automatic traveling means 14, can bypass an obstacle, or can travel over an obstacle. Then, the position detection unit 12 always acquires the position information of the travel route. Further, when the floor plan information is registered in the management device 20, the corresponding floor plan information is received from the management device 20, and therefore step S12 is omitted.

  Then, the obstacle information collecting means 17 determines a predetermined position as a reference point of coordinates among the position information acquired by the position detecting means 12 (step S13). The reference point may be, for example, a position where the self-propelled traveling device 10 is activated, or a corner (corner) in the room.

  The obstacle determination means 13 of the self-propelled traveling device 10 that has acquired the room information uses each sensor (step sensor 31, illuminance sensor 32, friction sensor 33, contact detection) of the sensor group 16 to determine a walking obstacle location. The sensor 34, the temperature sensor 35, the acceleration sensor 36, and the battery sensor 37) are activated (step S14). Next, when an obstacle is found while traveling and the obstacle determination means 13 determines that the obstacle is an aerial obstacle at a position equal to or higher than the “predetermined height” (step S15 “YES”). Depending on the height of the position of the obstacle in the air, when the self-propelled traveling device 10 is in the form of FIG. 3A, an instruction to extend the head in the height direction is sent. When the self-propelled traveling device 10 receives the extension instruction, the self-propelled traveling apparatus 10 extends the head of the imaging unit 15 to the height at which the instruction is given (step S17). The obstacle determination unit 13 measures the height of the air obstacle, acquires detailed information of the determined air obstacle (for example, captures an image of the air obstacle), and receives the obstacle information collection unit 17. (Step S18) The “predetermined height” used for the determination at this time can be changed depending on the setting. For example, an object at a position of 30 cm or less is determined as an obstacle on the floor surface, and an object at a position exceeding that is determined. Judge as an airborne obstacle.

  If the obstacle determination means 13 determines that the obstacle is not an air obstacle (step S15 “No”), it further determines whether or not the obstacle is a floor obstacle, and the floor obstacle If so, detailed information of the floor obstacle is acquired (for example, an image of the floor obstacle is taken) and transferred to the obstacle information collecting means 17 (step S16).

  The obstacle information collecting means 17 wirelessly transmits the collected information to the management device 20 via the communication means 11 each time a predetermined number (a predetermined range) of walking obstacles is acquired (step S19 “YES”). (Step S20). On the other hand, when the obstacle information collecting means 17 has not acquired a fixed number of walking obstacle locations and room information (“NO” in step S19), the obstacle information collecting means 17 returns to step 14 and the obstacle determining means 13 has not finished the determination. Travel around the place and determine where you are walking. The obstacle information collecting means 17 may wirelessly transmit information to the management device 20 every time a walking obstacle location is acquired. Moreover, it returns to S14 until it receives the stop command of the self-propelled traveling device 10, and it travels indoors in order to determine a walking obstacle location. Note that the self-propelled traveling device 10 can determine the search end condition internally without receiving a stop command from the outside, and can automatically end the search for an obstacle.

  Next, the processing operation of the management apparatus of this system will be described. As shown in FIG. 7, when the control means 25 of the management device 20 receives the walking obstacle location information and the room information (step S31), the control means 25 stores the walking obstacle location information in the walking obstacle location information DB 100, and stores the room information. The image is stored in the drawing DB 200 and an image processing command is sent to the image processing means 22. Subsequently, the image processing means 22 that has received the image processing command performs image processing based on walking obstacle location information and room information (step S32).

  Specifically, when the image processing means 22 performs planar image processing (a process in which the position of the obstacle is superimposed on the floor plan and the image of the obstacle is associated with the position) (step S32 “planar image”), The walking obstacle location information and the room information are extracted from the walking obstacle location information DB 100 and the floor plan DB 200, and two-dimensional image processing is performed (step S33). In addition, about the house where the floor plan DB200 is registered into the management apparatus 20, the information is extracted from the floor plan DB200, and plane image processing is performed based on the walking obstacle location information.

  On the other hand, when performing stereoscopic image processing (processing for associating an image of an obstacle with the position of the obstacle on the three-dimensional floor plan) (step S32 “stereoscopic image”), imaging information and obstacles from the walking obstacle location information DB 100 are used. Stereoscopic image processing is performed by obtaining distance information from the determination means 13 to the obstacle (step S34). Note that planar image processing and stereoscopic image processing may be used together for one obstacle.

  Next, the image generation unit 24 generates image data based on the image processing of the planar image or the stereoscopic image (step S35). Subsequently, the control unit 25 displays the image data by transmitting the image data to an external information terminal (for example, a customer terminal) via the display unit 23 or the communication unit 21 of the management apparatus 20 (step S36). ).

  For example, as shown in the screen example of FIG. 8, when existing floor plan data exists, a floor plan of the house is displayed on the screen, and the right part of this screen displays the walking obstacle location to be displayed. Specify the item. As already described, when there is no existing floor plan data, the self-propelled traveling device 10 travels through the room without fail and collects information on the obtained travel route and untravelable points to obtain a simple floor plan. Generate. For generating the floor plan, captured indoor images may be used. In the screen example of FIG. 8, the step position and the position of the obstacle are specified. Here, the step detected as an obstacle is indicated by a ☆, and the other detected obstacle is indicated by a ★. ing. Furniture and the like are indicated by shading. The detailed information of the obstacle is output by clicking on the mark of ☆ or ★ with a mouse or the like. For example, if you click on the ☆ mark on a step (including a threshold), the image and height of that step are displayed. In particular, if you click on the * mark indicated by reference numeral 90, an image of an aerial obstacle in the kitchen is displayed as shown in FIG. 8B, and it can be confirmed on the image that the drawer of the kitchen is half open. . The image at this time is taken from the position “A” in FIG. Therefore, the resident etc. can easily grasp the dangerous place (“careful point”) in the residential environment by checking such a screen as needed. Although not shown, it is also possible to separately display the floor obstacle and the air obstacle on the floor plan.

  FIG. 9 is a diagram showing functional blocks of another configuration of the self-propelled traveling device. In the present embodiment, only the self-propelled traveling device 10 does not go through the control of the management device 20 and automatically detects a walking obstacle location and notifies a resident or the like to grasp the walking environment in the residence. .

  The self-propelled traveling device 60 according to the present embodiment includes, as one configuration example, a communication unit 61, a position detection unit 62, an obstacle determination unit 63, an automatic traveling unit 64, an imaging unit 65, and a sensor group 66. Obstacle information storage means 67, rechargeable battery 68, floor plan acquisition means 70, floor plan generation means 71, voice interaction means 72, image generation means 73, obstacle confirmation means 74, The image projection means 75 and the control means 76 are provided. The obstacle determination means 63 and the sensor group 66 are collectively referred to as an obstacle detection means 69.

  The communication means 61 is the communication means 11, the position detection means 62 is the position detection means 12, the automatic running means 64 is the automatic running means 14, the imaging means 65 is the imaging means 15, and the sensor group 66 is The sensor group 16, the rechargeable battery 68, the rechargeable battery 18, and the obstacle detection means 69 have the same functions as those of the obstacle detection means 19 and the functional block diagram of FIG. .

  The floor plan acquisition means 70 acquires floor plan data from the portable terminal 80 possessed by a resident or the like and stores it. The floor plan data acquired here may be bitmap data or vector data.

  The floor plan generation means 71 travels through the corresponding room and hits an obstacle when the floor plan data does not exist in the floor plan acquisition means 70 or when there is no floor plan data that can be acquired anywhere. It has a function of storing a position where travel is disabled, capturing an image of the entire room by the imaging unit 65, collecting a shape (contour), and generating a simple floor plan.

  The voice interaction means 72 has a built-in microphone and speaker. When the obstacle detection means 69 finds a new obstacle, it appropriately notifies the resident or the like by voice. Moreover, the function which performs the voice dialogue for acquiring the resident's characteristic information in the obstacle determination means 63 is provided.

  In addition to the role of the obstacle determination unit 13 shown in FIG. 4, the obstacle determination unit 63 acquires the resident characteristic information from the voice interaction unit 72 and sets a determination criterion based on the characteristic information. The characteristic information is physical information related to the walking of the resident (including age, height, weight, visual acuity, hearing, etc., including information on a support level and a care level). Judgment standards for obstacles are set according to the level of support required or the level of care, such as lowering of the legs and weakness of the legs. In addition, for example, when determining the illuminance state in a room according to time of day or season, a resident with low visual acuity is determined to have a walking obstacle even in an indoor place with low illuminance by reducing the illuminance setting. . In addition, in the case of a resident with weak legs and legs, if a step is detected, it is determined that the gait is a walking obstacle even if the height of the step is 1 cm. In addition, the height standard for determining an airborne obstacle is changed according to the height of the resident. In addition, as a method for acquiring the characteristic information, for example, the resident or the like may be inquired in a question format by the voice interaction means 72. Alternatively, characteristic information may be input in advance to the mobile terminal 80 and transmitted to the self-propelled traveling device 60.

  In addition, it is preferable that the obstacle determination unit 63 searches for the same place a plurality of times in consideration of an error of a detection unit such as a sensor. At this time, the self-propelled traveling device 10 searches for different movements at each time in a plurality of searches. For example, when searching for a room vertically for the first time, searching horizontally for the second time, searching for the room clockwise for the first time, searching counterclockwise for the second time, etc. Change the movement. In addition, by searching the same place multiple times, a place that has been determined to be a “careful point” more than a predetermined number of times is designated as a “careful point”, and a place that has been determined as a “careful point” less than a specified number of times is designated as a “careful point” For example, a “lightening point” may be given a weight of danger.

  The obstacle information storage unit 67 records the captured image captured by the imaging unit 65 and the information of the obstacle detected by the obstacle detection unit 69 together with the date / time information. The reason why it is recorded together with the date and time information is that it is necessary to change the criteria for determining whether the object is an obstacle as time passes and depending on the season. For example, even a small obstacle that had no problem at the age of 65 can become an obstacle at the age of 75. Moreover, since the sun is early in the winter, even if the low illuminance location is not detected in the summer, it is necessary to re-examine the low illuminance location if the season changes. In addition, by the obstacle confirmation means 74 described later, based on the date and time information recorded together with the obstacle information, the resident or the like can confirm the location only for the newly discovered location, or after a certain time has elapsed since the previous search. May automatically start a re-search.

  The image generation means 73 generates an image in which the position of the obstacle is superimposed on the indoor floor plan. Specifically, based on the room information acquired from the floor plan acquisition unit 70 or the floor plan generation unit 71, a room image showing floor obstacles and aerial obstacles on the floor plan is generated. The floor plan that can be handled here is not limited to a two-dimensional floor plan, but may be a three-dimensional floor plan generated from a panoramic photograph.

  The obstacle confirmation means 74 causes the resident to confirm the obstacle information stored in the obstacle information storage means 67 ("Hidden point"). Delete from the obstacle information storage means 67. Specifically, as a confirmation method, a method of confirming from an information image of an obstacle projected on a wall surface or the like from the image projecting means 75, or a portable terminal 80 possessed by a resident or the like via the communication means 61, There is a method for transmitting information images to allow residents to confirm. However, regarding the deletion of obstacle information, there is a possibility that it will become an obstacle again in the future even if there is no problem at present, so it is desirable to keep at least the deletion history.

  As a method for confirming the “care points”, not only a method of projecting an image on a wall surface or the like as described above or a method of transmitting an image to the portable terminal 80 but also a method of notifying by voice may be used. For example, if a resident actually approaches the discovered “careful point”, the self-propelled traveling device will follow up and emit a warning sound, or the device will circulate in the room with the resident. , You may be allowed to check each discovered “careful point” on the spot. At this time, the intention of confirmation from the resident or the like may be displayed by voice or by performing a predetermined operation on the apparatus. By doing in this way, it is possible to make the resident or the like confirm the newly discovered “careful points”. This is because whether or not the “careful point” discovered is really a dangerous place depends on the individual resident and changes with time. This is particularly important not only in private housing but also in apartment houses such as nursing homes.

  The image projecting unit 75 is, for example, a projector, and projects an image in which the position of an obstacle is superimposed on the indoor floor plan generated by the image generating unit 73 on a wall surface in the room. Accordingly, the resident or the like can check the position of the obstacle and its detailed information while looking at the large floor plan projected on the wall surface.

  The control means 76 incorporates a CPU (microprocessor) and a memory, and plays a role of controlling each of the functional blocks 65 to 75 as the CPU sequentially reads and executes a program recorded in the memory.

(Effect of embodiment)
As described above, according to the present system, by using a self-propelled traveling device and running in a living space, walking obstacles such as steps are automatically detected, the management device performs image processing, and the resident In addition, it is possible to construct a system that can notify the family of the dangerous place so that the walking environment can be grasped and dealt with. be able to.

  In addition, the management device generates an image in which walking obstacles are superimposed on the room plan. By doing in this way, a gait obstacle location can be displayed visually.

  In addition, the self-propelled traveling device includes a friction sensor that measures the degree of friction with the floor surface, thereby detecting a rotational resistance during traveling by the automatic traveling means and measuring the degree of friction, thereby causing a slippery walking obstacle. The location can be specified.

  In addition, the self-propelled traveling device detects a three-dimensional walking obstacle part (aerial obstacle) from an image of an object acquired during traveling, for example, to detect an aerial obstacle that cannot be confirmed in plan view. It becomes possible.

  Moreover, the self-propelled traveling device includes a temperature sensor, and can determine a walking obstacle location having a low temperature by measuring the ambient temperature during traveling.

  In addition, the self-propelled traveling device includes a contact detection sensor, and an object located at a high position that cannot be detected by the self-propelled traveling device traveling on the floor surface by the imaging means or the contact detection sensor while traveling is in the air. It can be detected as an obstacle. If the imaging means can be vertically expanded and contracted, an object can be imaged from a high position, so that it is easier to determine an airborne obstacle.

  Further, when detecting an obstacle, the self-propelled traveling device can increase the detection accuracy by searching the same place with a plurality of different movements. Further, by searching a plurality of times, it is possible to give a light weight to the danger level of the place according to the number of times the obstacle is detected at the same place.

  In addition, the self-propelled traveling device includes a voice dialogue means, and can notify the resident or the like by voice when the obstacle detection means finds a new obstacle. By doing in this way, a resident etc. can grasp the position of an obstacle clearly.

  Further, the self-propelled traveling device includes an obstacle determination unit, the obstacle determination unit obtains resident characteristic information from the voice dialogue unit, and sets a determination criterion for the obstacle determination unit based on the characteristic information. can do. By doing in this way, it becomes possible to determine an obstruction according to the resident's characteristic information.

  Further, the self-propelled traveling device includes an image projecting unit (projector), and can project an image in which the position of the obstacle is superimposed on the floor plan generated by the image generating unit. By doing in this way, a resident etc. become easy to grasp the position of an obstacle.

  In addition, the self-propelled traveling device has obstacle confirmation means for allowing the occupants to confirm the obstacle information stored in the obstacle information storage means, and information on obstacles that are confirmed to be satisfactory as a result of the confirmation. Can be deleted from the obstacle information storage means.

  Moreover, although this system limited and demonstrated self-propelled traveling apparatus for indoor use, it can be applied also for outdoor use. For example, by moving the self-propelled traveling device 10 through an approach to the entrance, a garden, a parking lot, etc., dangerous areas such as trees and ponds are automatically detected, and the entire site of the house is managed by the management device 20 side. Can be managed and displayed. In addition, the present invention can be similarly applied not only to a residential area but also to a construction site. Furthermore, the present invention can be applied to a city dangerous spot investigation system that makes an outdoor self-propelled traveling device circulate around the city and investigate steps on a sidewalk.

  As mentioned above, although this invention was demonstrated using embodiment, it cannot be overemphasized that the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiments. Further, it is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Self-propelled traveling apparatus 11 Communication means 11a Antenna 12 Position detection means 13 Obstacle judgment means 13a Floor obstacle judgment means 13b Aerial obstacle judgment means 14 Automatic travel means 15 Imaging means 16 Sensor group 17 Obstacle information collection means 18 Rechargeable battery 19 Obstacle detection means 20 Management device 21 Communication means 22 Image processing means 23 Display means 24 Image generation means 25 Control means 31 Step sensor 32 Illuminance sensor 33 Friction sensor 34 Contact detection sensor 35 Temperature sensor 36 Acceleration sensor 37 Battery Sensor 38 Gyro sensor 50 Driving means 51 Auxiliary wheel 60 Self-propelled traveling device 61 Communication means 62 Position detecting means 63 Obstacle determining means 64 Automatic traveling means 65 Imaging means 66 Sensor group 67 Obstacle information storage means 68 Rechargeable battery 69 Obstacle Object detection means 0 between floor plan obtaining means 71 Mato view generation unit 72 speech dialogue unit 73 image generating unit 74 obstacle verifying unit 75 the image projecting means 76 control means 90 aerial obstacle imaging point 100 gait failure location information DB
200 Floor plan DB

Claims (12)

A self-propelled traveling device that automatically travels indoors,
Means for acquiring or generating floor plan data in the room;
Position detecting means for detecting its own position in the room;
Obstacle detection means for detecting an obstacle that may become a walking obstacle in the room;
An imaging means for capturing an image of surroundings during automatic driving;
An obstacle information storage means for recording a captured image captured by the imaging means and information of the obstacle detected by the obstacle detection means together with date and time information;
A self-propelled traveling device comprising: output means for displaying or outputting information of the obstacle stored in the obstacle information storage means by voice or sound . And the output means, e Bei the image generation means for generating an image of repeated position of the obstacle between the indoor floor plan,
The self-propelled traveling device according to claim 1, wherein the output unit causes the display device to display the image generated by the image generation unit .   The obstacle detection means includes a friction sensor that measures the friction with the floor surface, and detects a rotational obstacle during automatic running and identifies the slippery walking obstacle by measuring the friction. The self-propelled traveling device according to claim 1 or 2, characterized by the above.   The obstacle detection means includes an aerial obstacle determination means, and the aerial obstacle determination means determines an object located within a predetermined height range as an aerial obstacle from an image of the object acquired during traveling. The self-propelled traveling device according to any one of claims 1 to 3.   5. The obstacle detection unit according to claim 1, wherein the obstacle detection unit includes a temperature sensor, and identifies a walking obstacle portion having a low temperature by measuring an outside air temperature during traveling. Self-propelled traveling device.   The self-propelled traveling device according to claim 4, wherein the obstacle detection means includes a contact detection sensor, and detects the air obstacle by the contact detection sensor during traveling. And the output means, the wall surface of the chamber, according to claim, characterized in that it comprises an image projection means for projecting an image of the superimposed position of the obstacle to the floor plan between the chamber in which the image generating means has generated 2 The self-propelled traveling device described in 1.   The self-propelled traveling device has obstacle confirmation means for allowing a resident or a watcher to confirm the information of the obstacle stored in the obstacle information storage means, and the obstacle that is confirmed to be satisfactory as a result of the confirmation. 8. The self-propelled traveling device according to claim 7, wherein the information on the object is deleted from the obstacle information storage means. A self-propelled traveling device that automatically travels indoors,
  Means for acquiring or generating floor plan data in the room;
  Position detecting means for detecting its own position in the room;
  Obstacle detection means for detecting an obstacle that may become a walking obstacle for the resident in the room based on the characteristic information of the resident;
  An imaging means for capturing an image of surroundings during automatic driving;
  An obstacle information storage means for recording a captured image captured by the imaging means and information of the obstacle detected by the obstacle detection means together with date and time information;
  A self-propelled traveling device comprising: The self-propelled traveling device includes a voice interaction means,
The obstacle detection means includes obstacle determination means, the obstacle determination means obtains the resident characteristic information from the voice dialogue means, and the obstacle determination means determines based on the characteristic information The self-propelled traveling device according to claim 9 , wherein a reference is set. A walking obstacle location determination system comprising a self-propelled traveling device that automatically travels indoors, and a management device that communicates wirelessly with the self-propelled traveling device and notifies a walking obstacle location in the room,
The self-propelled traveling device is:
Position detecting means for detecting its own position in the room;
Obstacle detection means for detecting an obstacle that may become a walking obstacle in the room;
An imaging means for capturing an image of surroundings during automatic driving;
Obstacle information collecting means for collecting the captured image taken by the imaging means and information on the obstacle detected by the obstacle detecting means,
With
The management device
A walking obstacle location determination system comprising output means for displaying or outputting information of the obstacle collected by the obstacle information collecting means by voice or sound . A walking obstacle location determination system comprising a self-propelled traveling device that automatically travels indoors, and a management device that communicates wirelessly with the self-propelled traveling device and notifies a walking obstacle location in the room,
The self-propelled traveling device is:
Position detecting means for detecting its own position in the room;
Obstacle detection means for detecting an obstacle that may become a walking obstacle for the resident in the room based on the resident characteristic information ;
An imaging means for capturing an image of surroundings during automatic driving;
Obstacle information collecting means for collecting the captured image taken by the imaging means and information on the obstacle detected by the obstacle detecting means,
With
The management device
Image generating means for generating an indoor image including a walking obstacle portion obtained by image processing the captured image collected by the obstacle information collecting means and information on the obstacle;
A walking obstacle location determination system characterized by comprising:

Images