KR102537450B1 - Automatic map generation system for the vulnerable using deep learning-based object recognition technology - Google Patents

Abstract

The present invention relates to a system for creating maps for the transportation vulnerable and providing customized route guidance. More specifically, the present invention relates to a system for creating maps for the transportation vulnerable and providing customized route guidance, which enables users of electric mobility aids including electric wheelchairs to have help by promptly notifying outsiders of a falling accident if the falling accident occurs due to overturning of the electric mobility aid, can continuously monitor the condition of the road surface on which the electric mobility aid travels, thereby being easily used to improve the road environment including maintenance and repair of the road surface, can analyze the driving pattern and driving posture of the user who uses the electric mobility aid, thereby taking measures to prevent diseases such as bedsores if the user is riding the electric mobility aid for a long time, and can provide customized route guidance services for each type of the transportation vulnerable by establishing movement routes or spatial information for the convenience of movement of the transportation vulnerable including the disabled, through voluntary participation of the user or the transportation vulnerable using the electric mobility aid. The system comprises: a map information collection device; and a server including a database module, a data purifying module, a data labeling module, and a disabled access analysis module.

Description

Automatic map generation system for the vulnerable using deep learning-based object recognition technology}

The present invention relates to an automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology. More specifically, map information may be collected using the first map information collection device and the second map information collection device, and based on the collected map information, a map exclusively for the transportation handicapped may be automatically generated for the transportation handicapped. It is about an automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology.

Recently, in Korea, the demand for assistive devices is continuously increasing along with interest in the welfare system for the disabled and the elderly due to the expansion of the category of disabilities, the increase in the incidence of disabilities due to acquired causes, and the entry into a super-aged society. As the demand for assistive devices increases, the national public benefit budget for assistive devices also tends to increase rapidly.

Electric mobility aids, including electric wheelchairs, are medical devices mainly used for mobility of the disabled who cannot move freely due to congenital or acquired causes, and at the same time, they are representative walking aids that provide the convenience of movement to people with disabilities such as the elderly.

Currently, the transportation vulnerable who use wheelchairs have to take various types of risks for movement, and help to improve the mobility of wheelchairs, prevent tipover or fall accidents of wheelchairs, and respond to accidents in the event of an accident. system is required.

In the event of a fall accident with electric mobility aids, prompt response is required, but the actual countermeasures are limited to passive situations such as reporting by the person concerned, self-overcoming, assisting and reporting by witnesses of the accident, etc., and solving social issues of users. There is no integrated system or infrastructure composed of functions.

Therefore, there is a need for a route guidance and route recommendation system exclusively for the transportation vulnerable that can prevent these falls accidents in advance, and it is possible to respond quickly to safety accidents for users of electric mobility aids, and to improve the living environment and convenience of users of electric mobility aids. It is necessary to develop a system that can improve the road environment by utilizing data on accumulated road surface conditions.

However, since it takes a lot of money to build a database of mobility support facilities for the way guidance service used by the disabled or the transportation-impaired using a wheelchair, the service is in progress only in limited areas in many countries including Korea.

Therefore, with respect to this route guidance system, through the voluntary participation of users who use electric mobility aids or the transportation vulnerable, movement routes or spatial information for the convenience of the disabled or transportation vulnerable are established to provide route guidance services suitable for each type of transportation vulnerable. You need a system that can be shared.

Prior art literature: KR Patent Publication No. 10-2016-0047161 (published on May 2, 2016)

The present invention has been made to solve the above problems, and map information data is obtained by using a first map information collection module that can be carried by ordinary people and a second map information collection module that can be installed in electric mobility aids. Deep-learning-based object recognition technology that can provide route guidance services suitable for each type of transportation vulnerable by collecting data and create customized maps including tour course maps that can be conveniently used by the transportation vulnerable using electric mobility aids The purpose is to provide an automatic map generation system exclusively for the transportation vulnerable using

In order to achieve the above object, the automatic map generation system for the transportation disadvantaged using deep learning-based object recognition technology according to the present invention is a map information collection device capable of collecting road surface data including sidewalks and roads; A database module that can receive and store the data collected from the map information collection module, a data purification module that can exclude image information and image information that cannot be used for data learning from the data collected from the map information collection module, and an object A data labeling module capable of labeling selected objects by selecting a data labeling module, figuring out the dynamic route of the transportation vulnerable using electric wheelchairs, and searching for areas that are easily accessible to the transportation vulnerable through GIS spatial statistical analysis including major gathering places and major access areas. It includes a server that includes a disabled accessibility analysis module that can be used.

In addition, the map information collection device includes a first map information collection device that is carried by a map information collector and is capable of collecting map information data; and a second map information collection device installed in the motorized mobility aid and capable of collecting map information data, wherein the first map information collection device inputs personal information including name, ID, password, mobile phone number, and account information and inputs member information. Member registration department where you can sign up; A collection history inquiry unit where you can search the collection history of road surface data including sidewalks and roads, another history inquiry unit where you can select a date and search the road surface collection history of that date, and you can search the shooting history information and collection object list. a history inquiry unit including a detailed history inquiry unit, a collection object confirmation unit that selects a collection object from the collection object list and inquires information about the collection object; Preliminary information input unit to enter information including indoor/outdoor status, filming location, day and night, weather, automatic shooting selection unit to select automatic and manual shooting, guidance unit to indicate current location, shooting through camera A road surface information collection unit including a photographing screen unit that visualizes and displays a screen being displayed; and date selection unit to select the date for checking the reward, total revenue output unit to output total revenue, settlement process guide unit to guide the settlement process, collection object history output to output collected object history information including a reward portion comprising a portion; And the second map information collection device is an acceleration sensor for detecting the dynamic posture of the electric mobility aid, including the movement of the electric mobility aid, vibration and shock applied to the electric mobility aid, and the inclined posture of the electric mobility aid A movement detection sensor unit including a gyro sensor for detecting, obtaining location information of a user of an electric mobility aid to measure a driving distance and a driving path, and a user riding on an electric mobility aid When a fall accident occurs, it includes a driving location information sensor unit provided to track the location where the fall accident occurred, and a photographing unit provided to photograph the state of the road surface on which the electric mobility aid is driven.

In addition, the server includes a map generating module capable of displaying an optimal driving route of the electric mobility aid using the map information collected by the first map information collection device and the second map information collection device.

According to the present invention, there is an effect of preventing life-threatening accidents due to secondary accidents by supporting rapid response when a fall accident occurs in an electric mobility aid.

In addition, road surface information and space information are obtained while riding on the electric mobility aid, and based on this, road surface information is output in real time while driving, and customized route guidance for the transportation disadvantaged who rides the electric mobility aid is possible. there is

In addition, according to the present invention, by providing a map generation module of the server, based on AI algorithms such as data purification, information collection and registration for road guidance, electronic map mapping, electronic map update, and data learning, , can be acquired quickly, and it is effective in providing more customized route guidance services to users through repeated data learning.

In addition, according to the present invention, it is effective to create and provide a customized map including a tour course map that can be conveniently used using an electric mobility aid.

1 is a configuration diagram showing the configuration of an automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology according to a preferred embodiment of the present invention;
2 is a view showing a history inquiry unit of a first map information collection module;
3 is a view showing another history inquiry unit of the first map information collection module;
4 is a diagram showing a detailed history inquiry unit of a first map information collection module;
5 is a diagram showing a collection object confirmation unit of a first map information collection module;
6 is a diagram showing a dictionary information input unit of a first map information collection module;
7 is a diagram showing a road surface information collection information of a first map information collection module;
8 is a diagram showing a setting unit of a first map information collection module;
9 is a view showing a reward unit of a first map information collection module;
10 is a diagram visualized by point density of a place where the transportation vulnerable of the disabled accessibility analysis module frequently moves or gathers;
11 is a diagram showing places where the transportation vulnerable often move or gather in the accessibility analysis module for the disabled on a map;
12 is a view showing that a theme tour course for the transportation disadvantaged person is created in the theme tour course creation module.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. First, in adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted. In addition, although preferred embodiments of the present invention will be described below, the technical idea of the present invention is not limited or limited thereto and can be modified and implemented in various ways by those skilled in the art.

1 is a configuration diagram showing the configuration of a system for automatically generating a map exclusively for the transportation disadvantaged using deep learning-based object recognition technology according to a preferred embodiment of the present invention, and FIG. 2 is a diagram showing a history inquiry unit of a first map information collection module. FIG. 3 is a view showing another history inquiry unit of the first map information collection module, FIG. 4 is a view showing a detailed history inquiry unit of the first map information collection module, and FIG. 5 is a collection object of the first map information collection module. Figure 6 is a diagram showing the confirmation unit, Figure 6 is a diagram showing the preliminary information input unit of the first map information collection module, Figure 7 is a diagram showing the road surface information collection information of the first map information collection module, Figure 8 is a diagram showing the first map information collection module Figure 9 is a diagram showing the setting unit of the module, Figure 9 is a diagram showing the reward unit of the first map information collection module, Figure 10 is a diagram visualized by point density of places where transportation vulnerable people frequently move or gather in the accessibility analysis module for the disabled 11 is a diagram showing places where transportation disadvantaged people frequently move or gather on a map in the accessibility analysis module for the disabled, and FIG. 12 is a diagram showing that a theme tour course for the transportation disadvantaged is created in the theme tour course creation module.

Referring to FIG. 1, a system for automatically generating a map exclusively for the transportation disadvantaged using deep learning-based object recognition technology according to a preferred embodiment of the present invention includes a map information collection module 100 and a t-server 200.

First, the automatic map generation system for the transportation vulnerable using the deep learning-based object recognition technology according to the present invention can recognize movement-related objects of the transportation vulnerable through a photo obtained together with the coordinate information, thereby automatically creating a map for the transportation vulnerable. It is characterized by being able to provide customized route guidance to the transportation vulnerable who board the electric mobility aid by creating a mobile device.

In addition, the condition of the road surface on which the electric mobility aid drives is continuously monitored and accumulated as data, and the information on the monitored road surface condition can be easily used to improve the road environment including maintenance and repair of the road surface. there is.

In addition, it is characterized in that customized information can be shared for each type of the transportation vulnerable by allowing the user to directly participate when creating a route guidance or navigation system for the transportation vulnerable or performing an update on the created electronic map.

In addition, it can provide information on the route the electric mobility aids travel, the current status of buildings where the electric mobility aids can enter the visiting area, and convenience facilities including charging locations for the electric mobility aids. there is.

In addition, a theme tour course for the transportation disadvantaged can be created and provided, which can create customized maps including a map of recommended spots for easy accessibility for the disabled analyzed by the accessibility analysis module and a tour course map that can be conveniently used using electric mobility aids. It can be done, but it has its own characteristics.

Hereinafter, the components constituting the automatic map generation system for the transportation disadvantaged using deep learning-based object recognition technology according to a preferred embodiment of the present invention will be described in detail with reference to FIG. 1 attached.

The map information collection device 100 may collect road surface data including sidewalks and roads, and includes a first map information collection device 100a and a second map information collection device 100b.

The map information collection module 100 of the present invention is installed in the first map information collection module 100a that can collect map information data while possessed by the general public, and is installed in an electric mobility aid to provide information about places where the transportation disadvantaged person mainly uses. By dividing the second map information collection module (100b) that can collect map information data, the map information collected by the general public and the map information installed in the electric mobility aid are collected by dividing them into two categories. Its feature is that it can create maps suitable for people with transportation disabilities.

The first map information collection device 100a is possessed by a general worker and can collect map information data, and includes a member signup unit 10, a history inquiry unit 20, a road surface information collection unit 30, a reward unit ( 40) is included.

The first map information collection device 100a may be a smart device on which a map information collection app is installed, and a user logs in to a map information collection app installed in the smart device and uses a map information collection app linked to a camera built into the smart device. It can be used to collect road surface objects.

In addition, the user can inquire the collected road surface objects, the user can transmit and delete the collected road surface objects to the server, and the user can inquire the points assigned to the collected road surface objects on a monthly basis.

The member registration unit 10 may sign up as a member by inputting personal information including name, ID, password, mobile phone number, and account information of a person collecting map information through the first map information collection module.

Referring to FIG. 2 , the history inquiry unit 20 includes a collection history inquiry unit, another history inquiry unit, a detailed history inquiry unit, and a collection object identification unit, and can inquire the history of collecting road surface information.

More specifically, through the collection history inquiry unit, road surface data collection history including sidewalks and roads can be inquired, and collection history including data collection date, number of objects collected in photographing history, etc. can be inquired.

In addition, referring to FIG. 3, in another history inquiry unit, a date can be selected from a calendar image, a date with a data collection history can be displayed in green, etc., and when data is transmitted to the database module, a red circle is displayed at the bottom of the date. If there is data that has not been transmitted to the database module, a gray circle may be displayed at the bottom of the corresponding date.

In addition, referring to FIG. 4 , the detailed history inquiry unit can inquire the collection object list including the date the data was captured and the photo.

Referring to FIG. 5 , the collection object confirmation unit selects a collection object from the collection object list and inquires information about the collection object including a photo, and detailed information including time taken by the collection object and image information of the collection object. can be viewed.

The road surface information collection unit 30 may collect road surface information, and includes a preliminary information input unit, an automatic shooting selection unit, a guidance unit, a shooting screen unit, and a camera unit.

Referring to FIG. 6 , the advance information input unit can select whether the shooting location is outdoors or indoors before shooting using the camera unit, and the camera unit includes the address of the shooting location, whether it is daytime or nighttime, and weather information. You can input necessary assessment information before taking a picture using it.

Referring to FIG. 7 , the automatic shooting selection unit can select whether to automatically take a picture or manually take a picture using the camera unit.

The map display unit may display the current location of the holder of the first map information collection module through a map.

The camera unit can capture images and photos, and the capturing screen unit can visualize and display a screen being captured through the camera unit.

Referring to FIG. 9, the reward unit outputs a date selection unit for selecting a reward confirmation date, a total revenue output unit for outputting total revenue, a settlement process information unit for guiding the settlement process, and information on collected object history. It is configured to include a collection object history output unit that can

The second map information collection device 100b is installed in an electric mobility aid to collect map information data, and includes a movement detection sensor unit 110, a driving location information sensor unit 120, and a photographing unit 130. It is composed by

The second map information collection device 100b not only collects map information data from the electric mobility aid, but also the movement detection sensor unit 110 includes an acceleration sensor and a gyro sensor, and the user of the electric mobility aid It is used to obtain information about the road surface condition, including the degree of inclination of the road surface and the subsidence of the road surface when driving on the road surface while riding the electric mobility aid.

The acceleration sensor is provided to detect the dynamic posture of the electric mobility aid, including movement of the electric mobility aid, shaking, vibration, and impact applied to the electric mobility aid.

The gyro sensor is provided to detect an inclined posture of the electric mobility assistance device, including the angular velocity of the electric mobility assistance device and the tilted state of the road surface on which the electric mobility assistance device is running.

In addition, the second map information collection module 100b may also include a pressure measurement sensor unit, and the pressure measurement sensor unit detects whether the user is on board or falls by measuring the pressure in the seated state of the user riding the electric mobility aid. In order to do so, one or more are provided in the seat or cushion of the electric mobility aid.

The driving location information sensor unit 120 obtains location information of the electric mobility aid that the user rides to measure the driving distance and driving path, and causes the user riding the electric mobility aid to fall due to the overturning of the electric mobility aid. When an accident occurs, the driving pattern posture determination module of the server, which is provided to track the location of the fall accident, is used to determine the driving pattern of the user using the electric mobility aid.

In addition, the driving location information sensor unit 120 is used to obtain location information of the user of the electric mobility aid including GPS, and a fall accident caused by the overturning of the electric mobility aid by the user riding the electric mobility aid In this case, the location of the fall accident is tracked.

The photographing unit 130 is provided to capture an image of the condition of the road surface on which the electric mobility aid is traveling, and the image acquired by the photographing unit is transmitted to the server 200 described below and the road surface on which the electric mobility aid is driven. It is used as data to determine whether the state is abnormal and acquire road surface information, space information, fixed object and moving object information from the photographed copy.

That is, the second map information collection module 100b of the present invention is installed in the motorized mobility aid to perform a function of collecting map information data, as well as to prevent a user riding the motorized mobility aid from falling. It can detect accidents, including falls, and track the location of falls.

The server 200 receives the map information transmitted from the map information collection module 100, processes and stores the received information, and stores the stored information externally, including the electric mobility aid user app provided in the user's smart device. It is transmitted to the device, and it is composed of a rollover fall determination module 210, a map generation module 220, a road surface condition determination module 230, and a driving pattern attitude determination module 240, and performs the following specific functions. do.

The rollover fall determination module 210 of the server 200 collects information about the posture of the electric mobility aid transmitted from the movement detection sensor unit and the pressure measurement sensor unit included in the second map information collection module 100b, It determines whether the mobility aid has overturned and whether the user has fallen.

The map generation module 220 is a movement detection sensor unit, a pressure measurement sensor unit, and a driving location information sensor unit of the second map information collection module 100b through driving of the first map information collection module 100a and the electric mobility assist device. , It is configured to create a dedicated map that can provide the optimal driving route of electric mobility aids through pre-processing by receiving information collected by the camera unit.

In addition, the map generation module 220 includes a data refinement module 221, a data labeling module 222, a mobile obstacle collection prediction unit 223, a registration data review unit 224, a data mapping unit 225, and a road abnormality registration unit. 226, and will be described in detail starting with the data purification module 221 below.

The data purification module 221 includes the map information transmitted from the first map information collection module 100a, the movement detection sensor unit, the pressure measurement sensor unit, the driving location information sensor unit, and the camera of the second map information collection module 100b. We select available data by identifying among the information collected through the department.

In addition, the data purification module 221 selects unusable data, but includes blurry or distorted photographs, tilted horizon angles of more than a certain angle, only road surfaces or no road surfaces, and objects unrelated to the road surface environment. A photographed copy that includes, a photographed copy that does not include an object for guidance for the transportation vulnerable who rides an electric mobility aid, and a photographed copy that includes personal information are judged to be unusable data.

The data labeling module 222 may register information by selecting an object, segmenting the selected object, and includes a road surface information registration unit, a spatial information registration unit, and a fixed obstacle registration unit.

The road surface information registration unit uses the data refined through the data purification module 221 to determine the flatness of the road surface, the material of the footpath (sidewalk), the condition of the pavement, the type and condition of the sidewalk block, the location of the curb, the location of the drain cover, the location of the braille block, and the slope. , and road surface information including the location of the bicycle road is collected, and the information is registered by segmenting it.

The geospatial information registration unit collects spatial information, including the status and location of stairs, the location of walls, the location of windows, the location of columns, the location of elevators, the shape of doors, and traffic direction signs, from the data filtered through the data purification module, and divides them into information. register

The fixed obstacle registration unit collects location information of fixed obstacles including barricade location, bench location, bollard location, stop location, traffic light location, and street tree location from the data filtered through the data purification module, and registers the information by segmenting it. do.

The movable obstacle collection prediction unit 223 collects location information of movable obstacles including bicycle location, vehicle location, other person location, stroller location, and wheelchair location from the data filtered through the data purification module 221 in real time. and analyze the collected information to predict the direction of movement and provide driving notifications.

The registration data review unit 224 checks whether the matching between the partitions of the data registered in the road surface information register, the spatial information register, and the fixed obstacle register and the information registered in the specified and partitioned object is correct, and It is checked whether there is an error, but the criterion for checking this is to check whether the partition of the object for each data frame moves according to the movement of the motorized mobility aid in which the second map information collection module 100b is installed, and the partitioned object moves over a certain frame. location to increase data accuracy.

The data mapping unit 225 includes a geospatial information mapping function, reflects and updates the geospatial information reviewed by the registration data review unit in the database of a route guidance program, and has a function of mapping the updated information on an electronic map. carry out

When the road surface condition registration unit 226 determines that the road surface condition is abnormal through the road surface condition determination module of the server in the geospatial information mapping function, the location information of the abnormal road surface area, the distance section of the abnormal road surface, and the magnitude of vibration , road surface condition information including the duration of vibration is reflected in the database of the route guidance program and updated in addition to the information mapped by the data mapping unit.

The road surface condition determination module 230 of the server 200 is operated by the movement detection sensor unit 110 of the second map information collection module 100b when the user of the electric mobility aid rides on the electric mobility aid and drives on the road surface. The magnitude of the vibration measured, the duration of the vibration, the vibration pattern, the gradient of the road surface, and the image signal of the road surface captured by the camera unit are received and stored from the second map information collection module, and the condition of the road surface on which the electric mobility aid is running is analyzed to update the electronic map of the road guidance program through the road abnormality registration unit 226 of the map generation module 220.

The result analyzed through the road surface condition determination module of the server 200 is transmitted from the electric mobility aid user app installed in the user's smart device so that the user can grasp the condition of the road surface on which the user is driving in real time.

On the other hand, when a user with a paraplegic disorder sits on an electric mobility aid for a long time, there is a possibility that diseases such as bedsores may occur due to numbness in the lower body.

At this time, in order to prevent bedsores of the user, it is necessary to take measures to prevent bedsores by informing the user of the boarding time of the electric mobility aid or the user's riding posture. This is done in module 240.

The driving pattern posture determination module 240 of the server 200 receives signals related to the user's sitting state and the driving distance from the pressure measurement sensor unit and the driving location information sensor unit of the second map information collection module 100b and moves by electric motion. It is determined by analyzing the driving pattern and driving posture of the user using the assistive device.

In addition, the driving pattern posture determination module 240 of the server 200 uses the signal transmitted from the pressure measurement sensor unit and the driving location information sensor unit of the second map information collection module 100b to allow the user to access the electric mobility aid. The time spent on boarding and using is measured, and the sitting posture of the electric mobility aid is analyzed.

Specifically, the server 200 sets and stores the continuous boarding time of the electric mobility aid in advance.

If the user gets on and uses the electric mobility aid beyond the set time, or if it is determined that the pressure in a specific part of the signal transmitted from the pressure measuring sensor unit is continuing for too long, the server determines the driving pattern posture The module determines a dangerous driving posture and transmits a pressure sore prevention message to the electric mobility aid user app installed on the user's smart device to prevent the electric mobility aid user from using the electric mobility aid for a long time.

The convenience providing module 250 of the server receives the location information of the user riding the electric mobility aid from the driving location information sensor unit of the second map information collection module, and the electric mobility aid from the electronic map generated by the map generation module. It provides users with information on the route traveled, the current status of buildings where electric mobility aids can enter in the visited area, and convenience facilities including charging locations for electric mobility aids.

On the other hand, if the server 200 determines that a fall accident has occurred because the electric mobility aid on which the user rides has overturned or the user on board has fallen as a result of the decision made by the overturn fall determination module 210, the server 200 determines the overturn or fall accident. It includes a function to send notifications to the outside, including the motorized mobility aid device guardian app.

In addition, the server has a function of transmitting a notification to the outside including the electric mobility assistant user app that the road surface area is abnormal when the road surface condition is judged by the road surface condition determination module 230 to be an abnormal road surface area. This is included.

On the other hand, since the electric mobility aid directly receives the user's weight and travels on the road surface, a bump in the road, a depression, a slope, etc., frequent breakdowns occur due to impact applied to the electric mobility aid.

If the electric mobility aid user rides on the electric mobility aid and the motorized mobility aid does not operate due to a malfunction of the electric mobility aid, the user can use the electric mobility aid user app to activate the motorized mobility aid When the failure state of the device is transmitted to the server, the server can request help by sending a notification to the outside.

In addition, if a fall accident occurs while the user of the electric mobility aid is riding on the electric mobility aid, the user transmits the fact of the fall accident to the server through the electric mobility aid user app. At this time, the location information on the accident point acquired by the driving location information sensor unit of the second map information collection module is also transmitted to the server.

The server that receives the user's fall accident occurrence and location information transmits the fall accident to an external rescue agency, and the rescue agency dispatches rescuers to the accident area to rescue users who have fallen accidents.

In addition, when a rescuer rescues a user of an electric mobility aid in which a fall accident has occurred, the user transmits the accident handling completion status to the server through the electric mobility aid user app.

Here, rescue agencies are operated by rescue agencies under the jurisdiction of local governments, including 119 emergency rescue centers, ward offices, and police stations, and receive fall accident information transmitted from the server and send rescuers to the location of the fall accident so that emergency response measures can be taken.

On the other hand, the server 200 includes the steady-state vibration signal of the road surface and the slope of the steady-state road surface generated when the electric mobility aid is driven according to the finishing materials of the road surface, including granite block, cement block, asphalt, rubber, and clay block. Information about signals is stored in advance, respectively.

Information on the road surface in a normal state stored in the server 200 is obtained through the movement detection sensor unit and the photographing unit of the second map information collection module 100b, or a separate device including an acceleration sensor, a gyro sensor, and a camera is used. It can be obtained by using, and the information on the acquired road surface condition predicts the abnormal road surface condition after learning by artificial intelligence.

That is, the road surface condition determination module 230 of the server 200 determines the normal state of the road surface that occurs when the electric mobility aid drives according to the finishing materials of the road surface including granite block, cement block, asphalt, rubber, and clay block. Using the vibration signal and the slope signal of the normal road surface as learning materials, it is possible to predict the abnormal road surface condition by linking with AI API learned through deep learning method (Convolutional Neural Network, CNN) or machine learning method.

In addition, the learning data is provided with a data amplification unit to increase the learning rate, and the data amplification unit brightens or blurs the brightness of the original image, inverts left and right, moves left and right, rotates, flips up and down, moves up and down, adds noise, etc. Through processing, one image can be used as multiple images to increase the learning rate.

Obtained through the movement detection sensor unit 110 and the photographing unit 130 of the second map information collection module 100b when the user of the electric mobility aid rides on the motorized mobility aid and drives on a road surface with different finishing materials It transmits information about the road surface condition to the server.

In the server 200, if the information about the condition of the road surface transmitted through the road surface condition determination module 230 is greater than the steady-state vibration signal learned and stored in advance according to the finishing material of the road surface, or if the change in slope of the road surface is large, abnormal It is determined by the road surface condition, and information on the road surface condition is sent to the electric mobility aid user app.

Thereafter, in the server 200, the driving location information sensor unit of the IoT-based device transmits the corresponding location to an external agency in order to maintain the road surface at the corresponding location.

On the other hand, the server includes a route guidance program equipped with an electronic map to guide the movement route when the transportation vulnerable, including users using electric mobility aids, moves, and the route guidance program included in the server is It is an assistive device user app and is provided in the form of voice or text.

The geospatial information mapping function of the server 200 determines the location information of the abnormal road surface area, the distance section of the abnormal road surface, and vibration when the road surface condition is determined to be an abnormal road surface condition through the road surface condition determination module 230 of the server. Information on the road surface condition including the size and duration of vibration is reflected and updated in the database of the road guidance program through the road surface anomaly registration unit, and the updated information is mapped on the electronic map.

On the other hand, the electric mobility aid user app includes a user collection information provision function, and the user collection information provision function allows the user using the electric mobility aid user app to directly update the route guidance program provided on the server with the latest information. perform a function in

The user-collected information provision function of the electric mobility aid user app is for the user using the electric mobility aid user app to directly input updated information including geographic or spatial information while on the move through the electric mobility aid user app and transmit it to the server. Then, the server reflects the updated information transmitted by the user through the geospatial information mapping function to the database of the route guide program through the map generation module, and performs a mapping operation to display the updated information on the electronic map.

For example, when a user wants to enter a specific building displayed on an electronic map of a route guidance program, and there is no passageway or wheelchair lift in the building, the user can send the information to the electric mobility aid When transmitted to the server through the user app, the server updates the route guidance program by reflecting the transmitted information on the electronic map of the route guidance program.

In other words, the user-collected information provision function of the user app for electric mobility aids enables users to directly participate in updating directions or navigation systems for the transportation-disadvantaged or created electronic maps, providing customized information for each type of transportation-disadvantaged person. It provides the ability to share.

The accessibility analysis module for the disabled can identify the dynamic routes of the transportation vulnerable using electric mobility aids and search for accessible areas for the transportation vulnerable through GIS spatial statistical analysis including major gathering places and major access areas.

The accessibility analysis module for the disabled uses the second map information collection module (100b) installed in the electric mobility assist device to receive real-time dynamic movement route information to the server, and analyzes the main assembly points and safe places of the transportation vulnerable in the server to the user terminal. can be sent

In other words, it is possible to explore the accessibility to areas and routes where safety is guaranteed empirically as places and routes frequented by the transportation vulnerable through GIS spatial analysis.

In addition, it can be visualized as the point density of a place where the transportation vulnerable often move or gather. For example, point vectors in areas where a lot of access and movement occur will form a certain cluster, which can be expressed as a raster image using the density algorithm of GIS.

Referring to FIGS. 10 and 11, points represent concentrated user density, and the main algorithm can be defined as first calculating the density of point features around each raster cell, and the neighboring cells of each raster cell are each random. The number of points indicates the density of the area with the center cell, and the point density calculates the point feature density around each output raster cell. Again, the neighboring pixels are defined around the center of each raster cell, and the number of points belonging to the neighboring area is divided by the sum of the number of points belonging to the neighboring area. The number of times to calculate points is determined.

So if the item value is 3, the score is calculated as 3 points, the value can be integer or floating point, and if area units are selected, the computed density for the cell is multiplied by the appropriate factor before writing to the output raster.

For example, when the input ground unit is a meter, the output value is multiplied by 1,000,000 (1,000 * 1,000) when the unit scale factor of the meter is compared to the kilometer.

Depending on the meaning, point density can be used to weight some points more heavily than others, or to have one point represent multiple observations, increasing the radius does not change the computed density value much, and more points are larger. It belongs to the neighborhood, but when calculating the density, this number is divided by the larger area.

The main effect of a larger radius is that the density is computed considering a larger number of points, which can be farther from the raster cell.

In addition, the present invention is based on the map information collected by the first map information collection module 100a and the second map information collection module 100b, and analyzes the object map of bad roads determined in the map generation module and the accessibility analysis module for the disabled. It can be provided with a theme tour course creation module for the transportation disadvantaged that can create a customized map including a map of recommended spots for easy accessibility for the disabled and a tour course map that can be conveniently used using electric mobility aids.

For example, referring to FIG. 12, it is possible to create a themed tour course for each theme by using a road that users of electric mobility aids and people with reduced mobility can conveniently use, and users of electric mobility aids and people with reduced mobility can use the created There is an effect that you can tour conveniently by using the theme tour course.

The above description is merely an example of the technical idea of the present invention, and those skilled in the art can make various modifications, changes, and substitutions without departing from the essential characteristics of the present invention. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings. . The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100 - map information collection device 100a - first map information collection device
100b - second map information collection device 200 - server
210 - Overturn fall judgment module 220 - Map generation module
230 - Road surface condition determination module 240 - Driving pattern attitude determination module
250 - convenience provision module

Claims (10)

A map information collection device capable of collecting road surface data including sidewalks and roads;
A database module that can receive and store the data collected from the map information collection module;
A data purification module capable of performing a process of excluding image information and video information that is not usable for data learning from the data collected by the map information collection module;
A data labeling module that can select an object and label the selected object;
Accessibility analysis module for the disabled that can identify the dynamic route of the transportation vulnerable who use electric wheelchairs and search for accessible areas for the transportation vulnerable through GIS spatial statistical analysis including major gathering places and main access areas
server with
including,
Map information collection device
a first map information collecting device that is carried by a map information collector and is capable of collecting map information data; and
A second map information collection device installed in the motorized mobility aid and capable of collecting map information data;
The first map information collection device
A collection history inquiry unit that can query road surface data collection history, including sidewalks and roads;
Another history inquiry unit that allows you to select a date and search the road surface collection history for that date;
A detailed history inquiry unit that can search shooting history information and a list of collection objects;
A collection object confirmation unit that selects a collection object from the collection object list and inquires information about the collection object;
A history inquiry unit including a;
A preliminary information input unit that can enter advance information including whether indoors or outdoors, filming location, day or night, weather,
An automatic shooting selection unit that can select automatic shooting and manual shooting,
leadership that can indicate the current position;
A photographing screen unit that visualizes and displays a screen being photographed through a camera;
Road surface information collection unit including; and
A date selector where you can select the date for checking the reward;
A total revenue output unit capable of outputting total revenue;
Settlement process information department that can guide the settlement process;
A collection object history output unit capable of outputting collected object history information;
Reward unit including;
to include; and
The second map information collection device
An acceleration sensor for detecting the dynamic posture of the electric mobility aid, including movement of the electric mobility aid, vibration and shock applied to the power mobility aid, and
Gyro sensor for detecting tilt posture of electric mobility aids
A movement detection sensor unit comprising a,
Acquire the location information of the user of the electric mobility aid to measure the driving distance and driving route, and track the location of the fall accident in case a fall accident occurs due to the overturning of the electric mobility aid by the user riding the electric mobility aid A driving position information sensor unit provided to do so, and
A photographing unit provided to photograph the condition of the road surface on which the electric mobility aid is driven.
to include
Automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology including A map information collection device capable of collecting road surface data including sidewalks and roads;
A database module that can receive and store the data collected from the map information collection module;
A data purification module capable of performing a process of excluding image information and video information that is not usable for data learning from the data collected by the map information collection module;
A data labeling module that can select an object and label the selected object;
Accessibility analysis module for the disabled that can identify the dynamic route of the transportation vulnerable who use electric wheelchairs and search for accessible areas for the transportation vulnerable through GIS spatial statistical analysis including major gathering places and main access areas
server with
including,
Map information collection device
a first map information collecting device that is carried by a map information collector and is capable of collecting map information data; and
A second map information collection device installed in the motorized mobility aid and capable of collecting map information data;
The first map information collection device
A collection history inquiry unit that can query road surface data collection history, including sidewalks and roads;
Another history inquiry unit that allows you to select a date and search the road surface collection history for that date;
A detailed history inquiry unit that can search shooting history information and a list of collection objects;
A collection object confirmation unit that selects a collection object from the collection object list and inquires information about the collection object;
A history inquiry unit including a;
A preliminary information input unit that can enter advance information including whether indoors or outdoors, filming location, day or night, weather,
An automatic shooting selection unit that can select automatic shooting and manual shooting,
leadership that can indicate the current position;
A photographing screen unit that visualizes and displays a screen being photographed through a camera;
Road surface information collection unit including; and
A date selector where you can select the date for checking the reward;
A total revenue output unit capable of outputting total revenue;
Settlement process information department that can guide the settlement process;
A collection object history output unit capable of outputting collected object history information;
Reward unit including;
to include; and
The second map information collection device
An acceleration sensor for detecting the dynamic posture of the electric mobility aid, including movement of the electric mobility aid, vibration and shock applied to the power mobility aid, and
Gyro sensor for detecting tilt posture of electric mobility aids
A movement detection sensor unit comprising a,
Acquire the location information of the user of the electric mobility aid to measure the driving distance and driving route, and track the location of the fall accident in case a fall accident occurs due to the overturning of the electric mobility aid by the user riding the electric mobility aid A driving position information sensor unit provided to do so, and
A photographing unit provided to photograph the condition of the road surface on which the electric mobility aid is driven.
to include
Including,
server is
Including a map generation module capable of displaying the optimal driving route of the electric mobility aid using the map information collected by the first map information collection device and the second map information collection device
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. A map information collection device capable of collecting road surface data including sidewalks and roads;
A database module that can receive and store the data collected from the map information collection module;
A data purification module capable of performing a process of excluding image information and video information that is not usable for data learning from the data collected by the map information collection module;
A data labeling module that can select an object and label the selected object;
Accessibility analysis module for the disabled that can identify the dynamic route of the transportation vulnerable who use electric wheelchairs and search for accessible areas for the transportation vulnerable through GIS spatial statistical analysis including major gathering places and main access areas
server with
including,
Map information collection device
a first map information collecting device that is carried by a map information collector and is capable of collecting map information data; and
A second map information collection device installed in the motorized mobility aid and capable of collecting map information data;
The first map information collection device
A collection history inquiry unit that can query road surface data collection history, including sidewalks and roads;
Another history inquiry unit that allows you to select a date and search the road surface collection history for that date;
A detailed history inquiry unit that can search shooting history information and a list of collection objects;
A collection object confirmation unit that selects a collection object from the collection object list and inquires information about the collection object;
A history inquiry unit including a;
A preliminary information input unit that can enter advance information including whether indoors or outdoors, filming location, day or night, weather,
An automatic shooting selection unit that can select automatic shooting and manual shooting,
leadership that can indicate the current position;
A photographing screen unit that visualizes and displays a screen being photographed through a camera;
Road surface information collection unit including; and
A date selector where you can select the date for checking the reward;
A total revenue output unit capable of outputting total revenue;
Settlement process information department that can guide the settlement process;
A collection object history output unit capable of outputting collected object history information;
Reward unit including;
to include; and
The second map information collection device
An acceleration sensor for detecting the dynamic posture of the electric mobility aid, including movement of the electric mobility aid, vibration and shock applied to the power mobility aid, and
Gyro sensor for detecting tilt posture of electric mobility aids
A movement detection sensor unit comprising a,
Acquire the location information of the user of the electric mobility aid to measure the driving distance and driving route, and track the location of the fall accident in case a fall accident occurs due to the overturning of the electric mobility aid by the user riding the electric mobility aid A driving position information sensor unit provided to do so, and
A photographing unit provided to photograph the condition of the road surface on which the electric mobility aid is driven.
to include
Including,
server is
A map generating module capable of displaying an optimal driving route of the electric mobility aid using the map information collected by the first map information collection device and the second map information collection device; and
By collecting information on the posture of the electric mobility aid transmitted from the movement detection sensor unit and the pressure measurement sensor unit included in the second map information collection module, it is possible to determine whether the electric mobility aid has overturned and whether the user has fallen. Overturn fall judgment module
to include
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. A map information collection device capable of collecting road surface data including sidewalks and roads;
A database module that can receive and store the data collected from the map information collection module;
A data purification module capable of performing a process of excluding image information and video information that is not usable for data learning from the data collected by the map information collection module;
A data labeling module that can select an object and label the selected object;
Accessibility analysis module for the disabled that can identify the dynamic route of the transportation vulnerable who use electric wheelchairs and search for accessible areas for the transportation vulnerable through GIS spatial statistical analysis including major gathering places and main access areas
server with
including,
Map information collection device
a first map information collecting device that is carried by a map information collector and is capable of collecting map information data; and
A second map information collection device installed in the motorized mobility aid and capable of collecting map information data;
The first map information collection device
A collection history inquiry unit that can query road surface data collection history, including sidewalks and roads;
Another history inquiry unit that allows you to select a date and search the road surface collection history for that date;
A detailed history inquiry unit that can search shooting history information and a list of collection objects;
A collection object confirmation unit that selects a collection object from the collection object list and inquires information about the collection object;
A history inquiry unit including a;
A preliminary information input unit that can enter advance information including whether indoors or outdoors, filming location, day or night, weather,
An automatic shooting selection unit that can select automatic shooting and manual shooting,
leadership that can indicate the current position;
A photographing screen unit that visualizes and displays a screen being photographed through a camera;
Road surface information collection unit including; and
A date selector where you can select the date for checking the reward;
A total revenue output unit capable of outputting total revenue;
Settlement process information department that can guide the settlement process;
A collection object history output unit capable of outputting collected object history information;
Reward unit including;
to include; and
The second map information collection device
An acceleration sensor for detecting the dynamic posture of the electric mobility aid, including movement of the electric mobility aid, vibration and shock applied to the power mobility aid, and
Gyro sensor for detecting tilt posture of electric mobility aids
A movement detection sensor unit comprising a,
Acquire the location information of the user of the electric mobility aid to measure the driving distance and driving route, and track the location of the fall accident in case a fall accident occurs due to the overturning of the electric mobility aid by the user riding the electric mobility aid A driving position information sensor unit provided to do so, and
A photographing unit provided to photograph the condition of the road surface on which the electric mobility aid is driven.
to include
Including,
server is
A map generating module capable of displaying an optimal driving route of the electric mobility aid using the map information collected by the first map information collection device and the second map information collection device;
By collecting information on the posture of the electric mobility aid transmitted from the movement detection sensor unit and the pressure measurement sensor unit included in the second map information collection module, it is possible to determine whether the electric mobility aid has overturned and whether the user has fallen. overturn fall judgment module; and
The size of the vibration, the duration of the vibration, the vibration pattern, and the slope of the road measured by the movement sensor unit of the second map information collection module and the image signal of the road surface captured by the camera unit are received and stored from the second map information collection module. and a road surface condition determination module capable of updating the electronic map of the road guidance program through the road surface abnormality registration unit of the map generation module by analyzing the condition of the road surface on which the electric mobility aid is traveling;
to include
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. According to claim 4
server is
Receives signals related to the user's seating condition and driving distance from the pressure measurement sensor unit and the driving location information sensor unit of the second map information collection module, analyzes and determines the driving pattern and driving posture of the user using the electric mobility aid Driving pattern posture judgment module
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. According to claim 4
The map creation module
Including a data purification module and a data labeling module,
The data purification module identifies and uses map information transmitted from the first map information collection module and information collected through the movement detection sensor unit, pressure measurement sensor unit, driving location information sensor unit, and camera of the second map information collection module. screening available data; and
The data labeling module can register information by selecting an object and compartmentalizing the selected object.
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. According to claim 6
The map creation module
Further comprising a mobile obstacle collection prediction unit, a registration data review unit, and a data mapping unit,
The mobile obstacle collection prediction unit collects location information of movable obstacles, including bicycle location, vehicle location, pedestrian location, stroller location, and wheelchair location, in real time from the data filtered through the data purification module, and analyzes the collected information. predicting the direction of movement to provide driving notification;
The registration data review unit checks whether the partition of the object for each data frame moves according to the movement of the motorized mobility aid in which the second map information collection module is installed, and checks whether the partitioned object is positioned over a certain frame;
The data mapping unit includes a geospatial information mapping function, updates the geospatial information reviewed by the registration data review unit in the database of the route guidance program, and performs a function of mapping the updated information on the electronic map.
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. According to claim 6
Data labeling module
It further includes a road surface information registration unit, a spatial information registration unit, and a fixed obstacle registration unit,
The road surface information registration unit calculates the flatness of the road surface, the material of the footpath, the condition of the pavement, the type and condition of the mod block, the location of the curb, the location of the drain cover, the location of the braille block, the slope, and the location of the bicycle road from the data refined through the data purification module. Collecting road surface information including, and registering the information by converting it into a sphere;
The geospatial information registration unit collects spatial information, including the status and location of stairs, the location of bottles, the location of windows, the location of pillars, the location of elevators, the shape of doors and traffic directions, from the data filtered through the data purification module, and divides them into information to register; and
The fixed obstacle registration unit collects location information of fixed obstacles including barricade location, bench location, bollard location, stop location, traffic light location, and street tree location from the data filtered through the data purification module, and registers the information by segmenting it. to do;
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. According to claim 4
server is
In the geospatial information mapping function, when the road surface condition is determined to be an abnormal road surface condition through the road surface condition judgment module of the server, the location information of the abnormal road surface area, the distance section of the abnormal road surface, the size of the vibration, and the duration of the vibration are included. Including a road surface anomaly registration unit that can reflect information on the road surface condition and add it to the information mapped by the data mapping unit
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes. According to claim 4
server is
The location information of the user riding the electric mobility aid is received from the driving location information sensor unit of the second map information collection module, and the route the electric mobility aid is traveling and the area to be visited are displayed from the electronic map generated by the map generation module. Including a convenience provision module that can provide information on convenience facilities, including the status of buildings where electric mobility aids can enter and charging locations for electric mobility aids
An automatic map generation system for the transportation vulnerable using deep learning-based object recognition technology that includes.

Images