KR102447643B1 - Automatic map generation system for the disabled using deep learning-based object recognition technology - Google Patents

Abstract

The present invention relates to an automatic map generating system only for the disabled using a deep learning-based object recognition technology, which allows safe and smooth driving, comprising: a learning data generation unit; a learning model generation unit; a geospatial information mapping unit; an electronic map generation unit; a shortest recommended route setting unit; a route guide recommendation information providing unit; and a customized place recommendation information providing unit.

Description

Automatic map generation system for the disabled using deep learning-based object recognition technology}

The present invention relates to an automatic map generation system for the disabled using deep learning-based object recognition technology. In more detail, it collects and analyzes real-time spatial information and road surface information that users of electric mobility aids can pass through to create a learning model that recognizes objects that can help or interfere with driving of electric mobility aids, Based on the created learning model, it enables automatic recognition of objects including road surface information and spatial information that users of electric mobility aids face when they move. It relates to an automatic map generation system for the disabled using deep learning-based object recognition technology that automatically creates

Disabled people who use electric assistive devices account for 6% of the total number of disabled people, and as the aging population accelerates, the use of electric assistive devices is expanding to non-disabled areas.

However, since most of the services provided for movement of electric assistive device users are mainly provided for short-distance travel within an area, it is difficult for users to move long-distance between areas.

On the other hand, the conventional service providing map information consists of the road and the general public, and there is no indication of any obstacles in the movement, so the user's right to move is not guaranteed.

In addition, there is no map service that provides a clear indication of whether or not users of electric assistive devices can use convenient facilities for travel or living and provides a guide for a movable passage.

On the other hand, although development of artificial intelligence technology for road surface information and object recognition is limited, development of a precision map including road surface information of sidewalks and bicycle paths has not been made.

Therefore, it is required to develop map information that recognizes road surface and object information without spatial constraints and is reflected in real-time on the map, which is practically helpful to electric assist device users.

Prior art document: KR Patent Publication No. 10-2020-0064409 (published on June 6, 2020)

The present invention has been devised to solve the above problems, and based on a technology that can automatically recognize factors that threaten driving, etc., it is a map for the disabled that enables safe driving by avoiding factors that threaten driving. The purpose of this is to provide an automatic map generation system for the disabled using deep learning-based object recognition technology that can generate and provide driving safety for motorized mobility aid drivers.

The automatic map generation system for the disabled using the deep learning-based object recognition technology according to the present invention devised to achieve the above object is image information including road surface information and spatial information that affects the movement of the electric mobility assistance device user. a learning data generation unit for collecting, refining and processing, and constructing learning data for recognizing an object related to driving safety when driving an electric mobility aid device; a learning model generation unit for generating a learning model by learning whether the image information labeled with the image information and the object information is a factor that is an obstacle or a factor helpful for movement while driving the electric mobility aid device using an artificial neural network learning method; a geospatial information mapping unit that reflects and updates object information in a database of a navigation program based on the learning model generated by the learning model generation unit, and maps the updated information on an electronic map; and an electronic map generator that implements a function of automatically recognizing an object that may affect the movement of the electric mobility aid on the electronic map based on the learning model generated by the learning model generator.

In addition, based on the learning model generated by the learning model generator, points are given to the space and distance elements facing each route while driving, and the scores are calculated for each route, and the routes are ranked according to the calculated results. It includes a shortest recommended path setting unit listed as

In addition, according to the route selected as the preferred route from the shortest recommended route setting unit, the electric mobility assistance device preferentially selects a drivable road and provides a route guidance recommendation information providing unit to the app of the user driving the electric mobility assistance device. .

According to the present invention, a learning model is created by learning object information encountered while driving an electric mobility aid device by an artificial neural network learning method, and based on the learning model, a threat to driving is reduced by automatically recognizing factors that are helpful or obstructive during driving. It has the effect of enabling safe and smooth driving by avoiding the elements.

1 is a conceptual diagram of a system for automatically generating maps for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention;
2 is a block diagram of a system for automatically generating maps for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention;
3 is a configuration diagram of a learning data generation unit of a system for automatically generating maps for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. First of all, it should be noted that in adding reference numerals to the components of each drawing, the same components are given the same reference numerals as much as possible even though they are indicated on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, preferred embodiments of the present invention will be described below, but the technical spirit of the present invention is not limited thereto or may be variously implemented by those skilled in the art without being limited thereto.

1 is a conceptual diagram of a system for automatically generating maps for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention, and FIG. 2 is a deep learning-based object recognition technology according to a preferred embodiment of the present invention. It is a configuration diagram of an automatic map generation system for the disabled, and FIG. 3 is a configuration diagram of a learning data generator of an automatic map generation system for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention.

1 to 2 , the system 1000 for automatically generating maps for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention includes spatial information and road surfaces that users of electric mobility aids can pass through. By collecting and analyzing information in real time, a learning model is created that recognizes objects that can help or hinder the driving of the electric mobility aid, and based on the created learning model, information on the road surface faced by the user of the electric mobility aid when moving , and to a technology for automatically recognizing objects including spatial information and automatically generating maps for users of electric mobility aids based on such object automatic recognition technology.

Hereinafter, detailed components of the automatic map generation system 1000 for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention will be described in detail.

1 to 2 , the automatic map generation system 1000 for the disabled using deep learning-based object recognition technology according to a preferred embodiment of the present invention includes a learning data generation unit 100 and a learning model generation unit 200 . ), geospatial information mapping unit 300, electronic map generation unit 400, status notification unit 500, shortest recommended route setting unit 600, route guide recommendation information providing unit 700, customized place recommendation information system It is configured to include a study (800).

The learning data generation unit 100 collects data including road surface information and spatial information affecting the movement of the electric mobility aid user, and builds the learning data through refining and processing.

Specifically, referring to FIG. 3 , the learning data generation unit 100 includes an image acquisition unit 110 , a data collection unit 112 , an object collection auxiliary means interworking unit 114 , a data purification unit 116 , and an object area. It is configured to include a selection unit 118 and a data labeling unit 120 .

The image acquisition unit 110 acquires image information including road surface information and spatial information through the camera module of the smart device. In this case, the image information includes both still image and video information obtained by photographing road surface information including sidewalk blocks, drain covers, partial ramps, pavements, and chins, and spatial information including stairs, walls, columns, and elevators.

When shooting for data collection including road surface information and spatial information starts, the image acquisition unit 110 records an image from the moment the shooting start button is pressed to the moment when the end is pressed and recorded as data.

The data collection unit 112 collects image information including road surface information and spatial information obtained from the image acquisition unit 110 .

The object collection auxiliary means interlocking unit 114 links with the image information obtained from the image acquisition unit 110 to match and learn the degree of impact when the electric movement assistance device crosses the chin. By linking the object collection auxiliary means including the measurement sensor and the image acquisition unit 110, the image information about the road surface photographed from the image acquisition unit 110 and the vibration data measured from the vibration measurement sensor are matched with each other to form a learning model to learn in

Among the vibration data sensed by the vibration sensor, the vibration data in which the amplitude changes abruptly or the vibration pattern changes when compared with the vibration data of the normal road surface is collected as abnormal road surface data, and the collected abnormal road surface data is called a chin or dangerous area. By labeling, the data is trained on the learning model.

The data refining unit 116 analyzes the object in the image information captured by the data collection unit 112 to determine whether it is image information usable for object recognition. For example, the data refiner 116 analyzes the image information to determine an image in which an object is distorted or distorted in the image information, an image that is blurry enough to not recognize the object or the characteristic of the object is not recognizable, and the angle of the horizon is predetermined from a standardized angle. The image is extracted from the learning data by extracting an image of an angle twisted or photographed image of only the floor, an image of an object unrelated to the Indian environment, an image of a person's face, and personal information including a license plate.

The object region selection unit 118 receives only image information usable for object recognition from the data refiner 116 and selects a valid object region from the image information. Specifically, the object area selection unit 118 selects the object area by a polygon method for selecting an area along the outer edge of the object, or a bounding box processing method for selecting an area so that the object falls within a rectangular range.

The data labeling unit 116 labels object information corresponding to the area selected by the object area selection unit 118 . Specifically, the data labeling unit 116 is provided with object information including the characteristics, state, and name of the object in the form of a list, and selects any one or more object information from the object information list and sets the object area selection unit 118. The object information corresponding to the object area is labeled.

The object information list includes spatial information including stairs, stair-damage, walls, windows, columns, elevators, traffic direction signs, guide signs, protective walls, and braille signs, as well as flatness, pavement conditions, braille blocks, bicycle paths, and flat crosswalks. , road surface information including protruding curbs and tire sills, and fixed obstacle information including barricades, benches, parking toll machines, pillars, traffic lights, and traffic light controllers, and mobile obstacles including bicycles, mobile information boards, trucks, wheelchairs, and bicycles Information, attention area information including manhole, maintenance area, stairs, and street tree area is stored.

The learning model generating unit 200 generates a learning model by learning the learning data generating unit 100 using an artificial neural network learning method. Specifically, the learning model generation unit 200 uses the information labeled with image information and object information as input data of the learning data, and whether the input data is an obstacle during driving of the electric mobility aid device or a factor that helps movement Using cognition as output data, input data and output data are grouped into one dataset, and a learning model is created by learning using an artificial neural network learning method.

The learning data generation unit 100 subdivides class classification for the core object in order to increase the learning accuracy of the learning model to increase the recognition rate of the core object. For example, a core object is selected from among a plurality of objects, the selected object is classified into a large category and a small category, and image information corresponding to each classification is bundled and stored.

For example, if 'road damaged state' is selected as a major category among objects, it is subdivided into sub-categories, road pits, road subsidence, road cracks, and roads (holes), and image information corresponding to each sub-classification group is searched and classified and stored. By doing so, it is possible to learn various embodiments of the road damage state, thereby increasing the recognition rate of the object.

In addition, even if the objects are classified into the same type of class, when the shapes and shapes of objects are different, they are classified into separate classes so that learning of various shapes can be made. For example, among the image information classified as 'road crack', the longitudinal crack image and the tortoiseshell crack image are each separated and classified into a separate class.

The geospatial information mapping unit 300 reflects and updates the object information in the database of the navigation program based on the learning model generated by the learning model generation unit 200, and maps the updated information on the electronic map.

The electronic map generator 400 implements a function of automatically recognizing an object that may affect the movement of the electric mobility assistance device on the electronic map of the user app.

When specific object information is recognized while driving of the electric mobility assistance device, the state notification unit 500 classifies whether the object information is a helpful element or an obstacle to the movement of the electric mobility assistance device, and provides notification information through the user app. to output. At this time, the status notification unit 500 is output by dividing the color, the notification sound, etc. so as to distinguish between a helpful object and an obstacle object.

The shortest recommended path setting unit 600 gives a score to the space and distance elements facing each other during driving of each path based on the learning model generated by the learning model generating unit 200, and calculates the score given to each path. A function of arranging routes in ranking order according to the calculated results is provided.

In the shortest recommended route setting unit 600, scores are differentially given in the order of the shortest route from the origin to the destination for the distance element, and the space element is added to the elements helpful and obstacles to the driving of the electric mobility aid device, respectively. (+) or minus (-) scores are given. In addition, a plus (+) score is given to convenience factors that exist while the user of the electric mobility aid is driving. Accordingly, scores on distance, space, and convenience factors among a plurality of routes are summed, and the optimal route ranking is selected according to the summed scores.

The route guidance recommendation information providing unit 700 preferentially selects a road in which the electric mobility assisting device can run according to the route selected as the preferred route from the shortest recommended route setting unit 600, and the user's app for driving the electric mobility assisting device provided to Specifically, the route guidance recommendation information providing unit 700 preferentially recommends spatial information for easy movement of the electric mobility assistance device, an area with an even road surface, and a path without an entrance sill.

The customized place recommendation information providing unit 800 is pre-programmed with an algorithm for determining the emotional state according to the bio-signal, and collects the user's location information and bio-signals from the user app that drives the electric mobility aid device, which is pre-programmed. According to the algorithm, the emotional state corresponding to the biosignal is determined, and other users who have the same or similar emotions as the user are searched for and a customized emotional place is recommended. At this time, a customized emotional place is recommended by reflecting the route that the electric mobility assist device is easy to drive.

The above description is merely illustrative of the technical idea of the present invention, and various modifications, changes and substitutions are possible within the scope that does not depart from the essential characteristics of the present invention by those of ordinary skill in the art to which the present invention pertains. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are for explaining, not limiting, the technical spirit of the present invention, and the scope of the technical spirit of the present invention is not limited by these embodiments and the accompanying drawings. . The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: learning data generation unit 200: learning model generation unit
300: geospatial information mapping unit 400: electronic map generation unit
500: status notification unit 600: shortest recommended path setting unit
700: route guide recommendation information providing unit 800: customized place recommendation information providing unit

Claims (3)

An image acquisition unit that acquires image information including sidewalks and roadways using a smart device camera module, a vibration measurement sensor capable of measuring the road surface condition is provided, and the road surface image information obtained from the image acquisition unit and the vibration measured from the vibration sensor An object collection auxiliary means interlocking unit that matches information with each other, a data refining unit that analyzes photographed image information to determine whether object recognition is possible and refines only an object recognizable image, and receives object recognizable image information from the data refining unit and operates a learning data generation unit including an object area selection unit for selecting an effective object area that is disabled or helpful when moving the movement assistance device, and a data labeling unit for labeling object information corresponding to the area selected from the object area selection unit;
A learning model generation unit for generating a learning model by learning whether the image information labeled with the image information and the object information is a factor that becomes a heavy obstacle in driving or a factor that helps in movement using an artificial neural network learning method;
a geospatial information mapping unit that reflects and updates object information in a database of a navigation program based on the learning model generated by the learning model generation unit, and maps the updated information on an electronic map;
an electronic map generator that implements a function of automatically recognizing an object that may affect the movement of an electric mobility aid on an electronic map based on the learning model generated by the learning model generator;
Points are differentially assigned to the distance factor in the order of the shortest route from the origin to the destination, plus or minus scores are given to the driving help factor and obstacle factor of the electric mobility aid device for the spatial factor, and convenience available to users of the motor mobility aid device a shortest recommended route setting unit that selects and proposes an optimal route according to the summed score by adding a plus score to the element, and adding the distance factor, space factor, and convenience factor scores among a plurality of routes;
a route guide recommendation information providing unit that preferentially recommends an easy path for the movement of the electric mobile device, including spatial information for easy movement of the electric mobile device, an area with an even road surface, and a path without an entrance sill;
The algorithm for determining the emotional state according to the bio-signal is pre-programmed, and the user's location information and bio-signals are collected from the user app driving the electric mobility assistance device, and the emotional state corresponding to the bio-signal is determined according to the programmed algorithm. A customized place recommendation information providing unit that recommends a customized emotional place by identifying and searching for other users who have the same and similar emotions as the user
An automatic map generation system for the disabled using deep learning-based object recognition technology, including An image acquisition unit that acquires image information including sidewalks and roadways using a smart device camera module, a vibration measurement sensor capable of measuring the road surface condition is provided, and the road surface image information obtained from the image acquisition unit and the vibration measured from the vibration sensor An object collection auxiliary means interlocking unit that matches information with each other, a data refining unit that analyzes photographed image information to determine whether object recognition is possible and refines only an object recognizable image, and receives object recognizable image information from the data refining unit and operates a learning data generation unit including an object area selection unit for selecting an effective object area that is disabled or helpful when moving the movement assistance device, and a data labeling unit for labeling object information corresponding to the area selected from the object area selection unit;
A learning model generation unit for generating a learning model by learning whether the image information labeled with the image information and the object information is a factor that becomes a heavy obstacle in driving or a factor that helps in movement using an artificial neural network learning method;
a geospatial information mapping unit that reflects and updates object information in a database of a navigation program based on the learning model generated by the learning model generation unit, and maps the updated information on an electronic map;
an electronic map generator that implements a function of automatically recognizing an object that may affect the movement of an electric mobility aid on an electronic map based on the learning model generated by the learning model generator;
Points are differentially assigned to the distance factor in the order of the shortest route from the origin to the arrival, plus or minus points are given to the driving help factor and obstacle factor of the electric mobility aid device for the spatial factor, and a shortest recommended route setting unit that selects and proposes an optimal route according to the summed score by adding a plus score to the convenience factor and summing the distance factor, space factor, and convenience factor scores among a plurality of routes;
a route guide recommendation information providing unit that preferentially recommends an easy path for the movement of the electric mobile device, including spatial information for easy movement of the electric mobile device, an area with an even road surface, and a path without an entrance sill;
The algorithm for determining the emotional state according to the bio-signal is pre-programmed, and the user's location information and bio-signals are collected from the user app driving the electric mobility assistance device, and the emotional state corresponding to the bio-signal is determined according to the programmed algorithm. A customized place recommendation information providing unit that recommends a customized emotional place by identifying and searching for other users who have the same and similar emotions as the user, and reflecting the easy route for driving the electric mobility assistance device;
a state notification unit for outputting notification information through a user app by distinguishing whether the object information is a helpful element or an obstacle to the movement of the electric mobility aid device when specific object information is recognized while driving of the motorized mobility aid device;
An automatic map generation system for the disabled using deep learning-based object recognition technology, including










delete

Images