KR102271321B1 - Artificial intelligence-based electric kickboard for accident prevention and its accident prevention method - Google Patents

Abstract

The present invention relates to an artificial intelligence-based electric kickboard for accident prevention. In accordance with the present invention, the electric kickboard includes: a driving motor reducing speed or braking in accordance with a control signal; a communication part receiving image data made by photographing a driving direction from a camera; an image processing part recognizing a person, a vehicle, a traffic signal and a stop line in the driving direction from the received image data, calculating a distance to the person, a distance to the vehicle and a distance to the stop line based on a result of the recognition, and determining a moving direction of the person, a moving direction of the vehicle and a light-on state of the traffic signal; a weight measurement part measuring the weight of a passenger based on at least one of measured values of a weight sensor installed in a board frame and a torque sensor installed in the driving motor; a calculation part calculating a braking distance based on at least one of the distance calculation result, the moving direction determination result, the light-on state of the traffic signal, the current speed, the current driving direction and the weight of the passenger; and a control part outputting a control signal for the driving motor for speed reduction or braking based on the calculated braking distance. Therefore, the present invention is capable of safely securing a braking distance and preventing a car accident.

Description

Artificial intelligence-based electric kickboard for accident prevention and its accident prevention method {ARTIFICIAL INTELLIGENCE-BASED ELECTRIC KICKBOARD FOR ACCIDENT PREVENTION AND ITS ACCIDENT PREVENTION METHOD}

This specification relates to an artificial intelligence-based electric kickboard for accident prevention.

An electric kickboard is a type of transportation means that moves passengers by receiving power from a charged battery without using a separate fuel and operating a motor. Since it is portable and easy to operate, it is widely used for leisure or commuting. is being used

The electric kickboard is manufactured in the basic form of a kickboard that can be steered by a steering handle, and can be driven by electric power by attaching an electric device to the body or wheel of the kickboard.

However, these electric scooters allow driving speeds of up to 25Km/h, but have a relatively small wheel size and narrow wheel width compared to vehicles or bicycles, so the braking distance is longer, and the occupant rides upright, so the high weight Due to the center of gravity, the possibility of safety accidents during sudden braking is high, which is becoming a social problem.

Korean Patent Publication No. 2016-0081633, 2016.07.08

In the present specification, as the need for a technology for preventing safety accidents of an electric kickboard is increasing, it is intended to present an electric kickboard capable of predicting a collision and securing a braking distance based on artificial intelligence.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The present specification presents an artificial intelligence-based electric kickboard for accident prevention. The electric kickboard has a driving motor that decelerates or brakes according to a control signal, a communication unit that receives image data photographed in a driving direction from a camera, and recognizes people, vehicles, traffic lights and stop lines in the driving direction from the received image data, An image processing unit for calculating the distance to the person, the distance to the vehicle, and the distance to the stop line based on the recognition result, and determining the moving direction of the person, the moving direction of the vehicle, and the lighting state of the traffic light; A weight measuring unit for measuring the weight of the occupant based on at least one of a measurement value of a weight sensor installed on the board frame and a torque sensor installed on the drive motor, the distance calculation result, the moving direction determination result, the lighting state of the traffic light; a calculator configured to calculate a braking distance based on at least one of a current speed, a current driving direction, and the weight of the occupant; and a controller configured to output a control signal to the driving motor for deceleration or braking based on the calculated braking distance; may include

The electric kickboard and other embodiments may include the following features.

According to an embodiment, the image processing unit recognizes the road surface in the driving direction from the received image data, determines the state of the road surface based on the road surface recognition result, and outputs the determination result, and the operation unit recognizes the road surface The braking distance may be calculated further based on the determination result on the state of .

In addition, according to an embodiment, the determination result for the condition of the road surface is determined as at least one of a roadway, a sidewalk, a bicycle road, an accident occurrence road, a road under construction, an unpaved road, and a rain road, and the calculation unit is based on the condition of the road surface The braking distance may be calculated differently according to a result of the determination.

In addition, according to an embodiment, if a collision with the person or the vehicle is expected, or the lighting state of the traffic light is determined as a stop signal or a caution signal, the current speed, the weight of the occupant, and the state of the road surface A braking distance may be calculated based on a result of the determination.

Also, according to an embodiment, the camera may be included in an external device, and the communication unit may receive the image data from the external device.

Also, according to an embodiment, the electric kickboard may further include an alarm unit for generating a braking alarm, and the control unit may output a control signal for the driving motor and simultaneously generate the braking alarm to the occupant through the alarm unit. .

Meanwhile, the present specification provides an artificial intelligence-based accident prevention method performed by an electric kickboard. The artificial intelligence-based accident prevention method includes the steps of receiving image data of a driving direction from a camera, recognizing a person, a vehicle, a traffic light, and a stop line in the driving direction from the received image data, and based on the recognition result calculating the distance to the person, the distance to the vehicle, and the distance to the stop line, and determining the moving direction of the person, the moving direction of the vehicle, and the lighting state of the traffic light, a weight sensor installed in the board frame, and measuring the weight of the occupant based on at least one of the measured values of the torque sensor installed in the driving motor, the distance calculation result, the moving direction determination result, the lighting state of the traffic light, the current speed, the current driving direction, and the occupant calculating a braking distance based on at least one of the weight of , outputting a control signal to the driving motor for deceleration or braking based on the calculated braking distance, and controlling the driving motor according to the control signal to decelerate or brake.

The method and other embodiments may include the following features.

According to an embodiment, the method may include recognizing a road surface in the driving direction from received image data, determining a state of the road surface based on a result of the road surface recognition, and outputting the determination result, and The method may further include calculating the braking distance based on a result of the determination.

In addition, according to an embodiment, the determination result of the condition of the road surface is determined as at least one of a roadway, a sidewalk, a bicycle road, an accident road, a road under construction, an unpaved road, and a rain road, and the determination result on the condition of the road surface Calculating the braking distance further based on ? may include calculating the braking distance differently according to a result of determining the condition of the road surface.

According to an embodiment, the calculating of the braking distance based on the distance calculation result, the moving direction determination result, the lighting state of the traffic light, the current speed, the current driving direction, and the weight of the occupant may include: If a collision with a vehicle is expected or when it is determined that the lighting state of the traffic light is a stop signal or a caution signal, calculating a braking distance based on a result of determining the current speed, the weight of the occupant, and the state of the road surface; may include

The artificial intelligence-based electric kickboard for accident prevention according to the embodiment disclosed in this specification is capable of preventing safety accidents by automatically decelerating or stopping when the risk of collision and signal violation is expected by analyzing the image taken in the direction of travel with artificial intelligence. can have an effect.

In addition, the artificial intelligence-based electric kickboard for preventing accidents according to the embodiment disclosed in the present specification has an effect of safely securing a braking distance by controlling the vehicle to decelerate or stop based on the road surface condition of the driving road and the weight of the occupant.

On the other hand, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be able

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and serve to further understand the technical spirit of the present invention together with specific details for carrying out the invention, so the present invention is described in the drawings It should not be construed as being limited only to matters.
1 shows an electric kickboard for preventing accidents based on artificial intelligence according to an embodiment of the present invention.
2 illustrates an artificial intelligence-based electric kickboard for accident prevention according to an embodiment of the present invention.
3 is a block diagram of an AI device according to an embodiment of the present invention.
4 is an example of a DNN model to which the present invention can be applied.
5 is a flowchart illustrating an example of an artificial intelligence-based accident prevention method performed by an electric kickboard proposed in the present specification.
6 is a schematic diagram illustrating the concept of an artificial intelligence-based accident prevention method performed by the electric kickboard proposed in the present specification.

The technology disclosed herein may be applied to an accident prevention technology of an electric kickboard. However, the technology disclosed in the present specification is not limited thereto, and may be applied to all devices and methods to which the technical idea of the technology may be applied.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments, and are not intended to limit the spirit of the technology disclosed herein. In addition, the technical terms used in this specification should be interpreted in the meaning generally understood by those of ordinary skill in the art to which the technology disclosed in this specification belongs, unless otherwise defined in this specification. It should not be construed in a very comprehensive sense or in an excessively reduced meaning. In addition, when the technical term used in this specification is an erroneous technical term that does not accurately express the spirit of the technology disclosed in this specification, a technical term that can be correctly understood by those of ordinary skill in the art to which the technology disclosed in this specification belongs should be understood and replaced with In addition, general terms used in this specification should be interpreted according to the definition in the dictionary or contextual context, and should not be interpreted in an excessively reduced meaning.

As used herein, terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted.

In addition, in describing the technology disclosed in the present specification, if it is determined that a detailed description of a related known technology may obscure the gist of the technology disclosed in this specification, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easy understanding of the spirit of the technology disclosed in this specification, and should not be construed as limiting the spirit of the technology by the accompanying drawings.

Hereinafter, the structure of an artificial intelligence-based accident prevention electric kickboard according to an embodiment of the present invention will be described with reference to FIG. 1 .

1 to 2 show an electric kickboard for preventing accidents based on artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 1 , the electric kickboard 100 according to an embodiment of the present invention is structurally a board frame 110 , a steering unit 120 , a folding unit 130 , first and second wheels 140 , 150), the first and second motors 160 and 170, and a battery (not shown) may be included.

The board frame 110 may have a structure that allows the user's feet to be placed on its upper surface, and provides a flat footrest thereon.

The steering unit 120 is provided on the front side of the electric kickboard 100 to change the moving direction of the board frame 110 , and the steering unit 120 includes a manipulation unit 123 for controlling the overall operation of the electric kickboard 100 . can be provided. The steering unit 120 is, for example, a handle 121 in the form of a bar for the user's grip, and the handle 121 is coupled to the upper end, and the front side of the board frame 110 by the folding unit 130 . It may include a steering post 122 that is erected to be perpendicular to, or folded to be parallel to the board frame 110 . The operation unit 123 is installed on the handle 121 or the steering post 122, for example, to drive and stop the first and second motors 160 and 170, respectively, to adjust the rotational speed, and to set the driving speed range and brake. For operation, it may include a button, a switch or a lever or the like. Although the brake is not shown here, a device for mechanically or electrically limiting rotation of the first and second wheels 140 and 150 may be applied. Also, when the brake is operated by the manipulation unit 123 , the first and second motors 160 and 170 may be stopped by the control unit 250 to be described later.

The folding part 130 is provided so that the board frame 110 and the steering part 120 are folded to each other so that the occupant (user) can easily carry or carry it. As in this embodiment, one end is perpendicular to the steering post 122 . It may be rotatably coupled to the axis through a bearing or the like, and may have a structure in which the other end is rotatably hinged about an axis in which the other end is horizontal to the board frame 110, and to limit the folding and unfolding state , a locking part (not shown) including a locking piece or a hook member or a fastening piece operated by an elastic force such as a manual operation or a spring may be provided. In addition, since the folding part 130 is not provided according to the electric kickboard 100 , the electric kickboard may not be folded.

The first and second wheels 140 and 150 are the lower end of the steering unit 120 , specifically, the lower end of the steering post 122 (that is, the lower front side of the board frame 110 ) and the lower surface of the board frame 110 . Each is installed so as to be rotatable on the rear side. The first and second motors 160 and 170 are provided to rotate the first and second wheels 140 and 150, respectively, for example, motors or phosphors built into the first and second wheels 140 and 150, respectively. It may consist of an In Wheel Motor.

A battery (not shown) supplies power required for the operation of the first and second motors 160 and 170 , for example, embedded in the board frame 110 , or detachable from the board frame 110 or the steering post 122 , etc. possible to be installed. Meanwhile, a charging circuit unit for charging the battery with an external power source may be provided on the board frame 110 , the manipulation unit 123 , and the like.

1 to 2 , the controller 250 controls each of the first and second motors 160 and 170 according to the manipulation signal of the manipulation unit 123 . Accordingly, when the operation unit 123 controls or sets the driving and stopping, rotational speed adjustment, and traveling speed range of any one of the first and second motors 160 and 170 , the control unit 250 controls the operation of the operation unit 123 . A control signal for controlling the first and second motors 160 and 170 to respectively adjust the driving and rotational speeds of the first and second motors 160 and 170 according to the operation signal, and to maintain the driving speed within a predetermined range can occur.

A display (not shown) may be provided on the handle 121 or the steering post 122 of the steering unit 120 . The display may be controlled by the controller 250 to output information such as a moving distance, a speed, a remaining battery level, and a current location on a map. Here, the control unit 250 measures the moving distance and speed through the rotational speed of the first and second motors 160 and 170 and real-time location tracking by the GPS signal, measures the remaining amount of the battery, and the storage unit 270 . It is possible to calculate the current location on the map by the map data stored in the GPS module (not shown). The control unit 250 controls the rotation speed of the first and second motors 160 and 170 made of, for example, a stepper motor, a servo motor, etc. through a control signal, or, as another example, a first motor comprising a general motor or a BLDC motor. And it is possible to control the rotation speed of the motor through a value measured by a rotation speed measuring unit (not shown) such as an encoder that measures the rotation angle of the second motors 160 and 170 . The electric kickboard 100 according to the present invention may include a motor on both the front wheel and the rear wheel according to an embodiment, or only one of the two wheels may have a motor.

The above-described display may display (output) information processed by the electric kickboard 100 . The display unit includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, 3 It may include at least one of a dimensional display (3D display).

Again, referring to FIGS. 1 and 2 , the artificial intelligence-based electric kickboard for accident prevention according to an embodiment of the present invention includes a communication unit 210 , an image processing unit 220 , a weight measurement unit 230 , a calculation unit 240 , It may include a control unit 250 , an alarm unit 260 , and a storage unit 270 .

The communication unit 210 may receive image data obtained by photographing the driving direction from the camera 201 . The camera 201 may be provided in a form directly connected to the steering unit 120 and may transmit captured image data to the image processing unit 220 through the communication unit 210 . In another embodiment, the communication unit 210 transmits image data captured by a camera provided in the external device from an external device (eg, a smart device having a camera and wireless data transmission function, such as a smartphone owned by a passenger) to the wireless data of the external device. It can be received through the transmitter. The communication unit 210 may receive the image data transmitted from the camera 201 through a wired or wireless transmission means. Also, the communication unit 210 may connect to the passenger's smart device through the provided network and connect to an external network through the smart device.

The networks disclosed herein include, for example, wireless networks, wired networks, public networks such as the Internet, private networks, Global System for Mobile communication network (GSM) networks, and general packet wireless networks (General). Packet Radio Network (GPRN), Local Area Network (LAN), Wide Area Network (WAN), Metropolitan Area Network (MAN), Cellular Network, Public Switched Telephone Network ; PSTN), Personal Area Network, Bluetooth, Wi-Fi Direct, Near Field communication, Ultra-Wide band, a combination thereof, or any may be, but not limited to, other networks of

The image processing unit 220 may recognize a person and a vehicle located in the driving direction of the electric kickboard 100 , a traffic light, and a stop line under the traffic light from the image data received through the communication unit 210 . In addition, the image processing unit 220 determines the distance to the object and the movement direction of the object based on the recognition result of the objects (including people and vehicles, traffic lights, and stop lines) recognized from the image data, or determines the moving direction of the object. status can be determined. For example, if the recognized object is a person, the image processing unit 220 calculates and determines the distance from the electric kickboard 100 to the person and the moving direction of the person, and if the recognized object is a vehicle, from the electric kickboard 100 It is possible to calculate and determine the distance to the vehicle and the moving direction of the vehicle, and if the recognized object is a stop line of a traffic light, calculate the distance from the electric kickboard 100 to the stop line and determine the lighting state of the traffic light. The lighting state of the traffic light determined by the image processing unit 220 may include all indication states displayed by the traffic light, including a driving signal, a caution signal, a first stop signal, a stop signal, and the like.

The weight measuring unit 130 is based on at least one of the measured values of the weight sensor 202 installed on the upper surface of the board frame 110 and the torque sensor 175 installed on the driving motors 160 , 170 , and 171 , the weight of the occupant. can be measured. The occupant's weight includes not only the occupant's weight, but also the weight of their belongings, such as backpacks.

The calculating unit 240 calculates the distance of the image processing unit 220, the moving direction determination result and the lighting state determination result of the traffic light, the electric kickboard 100's own current speed and current driving direction, and the weight of the occupant based on the electric kickboard. If it is determined that (100) should be stopped, a required braking distance can be calculated. That is, when a collision with a person or a vehicle is expected, or when it is determined that the lighting state of the traffic light is a stop signal or a caution signal, the calculation unit 240 determines the current speed of the electric kickboard 100 and the weight of the occupant based on the determination result. The braking distance can be calculated.

Meanwhile, the image processing unit 220 may recognize the road surface in the driving direction of the electric kickboard 100 from the image data received from the communication unit 210 , and then determine the state of the road surface based on the road surface recognition result. The determination result on the condition of the road surface may be determined as a roadway, sidewalk, bicycle road, accident-prone road, road under construction, unpaved road, rain road, etc. In this case, the calculation unit 240 is further added to the determination result on the condition of the road surface. Based on the braking distance can be calculated. That is, the calculating unit 240 has to stop the electric kickboard 100 based on the distance calculation result of the image processing unit 220, the moving direction determination result, the lighting state determination result of the traffic light, and the electric kickboard 100's own current driving direction. If it is determined that the electric kickboard 100 does, the braking distance of the electric kickboard 100 may be calculated based on the result of the determination of the current speed of the electric kickboard 100, the weight of the occupant, and the condition of the road surface.

The calculation unit 240 may calculate different braking distances for each characteristic according to the condition of the road surface according to the determination result of the condition of the road surface. For example, if the result of determining the condition of the road surface is a road or bicycle road, braking is better than on a sidewalk or unpaved road, so the braking distance is calculated shorter than that of a sidewalk or unpaved road. In this case, the braking distance may be calculated differently by reflecting the condition of the road surface, such as calculating the braking distance by 20% longer than that of the case. Also, the calculator 240 may calculate the braking distance in proportion to the weight of the occupant and the size of the current driving speed. Since the weight when the electric kickboard 100 is used varies from time to time depending on the occupant, the electric kickboard 100 according to an embodiment of the present invention calculates a braking distance by reflecting the weight of various occupants and the weight of the occupant's belongings, thereby providing safe braking. It has the effect of deriving the distance. The weight of the occupant may be measured through the weight sensor 202 and/or the torque sensor 175 as described above, but a weight value of the occupant previously input to an external device such as the occupant's smartphone may be used.

The control unit 250 generates a control signal for deceleration or braking based on the braking distance calculated by the operation unit 240 and then inputs it to the driving motor 171 or an electric brake to set the electric kickboard 100 to a safe position. Accidents can be prevented by stopping at

For example, in a state in which the electric kickboard 100 is traveling at a certain speed, the operation unit 240 and the image processing unit 220 analyze the input image data to get closer to the person or vehicle in front, but the speed of the kickboard does not decrease. In this case, a collision is expected. Next, the calculator 240 calculates a braking distance capable of avoiding a collision in consideration of the current speed and the weight of the occupant and outputs it to the controller 250 , and the controller 250 generates a braking distance based on the input braking distance. An accident is prevented by controlling the driving motors 160, 170, and 171 so that the vehicle can be stopped in the vehicle by decelerating or braking.

In another embodiment, the control unit 250 may limit the maximum speed of the electric kickboard 100 according to the determination result of the road surface condition. For example, if it is determined that the road currently being driven is an accident road or a road under construction, the maximum speed may be lowered by 50% compared to the case of driving on a roadway or a bicycle road to limit the speed. In addition, if it is determined that the road currently being driven is a rain road, the maximum speed may be lowered by 30% compared to that of driving on a road or a bicycle road to limit. The maximum speed limit ratio can be determined in advance by experimentation.

The alarm unit 260 may generate an alarm by visual, auditory, or tactile sense during emergency braking to notify the occupant that the emergency braking is required. In this case, the control unit 250 may output a control signal for deceleration or braking to the driving motors 160 , 170 , and 171 , and control to generate a braking alarm to the occupant through the alarm unit 260 . The braking alarm may be generated as a warning sound through the sound output module, as a warning screen through the display panel, or as vibration through a haptic module provided in the handle 121 .

In addition, in addition to the block diagram shown in FIG. 2 , the electric kickboard 100 may further include the block diagram of the AI device of FIG. 3 to perform the method proposed in the present specification. That is, the electric kickboard 100 proposed in the present specification may individually include an AI device including an AI processor, a memory, and the like, which will be described later, or each component.

3 is a block diagram of an AI device according to an embodiment of the present invention.

The AI device 20 may include an electronic device including an AI module capable of performing AI processing, or a server including the AI module. In addition, the AI device 20 is included in the configuration of at least a part of the electronic device provided in the electric kickboard 100 and the electric kickboard 100 shown in FIG. 2 to perform at least a part of AI processing together. have.

The AI processing of the AI device 20 includes all operations related to the control of the electric kickboard 100 and accident prevention of the electric kickboard 100 proposed in the present specification and the accident of the electric kickboard 100 through artificial intelligence learning. It can include all actions for prevention. The AI device 20 may be included as some components of the image processing unit 220 , the calculation unit 240 , and the control unit 250 of FIG. 2 or may replace them. The AI device 20 may be implemented by being functionally embedded in the image processing unit 220 , the calculation unit 240 , and the control unit 250 provided in the artificial intelligence-based accident prevention control module of the electric kickboard 100 .

The AI device 20 may include an AI processor 21 , a memory 25 , and/or a communication unit 27 .

The AI device 20 is a computing device capable of learning a neural network, and may be implemented with various electronic devices such as a server, a desktop PC, a notebook PC, a tablet PC, or the like, or implemented as a single chip. In the present specification, the AI device 20 may be an artificial intelligence-based accident prevention control module of the electric kickboard 100 implemented in any one of the various electronic devices.

The AI processor 21 may learn the neural network using a program stored in the memory 25 . In particular, the AI processor 21 may learn a neural network for recognizing device-related data. Here, the neural network for recognizing device-related data may be designed to simulate a human brain structure on a computer, and may include a plurality of network nodes having weights that simulate neurons of the human neural network. The plurality of network modes may transmit and receive data according to a connection relationship, respectively, so as to simulate a synaptic activity of a neuron in which a neuron sends and receives a signal through a synapse. Here, the neural network may include a deep learning model developed from a neural network model. In a deep learning model, a plurality of network nodes may exchange data according to a convolutional connection relationship while being located in different layers. Examples of neural network models include deep neural networks (DNN), convolutional deep neural networks (CNN), recurrent neural networks (RNN), Restricted Boltzmann Machine (RBM), deep trust It includes various deep learning techniques such as neural networks (DBN, deep belief networks) and deep Q-networks, and can be applied to fields such as computer vision (CV), voice recognition, natural language processing, and voice/signal processing. can

Meanwhile, the processor performing the above-described function may be a general-purpose processor (eg, CPU), but may be an AI-only processor (eg, GPU) for artificial intelligence learning.

The memory 25 may store various programs and data necessary for the operation of the AI device 20 . The memory 25 may be implemented as a non-volatile memory, a volatile memory, a flash-memory, a hard disk drive (HDD), or a solid state drive (SDD). The memory 25 is accessed by the AI processor 21 , and reading/writing/modification/deletion/update of data by the AI processor 21 may be performed. Also, the memory 25 may store a neural network model (eg, the deep learning model 26 ) generated through a learning algorithm for data classification/recognition according to an embodiment of the present invention.

Meanwhile, the AI processor 21 may include a data learning unit 22 that learns a neural network for data classification/recognition. The data learning unit 22 may learn a criterion regarding which training data to use to determine data classification/recognition and how to classify and recognize data using the training data. The data learning unit 22 may learn the deep learning model by acquiring learning data to be used for learning and applying the acquired learning data to the deep learning model.

The data learning unit 22 may be manufactured in the form of at least one hardware chip and mounted on the AI device 20 . For example, the data learning unit 22 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics-only processor (GPU) to the AI device 20 . may be mounted. Also, the data learning unit 22 may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a computer-readable non-transitory computer readable medium. In this case, the at least one software module may be provided by an operating system (Operating System) or may be provided by an application (application program).

The data learning unit 22 may include a training data acquiring unit 23 and a model learning unit 24 .

The training data acquisition unit 23 may acquire training data required for a neural network model for classifying and recognizing data. For example, the training data acquisition unit 23 may acquire, as training data, image data and/or sample data for transmitters on a road infrastructure to be input to the neural network model.

The model learning unit 24 may use the acquired training data to learn so that the neural network model has a criterion for determining how to classify predetermined data. In this case, the model learning unit 24 may train the neural network model through supervised learning using at least a portion of the training data as a criterion for determination. Alternatively, the model learning unit 24 may learn the neural network model through unsupervised learning for discovering a judgment criterion by self-learning using learning data without guidance. Also, the model learning unit 24 may train the neural network model through reinforcement learning using feedback on whether the result of the situation determination according to the learning is correct. Also, the model learning unit 24 may train the neural network model by using a learning algorithm including an error back-propagation method or a gradient decent method.

When the neural network model is trained, the model learning unit 24 may store the learned neural network model in a memory. The model learning unit 24 may store the learned neural network model in the memory of the server connected to the AI device 20 through a wired or wireless network.

The data learning unit 22 further includes a training data preprocessing unit (not shown) and a training data selection unit (not shown) to improve the analysis result of the recognition model or to save resources or time required for generating the recognition model You may.

The learning data preprocessor may preprocess the acquired data so that the acquired data can be used for learning for situation determination. For example, the training data preprocessor may process the acquired data into a preset format so that the model learning unit 24 can use the acquired training data for learning for image data recognition for the transmitter.

In addition, the learning data selection unit may select data required for learning from among the learning data acquired by the learning data acquiring unit 23 or the training data preprocessed by the preprocessing unit. The selected training data may be provided to the model learning unit 24 . For example, the training data selector may select only data included in the specific field as training data by recognizing a specific field among the data sets collected through the network.

In addition, the data learning unit 22 may further include a model evaluation unit (not shown) in order to improve the analysis result of the neural network model.

The model evaluator may input evaluation data to the neural network model and, when an analysis result output from the evaluation data does not satisfy a predetermined criterion, may cause the model learning unit 22 to learn again. In this case, the evaluation data may be predefined data for evaluating the recognition model. As an example, the model evaluation unit may evaluate as not satisfying a predetermined criterion when, among the analysis results of the learned recognition model for the evaluation data, the number or ratio of evaluation data for which the analysis result is not accurate exceeds a preset threshold. .

The communication unit 27 may transmit the AI processing result by the AI processor 21 to an external electronic device. Here, the external electronic device may be defined as the electric kickboard 100 . Also, the AI device 20 may be defined as an external device (smart device) or a 5G network that communicates with the electric kickboard 100 . Meanwhile, the AI device 20 may be implemented by being functionally embedded in an artificial intelligence-based accident prevention control module provided in the electric kickboard 100 . In addition, the 5G network may include a server or module that performs image learning and control for accident prevention. Also, the AI device 20 may be implemented through a home server.

On the other hand, the AI device 20 shown in FIG. 3 has been described as functionally divided into the AI processor 21, the memory 25, the communication unit 27, and the like, but the above-described components are integrated into one module and the AI module Note that it may also be called

The components shown in FIGS. 2 to 3 are not essential, so the electric kickboard 100 for preventing accidents based on artificial intelligence having more or fewer components may be implemented. These components may be implemented as hardware or software, or may be implemented through a combination of hardware and software.

4 is an example of a DNN model to which the present invention can be applied.

A deep neural network (DNN) is an artificial neural network (ANN) composed of several hidden layers between an input layer and an output layer. Deep neural networks can model complex non-linear relationships like general artificial neural networks.

For example, in a deep neural network structure for an object identification model, each object may be expressed as a hierarchical configuration of image basic elements. In this case, the additional layers may aggregate the characteristics of the gradually gathered lower layers. This feature of deep neural networks enables modeling of complex data with fewer units (units, nodes) compared to similarly performed artificial neural networks.

As the number of hidden layers increases, the artificial neural network is called 'deep', and the machine learning paradigm that uses the sufficiently deep artificial neural network as a learning model is called deep learning. In addition, a sufficiently deep artificial neural network used for such deep learning is collectively referred to as a deep neural network (DNN).

In the present invention, data required for learning the POI data generation model may be input to the input layer of the DNN, and meaningful data that a user can use may be generated through the output layer while passing through the hidden layers.

In the specification of the present invention, an artificial neural network used for such a deep learning method is collectively referred to as a DNN, but if meaningful data can be output in a similar manner, of course, other deep learning methods may be applied.

Hereinafter, an example of a You Only Look Once (YOLO) model to which the method proposed in the present specification can be applied will be described.

The YOLO model used in the machine learning field or deep learning field is a technology that detects a semantic region in an image in real time. The semantic region is an image of an object such as a human face, car, tree, glasses, etc. that can be included in an arbitrary image. It's a pattern. The YOLO model is a supervised learning method machine learning algorithm, in which a semantic domain to be recognized and its classification information (ie, labeling) are paired to generate training data, learning model data using thousands of training data as input parameters It consists of a generation step and an object classification step of extracting a learned semantic region from an arbitrary image using the learning model data.

As an example of a field that uses the YOLO model, it consists of a pair of vehicle images and classification information (eg, trucks, buses, passenger cars, etc.) Prepare thousands of training data, run the YOLO model machine learning function using the training data as input parameters to generate model data, and execute the classification function of the YOLO model using the generated model data and arbitrary images as input parameters. The extraction result of the semantic domain can be provided in the form of a bounding box and classification information about the bounding box. The YOLO model can be used in real-time object recognition fields such as self-driving cars and person detection in images because of its high performance, accuracy, and fast execution speed in the field of machine learning image recognition.

In the object classification step of the YOLO model, the target reliability of the initial detection object is specifically set for accurate detection of the semantic region, and the input image is divided into bounding boxes of a certain size. The object detection process calculates the detection candidate object reliability that can be included in the boundary box for all the divided boundary boxes, and extracts the corresponding boundary box when the calculated detection candidate object reliability is greater than or equal to the initial detection target reliability. If the bounding box is not extracted through the object detection process, the size of the bounding box is adjusted and the object detection process is repeatedly executed.

Therefore, the above-described YOLO model is applied to the image processing unit 220 of the present invention and the AI device 20 according to an embodiment of the present invention to recognize people, vehicles, traffic light states, stop lines, etc. for accident prevention. can be applied.

5 is a flowchart illustrating an example of an artificial intelligence-based accident prevention method performed by an electric kickboard proposed in the present specification.

2 and 5 , the artificial intelligence-based accident prevention method performed by the electric kickboard may include the following steps.

First, the communication unit 210 of the electric kickboard 100 receives the image data of the driving direction from the camera 201 (S510).

Next, the image processing unit 220 processes the image data received from the communication unit 210 to recognize people, vehicles, and traffic lights located in the driving direction of the electric kickboard 100 and a stop line located under the traffic lights, , a distance to a person, a distance to a vehicle, and a distance to a stop line may be calculated based on the recognition result, and the moving direction of the person, the moving direction of the vehicle, and the lighting state of the traffic light may be determined ( S520 ).

The weight measurement unit 230 measures the weight of the occupant based on at least one of the measured values of the weight sensor 202 installed on the board frame 110 and the torque sensor 175 installed on the drive motors 160 , 170 , and 171 . do (S530).

Next, the calculating unit 240 calculates a braking distance based on the distance calculation result output from the image processing unit 220, the moving direction determination result, the lighting state of the traffic light, the current speed, the current driving direction, and the weight of the occupant (S540) ).

Next, the control unit 250 may output a control signal to the driving motors 160 , 170 , and 171 for deceleration or braking based on the braking distance calculated by the operation unit 240 ( S550 ).

Finally, the control unit 250 controls the driving motors 160, 170, and 171 according to the control signal to decelerate or brake, thereby causing the electric kickboard 100 to collide with a person or vehicle in front and an accident due to signal violation. can be prevented (S560).

In addition, the artificial intelligence-based accident prevention method performed by the electric kickboard recognizes the road surface in the driving direction from the image data received by the image processing unit 220, and determines the state of the road surface based on the road surface recognition result. The method may further include outputting a determination result, and calculating, by the calculator 240 , a braking distance further based on a determination result of the state of the road surface in the driving direction.

On the other hand, the determination result for the condition of the road surface is determined as at least one of a roadway, a sidewalk, a bicycle road, an accident road, a road under construction, an unpaved road, and a rain road, and at this time, based on the determination result on the condition of the road surface Thus, calculating the braking distance may include calculating the braking distance differently according to a result of determining the condition of the road surface.

In the method, calculating the braking distance based on the distance calculation result, the moving direction determination result, the lighting state of the traffic light, the current speed, the current driving direction, and the weight of the occupant may include: If a collision is expected or it is determined that the lighting state of the traffic light is a stop signal or a caution signal, calculating a braking distance based on a result of determining the current speed, the weight of the occupant, and the state of the road surface can

In the above description, operations S510 to S560 may be further divided into additional operations or combined into fewer operations according to an embodiment of the present invention. In addition, some operations may be omitted if necessary, and the order between the operations may be switched.

6 is a schematic diagram illustrating the concept of an artificial intelligence-based accident prevention method performed by the electric kickboard proposed in the present specification, and FIG. 6 (a) is a driving direction obtained by the electric kickboard 100 from the camera 201. When the electric kickboard 100 is predicted to collide with the person P located in front from the image data for In the case of outputting a motor control signal for stopping, FIG. 6(b) shows that when the traffic light is a stop signal from the image data for the driving direction acquired by the camera 201 of the electric kickboard 100, the electric kickboard 100 moves forward. When it is predicted that the located stop line L will be violated, the minimum braking distance D2 that can prevent the stop line violation is calculated and the motor control signal is output to stop before the braking distance D2 is reached. do.

As used herein, the term “unit” (eg, a control unit, etc.) may mean a unit including, for example, one or a combination of two or more of hardware, software, or firmware. The term “unit” may be used interchangeably with terms such as, for example, unit, logic, logical block, component, or circuit. A "part" may be a minimum unit of a component integrally formed or a part thereof. A “unit” may be a minimum unit or a part thereof that performs one or more functions. “Part” may be implemented mechanically or electronically. For example, a “unit” may be one of Application-Specific Integrated Circuit (ASIC) chips, Field-Programmable Gate Arrays (FPGAs), or programmable-logic devices, known or to be developed, that performs certain operations. It may include at least one.

At least a portion of an apparatus (eg, modules or functions thereof) or a method (eg, operations) according to various embodiments is, for example, a computer-readable storage medium in the form of a program module It can be implemented as a command stored in . When the instruction is executed by a processor, the one or more processors may perform a function corresponding to the instruction. The computer-readable storage medium may be, for example, a memory.

Computer-readable storage media/computer-readable recording media include hard disks, floppy disks, magnetic media (eg, magnetic tape), and optical media (eg, compact CD-ROMs). disc read only memory), digital versatile disc (DVD), magneto-optical media (e.g., floptical disk), hardware devices (e.g., read only memory (ROM), random (RAM) access memory), or flash memory, etc.) In addition, program instructions may include high-level language code that can be executed by a computer using an interpreter as well as machine code such as generated by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the various embodiments, and vice versa.

A module or program module according to various embodiments may include at least one or more of the above-described components, some may be omitted, or may further include additional other components. Operations performed by modules, program modules, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or in a heuristic manner. Also, some operations may be executed in a different order, omitted, or other operations may be added.

As used herein, the term “a” is defined as one or more than one. Also, use of introductory phrases such as “at least one” and “one or more” in a claim includes introductory phrases such as “at least one” and “one or more” and an obscure phrase such as “a” in the same claim. The introduction of another claim element by the ambiguous phrase "a", even if there is any, shall be construed to mean limiting any particular claim containing the claim element so introduced, to an invention containing only one such element. shouldn't be

Unless otherwise indicated, terms such as “first” and “second” are used to arbitrarily distinguish between the elements described by such terms. Accordingly, these terms are not necessarily intended to indicate a temporal or other priority of such elements, and the mere fact that certain measures are recited in different claims does not indicate that a combination of these measures cannot be used to advantage. . Accordingly, these terms are not necessarily intended to indicate the temporal or other priorities of such elements. The mere fact that certain measures are cited in different claims does not indicate that a combination of these measures cannot be useful.

Also, in the description and claims, “front”, “behind”, “before”, “after”, “top”, “top”, “bottom”, “bottom”, “above”, “below”, etc. The term is used for descriptive purposes, but is not necessarily used to describe a permanent relative position. It is understood that the terms so used are interchangeable under appropriate circumstances to enable the embodiments of the invention described herein to operate otherwise than, for example, as shown or otherwise described herein.

For simplicity and clarity of illustration, it will be understood that elements (elements) shown in the drawings are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.

The arrangement of components to achieve the same function is effectively “related” such that the desired function is achieved. Thus, any two components combined to achieve a particular functionality may be considered "related" to each other so that the desired function is achieved, regardless of structure or intervening component. Likewise, two components so associated may be considered “operably connected” or “operably coupled” to each other to achieve a desired function.

Also, those skilled in the art will recognize that the boundaries between the functionality of the operations described above are illustrative only. A plurality of operations may be combined into a single operation, the single operation may be distributed to additional operations, and the operations may be executed at least partially overlapping in time. Also, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments. However, other modifications, variations and alternatives are also possible. Accordingly, the detailed description and drawings are to be regarded in an illustrative rather than a restrictive sense.

The phrase “may be X” indicates that condition X may be satisfied. This phrase also indicates that condition X may not be met. For example, a reference to a system that includes a particular component should also include a scenario in which the system does not contain the specific component. For example, a reference to a method that includes a particular action should also include a scenario in which the method does not include the particular component. As another example, however, references to a system configured to perform a specific action should also include scenarios in which the system is not configured to perform a specific action.

The terms “comprising,” “having,” “consisting of,” “consisting of, and “consisting essentially of are used interchangeably. For example, any method may include at least the acts contained in the drawings and/or the specification, and may include only the acts contained in the drawings and/or the specification.

Those of ordinary skill in the art will recognize that the boundaries between logical blocks are exemplary only, and that alternative embodiments may incorporate logical blocks or circuit elements or impose an alternative decomposition of functionality on the various logical blocks or circuit elements. will recognize that Accordingly, it should be understood that the architecture shown herein is exemplary only, and in fact, many other architectures may be implemented that achieve the same functionality.

Also, for example, in one embodiment, the illustrated examples may be implemented as circuitry located on a single integrated circuit or within the same device. Alternatively, the above examples may be implemented as any number of individual integrated circuits or individual devices interconnected to each other in a suitable manner, and other variations, modifications, variations and alternatives are also possible. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Further, for example, the examples or portions thereof may be implemented as a software or code representation of a physical circuit or a logical representation convertible into a physical circuit, such as any suitable type of hardware description language.

Further, although the present invention is not limited to physical devices or units implemented in non-programmable hardware, mainframes, minicomputers, servers, workstations, personal computers, notepads, generally referred to herein as 'computer systems' , personal digital assistants (PDAs), electronic games, automobiles and other embedded systems, mobile phones, and various other wireless devices, such as programmable devices capable of performing desired device functions by operating in accordance with appropriate program code. Or it can be applied to units as well.

A system, apparatus, or device referred to in this specification includes at least one hardware component.

Connections as described herein may be any type of connection suitable for transmitting a signal to or from each node, unit or apparatus, for example via an intermediate apparatus. Thus, unless implied or otherwise stated, a connection may be, for example, a direct connection or an indirect connection. A connection may be described or depicted with reference to being a single connection, multiple connections, one-way connection, or two-way connection. However, different embodiments may change the implementation of the connection. For example, you can use a separate one-way connection rather than a two-way connection, and vice versa. In addition, a plurality of connections may be replaced with a single connection that transmits a plurality of signals sequentially or in a time multiplexing manner. Likewise, a single connection carrying multiple signals may be split into various connections carrying subsets of these signals. Therefore, many options exist for transmitting the signal.

Those of ordinary skill in the art will recognize that the boundaries between logical blocks are exemplary only, and that alternative embodiments may incorporate logical blocks or circuit elements or impose an alternative decomposition of functionality on the various logical blocks or circuit elements. will recognize that Accordingly, it should be understood that the architecture shown herein is exemplary only, and in fact, many other architectures may be implemented that achieve the same functionality.

In the above, preferred embodiments of the technology of the present specification have been described with reference to the accompanying drawings. Here, the terms or words used in the present specification and claims should not be construed as being limited to conventional or dictionary meanings, but should be interpreted as meanings and concepts consistent with the technical spirit of the present invention. The scope of the present invention is not limited to the embodiments disclosed herein, and the present invention can be modified, changed, or improved in various forms within the scope of the spirit and claims of the present invention.

100: electric kick board 110: board frame
120: steering unit 121: handle
122: steering post 123: control unit
130: folded portion 140: first wheel
150: second wheel 160: first motor
170: second motor 171: drive motor
175: torque sensor 201: camera
202: weight sensor 210: communication unit
220: image processing unit 230: weight measurement unit
240: calculation unit 250: control unit
260: alarm unit 270: storage unit

Claims (10)

a drive motor that decelerates or brakes according to a control signal;
a communication unit for receiving image data obtained by photographing a driving direction from a camera;
Recognizing a person, a vehicle, a traffic light, and a stop line in the driving direction from the received image data, and calculating a distance to the person, a distance to the vehicle, and a distance to the stop line based on the recognition result, and the person an image processing unit for determining a moving direction of the vehicle, a moving direction of the vehicle, and a lighting state of the traffic light;
a weight measuring unit configured to measure the weight of the occupant based on at least one of a measured value of a weight sensor installed on the board frame and a torque sensor installed on the driving motor;
a calculator configured to calculate a braking distance based on at least one of the distance calculation result, the moving direction determination result, a lighting state of the traffic light, a current speed, a current driving direction, and a weight of the occupant; and
A control unit for outputting a control signal to the driving motor for deceleration or braking based on the calculated braking distance
An electric kickboard for preventing accidents based on artificial intelligence comprising a. According to claim 1,
The image processing unit
Recognizing the road surface in the driving direction from the received image data, determining the state of the road surface based on the road surface recognition result, and outputting the determination result,
the calculation unit
calculating the braking distance further based on a result of the determination on the condition of the road surface
Electric kickboard for accident prevention based on artificial intelligence. 3. The method of claim 2,
The result of judgment on the condition of the road surface is
is determined as at least one of a roadway, sidewalk, bicycle path, accident road, road under construction, dirt road, and rain road;
the calculation unit
Calculating the braking distance differently according to the determination result on the condition of the road surface
Electric kickboard for accident prevention based on artificial intelligence. The method of claim 3, wherein the calculating unit
When a collision with the person or the vehicle is expected, or when the lighting state of the traffic light is determined as a stop signal or a caution signal, the braking distance is determined based on the determination result of the current speed, the weight of the occupant, and the condition of the road surface. to calculate
Electric kickboard for accident prevention based on artificial intelligence. According to claim 1,
The camera is included in an external device,
The communication unit receives the image data from the external device
Electric kickboard for accident prevention based on artificial intelligence. According to claim 1,
Further comprising an alarm unit for generating a braking alarm,
The control unit outputs a control signal for the driving motor and at the same time generates the braking alarm to the occupant through the alarm unit.
Electric kickboard for accident prevention based on artificial intelligence. In an artificial intelligence-based accident prevention method performed by an electric kickboard,
receiving image data obtained by photographing the driving direction from a camera;
Recognizing a person, a vehicle, a traffic light, and a stop line in the driving direction from the received image data, and calculating a distance to the person, a distance to the vehicle, and a distance to the stop line based on the recognition result, and the person determining a moving direction of the vehicle, a moving direction of the vehicle, and a lighting state of the traffic light;
measuring the weight of the occupant based on at least one of a weight sensor installed in the board frame and a torque sensor installed in a driving motor that decelerates or brakes according to a control signal;
calculating a braking distance based on at least one of the distance calculation result, the moving direction determination result, a lighting state of the traffic light, a current speed, a current driving direction, and a weight of the occupant;
outputting a control signal to the driving motor for deceleration or braking based on the calculated braking distance; and
and controlling the driving motor to decelerate or brake according to the control signal. 8. The method of claim 7,
recognizing a road surface in the driving direction from the received image data, determining a state of the road surface based on the road surface recognition result, and outputting the determination result; and
Accident prevention method further comprising the step of calculating the braking distance based on a result of the determination on the condition of the road surface. 9. The method of claim 8,
The result of judgment on the condition of the road surface is
is determined as at least one of a roadway, sidewalk, bicycle path, accident road, road under construction, dirt road, and rain road;
Calculating the braking distance further based on a result of the determination on the condition of the road surface includes:
and calculating the braking distance differently according to a result of determining the condition of the road surface. 10. The method of claim 9,
Calculating the braking distance based on the distance calculation result, the moving direction determination result, the lighting state of the traffic light, the current speed, the current driving direction, and the weight of the occupant includes:
When a collision with the person or the vehicle is expected, or when the lighting state of the traffic light is determined as a stop signal or a caution signal, the braking distance is determined based on the determination result of the current speed, the weight of the occupant, and the condition of the road surface. An accident prevention method comprising the step of calculating.

Images