KR20180067372A - Smart vehicle control system using field programmable gate array and raspberry pi - Google Patents

Abstract

Provided are a smart vehicle control system using a field programmable gate array (FPGA) and a Raspberry Pi, capable of recognizing a traffic situation in real time while preventing an accident caused by drowsy driving. The smart vehicle control system includes: an infrared sensor, an ultrasonic sensor, and a camera installed at a front portion of a vehicle; an FPGA module connected to the infrared sensor and the ultrasonic sensor for receiving sensing values from the infrared sensor and the ultrasonic sensor to control the driving of the vehicle; and a Raspberry Pi module connected to the FPGA module to receive each sensing value, in which the Raspberry Pi module analyzes an image photographed by the camera and outputs a drive control signal for the vehicle to the FPGA module based on the analysis results of the sensing values and the images.

Description

TECHNICAL FIELD [0001] The present invention relates to an intelligent automobile control system using a field programmable gate array and a raspberry pie,

The present invention relates to an intelligent automotive control system using a Field Programmable Gate Array (FPGA) and a Raspberry Pi.

In modern society, as the computer and sensor technologies are developed, automobile industries such as autonomous vehicles and electric vehicles are expanding in various fields.

If you sleep for a moment while driving, it will lead to a major accident. In addition, a person who has just started driving is more likely to make a mistake while driving.

Many people who are unfamiliar with driving are afraid of accidents and their families worry about inexperienced drivers. Workers who leave the workplace late at night often go to sleepy driving. Even if you pause for a while when you are drowsy, you may get drowsy momentarily, which leads to a major accident. Many drivers do not follow the speed limit to reach their destination quickly. For the same reason, there are also many drivers who run at high speed and suddenly stop, which not only increases the risk of accidents but also wastes energy due to unnecessary deceleration and acceleration.

In this situation, there is a need for automobile control technology that can run safely and use energy efficiently.

A problem to be solved by the present invention is to provide a driver with safer road driving environment, to grasp road conditions in real time, to reduce accidents due to insufficient driving, sleeping driving, etc., (FPGA) and Raspberry Pi, which are capable of realizing an intelligent automobile control system capable of realizing an intelligent automobile control system.

According to an aspect of the present invention, an intelligent automobile control system includes an infrared sensor, an ultrasonic sensor, and a camera, respectively, mounted on a front portion of an automobile, the infrared sensor, and the ultrasound sensor, A field programmable gate array (FPGA) module for receiving a sensing value, controlling a vehicle, and a field programmable gate array (FPGA) module for receiving the respective sensing values, Determining a running direction and a running direction of the automobile based on a result of analyzing each of the sensed values and a result of analyzing the sensed value, and transmitting a vehicle control signal according to the determination to the field programmable gate array Raspberry Pi modules that output to the FPGA module .

In addition, when the intelligent automobile control system senses rainfall, a rainfall sensor that outputs a rainfall occurrence signal, a servo motor that drives a wiper mounted on a windshield of an automobile, and a web cam mounted on the front of the automobile Including,

The Raspberry Pi module,

A traffic light sensing unit for analyzing the image and outputting the stop signal if the recognized traffic light color is red and outputting a traveling signal if the recognized traffic light color is green, A rain detection unit for outputting a signal for driving the servo motor when rainfall is detected from the rainfall sensor, and a video signal captured by the web cam through a WiFi communication, To a monitoring control unit.

The field programmable gate array (FPGA)

A lane recognition unit for outputting a signal for controlling the direction of the vehicle according to a lane recognized based on a sensed value received via the infrared sensor, and a lane recognition unit for stopping the vehicle when an obstacle is detected based on the sensed value received through the ultrasonic sensor An obstacle detection unit for outputting a stop signal or a deceleration signal, and a vehicle battery remaining amount measurement unit for measuring the remaining amount of the battery and displaying the remaining amount of the battery on the outside by varying the number of LEDs to which power is supplied in each step according to the remaining amount of the battery.

The field programmable gate array (FPGA)

A pulse width modulation (PWM) signal is generated by using a built-in clock, and a delay time is measured until the measurement cycle of the ultrasonic sensor, the ultrasonic wave generation time, and the ultrasonic wave reflection time are measured using the PWM signal, can do.

The field programmable gate array (FPGA)

A relay circuit may be used to receive the automotive control signal from the Raspberry Pi module.

According to the embodiment of the present invention, sensors and a camera are attached to a field programmable gate array (FPGA) board and a raspberry pie, so that the road situation can be grasped in real time and accidents caused by insufficient driving and drowsy driving can be reduced.

In addition, by allowing the driver's family to see the driving images in real time, it is possible to reduce the anxiety, promptly respond to an emergency situation, and eliminate the unnecessary driving, thus enabling more efficient use of the energy of the vehicle.

In addition, since the lane departure is avoided and obstacles are detected and operated accordingly, the risk of accidents due to the concentration of the user or an inexperienced driving skill is reduced.

In addition to vehicles, it is possible to reduce the risk of accidents by installing sensors on many things such as motorcycles, electric boards, etc., and to prevent the risk of collision by avoiding obstacles such as birds by attaching sensors to the helicopter forward have.

In addition, it can be applied to delivery drone. The sensor can be installed in the front of the drone to prevent obstacle collision, and it is possible to remotely view the flight image, so that the user can quickly cope with the change such as sudden weather change.

1 is a configuration diagram of an intelligent automobile control system according to an embodiment of the present invention.
Fig. 2 shows a detailed configuration of the raspberry pie module of Fig.
3 is a flowchart illustrating an intelligent automobile control method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. Also, the terms " part, "" module," and " module ", etc. in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software have.

Now, a configuration of an intelligent automobile control system using a Field Programmable Gate Array (FPGA) and a Raspberry Pi according to an embodiment of the present invention will be described with reference to the drawings.

In the present specification, automobiles are not limited to automobiles having four wheels but are used as a concept collectively referred to as means for moving the motors, such as motorcycles, electric boards, drums, etc.,

FIG. 1 is a configuration diagram of an intelligent automobile control system according to an embodiment of the present invention, FIG. 2 is a detailed configuration of the Raspberry module of FIG. 1, and FIG. 3 is a flowchart illustrating an intelligent automobile control method according to an embodiment of the present invention It is a flowchart.

1, the intelligent automobile control system includes an infrared sensor 101, an ultrasonic sensor 103, a rainfall sensor 105, a servo motor 107, a battery remaining amount display unit 109, a camera 111, A webcam 113, a smartphone app 115, a field programmable gate array (FPGA) module 117, and a raspberry pie module 119.

The infrared sensor 101 is mounted on the front portion of the vehicle. The infrared sensor 101 emits infrared rays from the light emitting unit and detects the reflection of the diverged infrared light on the object to detect the presence or absence of the object. Since the degree of reflection of light depends on the color and shape of the object to which the infrared rays are reflected, the object can be detected and distinguished.

The infrared sensor 101 outputs the sensor value? 1? To the field programmable gate array (FPGA) module 117 when the black object is recognized and the sensor value 0 when the object other than the black object is recognized. To the field programmable gate array (FPGA) module 117.

The ultrasonic sensor 103 is mounted on the front portion of the automobile. The ultrasonic sensor 103 receives the ultrasonic generation time, the measurement period, and the delay time from the field programmable gate array (FPGA) module 117. Ultrasonic waves are generated according to the ultrasonic wave generation time and measurement cycle. After delaying by the delay time, the time when the ultrasonic wave returns is measured, that is, the reflection time. The distance from the ultrasonic sensor to the object is measured based on the measured reflection time, and the obstacle is detected using the distance. If the delay time is not given, the reflected ultrasonic waves are recognized as overlapping, Can cause. Divide the measured time by 58 to get the distance value.

The ultrasonic sensor 103 detects whether an obstacle is detected according to the measured distance value, and outputs a sensing signal or sensing value to the field programmable gate array (FPGA) module 117 to notify whether an obstacle is detected.

The rainfall sensor 105 is installed at a specific position of the automobile and outputs a sensing signal or sensing value to the raspberry module 119 when it detects the occurrence of rainfall.

Servo motor 1 107 is connected to a field programmable gate array (FPGA) module 117 and thermo motor 2 121 is connected to a raspberry module 119. In the embodiment of the present invention, Lt; / RTI >

Servo motor 1 (107) is used for switching the direction of the automobile. The servo motor 1 107 switches the direction of the vehicle according to the PWM signal received from the field programmable gate array (FPGA) module 117.

Servo motor 1 (107) used for direction control is driven according to whether or not the rated voltage is supplied, and the speed control is controlled by the PWM signal. In the case of Servomotor 1 (107), it is driven at a constant rated voltage of 5 V, and the direction switching determines how much to rotate according to the PWM signal.

The thermo motor 2 (121) is mounted in connection with a wiper mounted on the front of the vehicle. The servomotor 107 controls the rotation of the wiper according to the PWM signal received from the raspberry module 119.

The battery remaining amount display unit 109 is mounted at a position that can be checked by the driver, and may be implemented by at least one LED or may be a display panel.

The remaining battery level display unit 109 changes the number of LEDs to which power is supplied according to the remaining battery level. For example, if the remaining amount is 100%, all 10 LEDs are turned on. If the remaining amount is 50%, only 5 LEDs are turned on. LED allows the operator to visually check the remaining battery level.

The camera 111 is mounted on the front portion of the vehicle, and outputs the photographed image to the raspberry module 119. The camera 111 recognizes the color of the road sign and the traffic light to help secure the operation.

The web cam 113 is mounted on the front side of the automobile and photographs the front side image in real time and outputs the photographed image to the raspberry module 119.

The smartphone application 115 is installed in a smartphone of a user and is executed and receives a video signal photographed by the web cam 113 through WiFi communication and streams it. Accordingly, the driver's family can check the traveling image in real time, and the image photographed through the web cam 113 can be stored in the smartphone application 115, thereby playing the role of a black box.

The smartphone application 115 is an application for photographing and outputting the driving situation using the web cam 113, and can be implemented on the basis of Android. At this time, you can use Mjpg-Streamer to create an image as a jpeg file, and link the JavaScript to the Android web view.

The smartphone application 115 can display the traveling streaming image received through the web cam 113 while displaying the remaining battery level at the lower end. The START or STOP button also transmits the stop or travel signal to the field programmable gate array (FPGA) module via the Raspberry pie module.

The field programmable gate array (FPGA) module 117 includes an infrared sensor 101, an ultrasonic sensor 103, a servo motor 1 107, a remaining battery level indicator 109, a raspberry module 119 and a DC motor 123, Lt; / RTI >

The field programmable gate array (FPGA) module 117 is an intermediate development type integrated circuit that is finally manufactured to verify the operation and performance of the hardware just before producing the designed hardware into the semiconductor.

The field programmable gate array (FPGA) module 117 controls the movement of the vehicle according to the vehicle control signal received from the raspberry module 119. In addition, a basic automobile motion control using an ultrasonic sensor, an infrared sensor, and the like is controlled through a sensor value in the FPGA module 117 itself, and the Raspberry module 119 transmits a signal lamp and a sign signal recognized through the camera 111 to the FPGA Module 117 to control the movement according to the signal. In other words, the FPGA module 117 can control the motion of the vehicle through not only the signal transmitted from the Raspberry module 119 but also the sensor signal recognized by itself.

The field programmable gate array (FPGA) module 117 may generate a PWM signal using an internal clock. When a voltage of 12 V was supplied, a voltage was supplied for 4/1 hours per cycle If the voltage is not supplied for the remaining 4/3 hours, it has the same effect as supplying the voltage of 3V. The amount of energy output in this manner can be controlled.

The field programmable gate array (FPGA) module 117 determines the delay time until the measurement cycle of the ultrasonic sensor 103, the ultrasonic generation time and the ultrasonic reflection time are measured using the PWM signal, and outputs the determined delay time to the ultrasonic sensor 103 .

For example, the field programmable gate array (FPGA) module 117 generates a PWM signal having a cycle of 20 ms and a duty ratio of 50%. It can be seen that the clock of the field programmable gate array (FPGA) module 117 is 50 MHz, and the clock is generated 50 times 10 6 times in 1 second, and the clock is generated 1 time in 0.02 us. Since a million clocks are generated during a period of 20ms, a signal of '1' is applied until the clock reaches 500,000 times, and a signal of '0' is output until a time of one million times from then on to generate a PWM signal having a duty ratio of 50% can do. When the signal is '1', the voltage is supplied. When the signal is '0', the voltage can be cut off. After the clock reaches 1 million times, it can be initialized and repeated from the beginning.

The field programmable gate array (FPGA) module 117 may receive a vehicle control signal from a Raspberry Pi module 119 using a relay circuit. Here, the relay circuit inputs the signal received from the raspberry module 119 to the field programmable gate array (FPGA) module 117 in an appropriate voltage swap.

The raspberry module 119 is connected to the field programmable gate array (FPGA) module 117 and is connected to the smartphone application 115 via Wi-Fi communication.

The DC motor 123 amplifies the PWM signal received from the field programmable gate array (FPGA) module 117 with a transistor. When the amplified signal drives the relay switch, a voltage is applied and the DC motor 123 is driven. At this time, the diode connected to the DC motor 123 prevents reverse voltage.

Here, the transistor is a device that amplifies and controls a current or a current of a voltage, and the application is different depending on how the three pins (B, C, and E) are connected. When the relay switch is driven using the amplified signal, the DC motor 123 applies the connected motor driving voltage. Here, the relay switch operates when the input reaches a certain value and opens / closes another circuit. When the value of the PWM signal is '1', the relay switch is driven by the amplified signal, and when '0', the relay switch is not driven. When the relay switch is driven, the speed of the motor can be controlled because the voltage required for driving the motor in the connected circuit is supplied. Thus, the motor speed and direction of the DC motor 123 are controlled through the PWM signal.

2, the field programmable gate array (FPGA) module 117 includes a lane recognition unit 117a, an obstacle detection unit 117b, and a vehicle battery remaining amount measurement unit 117c.

The lane recognizing unit 117a outputs a signal for controlling the direction of the vehicle according to the lane recognized based on the sensed value received via the infrared sensor 101. [ The lane recognizing section 117a recognizes the black lane through the infrared sensor 101 mounted on the automatic front both ends, thereby preventing the lane departure and automatically returning to the original lane even if the lane departure occurs. As described above, the infrared sensor 101 recognizes the lane, and then autonomously rides the road. The lane departure of the vehicle can be prevented, and the vehicle can travel at a straight course and at a certain angle.

The obstacle detecting unit 117b outputs a stop signal for stopping the traveling to the field programmable gate array (FPGA) module 117 when an obstacle is detected based on the sensing value received through the ultrasonic sensor 103. [

The obstacle sensing unit 117b senses an obstacle through the ultrasonic sensor 103 mounted on the front part of the automobile and maintains a proper gap with respect to the preceding vehicle. When the front vehicle suddenly brakes or sudden obstacle appearance occurs, . Therefore, it is possible to recognize obstacles in front of the automobile, thereby maintaining the gap with the front vehicle and preventing collision.

The vehicle battery remaining amount measuring unit 117c measures the remaining amount of the vehicle battery through the LM 3914. [ The remaining battery power is output through the LED. Then, the battery remaining amount is transmitted to the smartphone application 115.

The vehicle battery remaining amount measuring unit 117c measures the remaining amount of the vehicle battery and executes the power saving mode function when the remaining amount is 50% or less. At this time, the power saving mode can perform the maximum speed limitation, LED brightness reduction, and the like. In the power save mode, the vehicle control is changed according to the remaining battery level. When the remaining amount is below a certain level, the maximum speed that can be traveled is limited and the brightness of the LED attached to the vehicle is reduced to reduce battery consumption. Compared to other autonomous vehicles, the remaining battery level is displayed, which is effective for battery management of next generation electric vehicles.

The raspberry pie module 119 includes a traffic light sensing unit 119a, a sign recognition unit 119b, a rainfall sensing unit 119c, and a monitoring control unit 119d.

The traffic light detecting unit 119a analyzes the image and outputs a stop signal when the recognized color of the traffic light is red, and outputs a stop signal when it is green. The traffic light detecting unit 119a processes the captured image using openCV (Open Source Computer Vision) to recognize green or red. Depending on the color of the recognized signal, the signal to control the vehicle's stopping or starting is sent to the Raspberry module Output. Here, openCV is a real-time computer image program library of Intel Corporation, used in applications such as object, face, behavior recognition, and motion tracking.

The traffic light sensing unit 119a divides the image by the simple thresholding technique to specify the RGB range to distinguish colors. It then recognizes whether the traffic light color is red or green by removing the dirt using the cv2.erode (erosion) and cv2.dilate (expansion) functions, and extracting the traffic light model using Hough transform.

The sign recognition unit 119b analyzes the image photographed by the camera 111 using openCV to recognize the sign. If it is the START sign, it outputs the driving signal to the field programmable gate array (FPGA) module 117. At this time, when the vehicle is stopped, the sign recognition unit 119b outputs the vehicle start signal when the START sign is recognized.

The sign recognition unit 119b distinguishes the colors, differentiates the images, and recognizes the signs using the pyresseract.image_to_string function.

The rainfall sensing unit 119c measures the amount of rainfall and makes it possible to run safely even in the rain. The rainfall sensing unit 119c outputs a signal for driving the servo motor 2 (121) to the servo motor 2 (121) when the value sensed by the rainfall sensor 105 becomes equal to or greater than the set value. Here, in the case of a wiper (not shown), it moves at an angle of 180 degrees.

The rainfall sensing unit 119c operates the servo motor 2 (121) in accordance with the sense output terminal (TTL Level). When the sensing value outputted from the rainfall sensor 105 is 1, the servo motor 2 (121) is operated 180 degrees, and when the sensing value is 0, the operation of the servo motor 2 (121) is stopped.

The monitoring control unit 119d transmits a video signal captured by the web cam 113 to the smartphone application 115 via Wi-Fi communication.

Here, the field programmable gate array (FPGA) module 117 and the raspberry module 119 may be implemented using a VHSIC hardware description language (VHDL).

An intelligent automobile control method using the field programmable gate array (FPGA) module 117 and the raspberry module 119 will now be described with reference to FIG.

Referring to FIG. 3, when the vehicle is started, the Raspberry module 119 processes an image photographed through the camera (S101) and determines whether the start character is recognized (S103).

If it is recognized, the raspberry module 119 outputs the driving signal to the field programmable gate array (FPGA) module 117 (S105).

The field programmable gate array (FPGA) module 117 recognizes the lane through the infrared sensor 101 (S107) and determines whether it is the left lane or the right lane (S109).

At this time, if it is determined to be the left lane, the left turn signal is outputted to the servo motor 1 (107) (S111) and the right turn signal is outputted to the servo motor 1 (107) (S113).

On the other hand, the field programmable gate array (FPGA) module 117 determines whether an obstacle detection signal is generated from the ultrasonic sensor 103 (S115), and if it is sensed, decelerates the speed or outputs a stop signal (S117).

Thereafter, the raspberry module 119 detects the color of the traffic light through analysis of the captured image (S119). It is determined whether the traffic light color is red (S121), and if it is red, a stop signal is outputted (S123).

When the rainwater is sensed through the rainfall sensor 105 (S125), the raspberry module 119 operates the wiper through the servo motor 121 (S127). When the rainwater disappears (S129), the wiper drive is stopped (S131).

The Raspberry pie module 119 transmits the video shot by the web cam to the smartphone application 115 in real time (S133).

The field programmable gate array (FPGA) module 117 measures the remaining battery power (S135) and outputs it to the raspberry module 119 (S137).

The field programmable gate array (FPGA) module 117 outputs an LED according to the measured remaining battery power (S139).

The raspberry module 119 displays the battery remaining amount at the bottom of the running stream of the smartphone application (S141).

The embodiments of the present invention described above are not implemented only by the apparatus and method, but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (5)

An infrared sensor, an ultrasonic sensor and a camera,
A Field Programmable Gate Array (FPGA) module connected to the infrared sensor and the ultrasonic sensor for receiving sensing values from the infrared sensor and the ultrasonic sensor and controlling the vehicle,
A sensor module coupled to the field programmable gate array (FPGA) module for receiving the respective sensing values, analyzing images taken by the camera, analyzing the sensed values, and analyzing the images (Raspberry Pi) module for determining whether or not the vehicle is traveling and the traveling direction, and outputting a vehicle control signal according to the determination to the field programmable gate array (FPGA) module
Wherein the control unit is operable to control the control unit. The method of claim 1,
When detecting rainfall, a rainfall sensor that outputs a rainfall occurrence signal,
A servomotor for driving a wiper mounted on a windshield of an automobile, and
Further comprising a web cam mounted on the front of the vehicle,
The Raspberry Pi module,
A signal light detector for outputting the stop signal when the color of the signal light is red and analyzing the image,
A sign recognition unit for analyzing the image to determine whether to travel according to the recognized signboard and outputting a vehicle control signal according to the determination,
A rainfall sensing unit for outputting a signal for driving the servo motor when rainfall is detected from the rainfall sensor,
A monitoring controller for transmitting a video signal photographed by the web cam to a smartphone app through Wi-
Wherein the control unit is operable to control the vehicle. The method of claim 1,
The field programmable gate array (FPGA)
A lane recognition unit for outputting a signal for controlling the direction of the automobile according to a lane recognized based on a sensing value received via the infrared sensor,
An obstacle detection unit for outputting a stop signal or a deceleration signal for stopping the traveling when an obstacle is detected based on a sensing value received through the ultrasonic sensor,
A battery remaining amount measuring unit for measuring a remaining battery level and displaying the remaining battery level on the outside by varying the number of LEDs to be supplied with power in accordance with the remaining battery level;
Wherein the control unit is operable to control the vehicle. 4. The method of claim 3,
The field programmable gate array (FPGA)
A pulse width modulation (PWM) signal is generated by using a built-in clock, and a delay time is measured until the measurement cycle of the ultrasonic sensor, the ultrasonic wave generation time, and the ultrasonic wave reflection time are measured using the PWM signal, , Intelligent automotive control system. 4. The method of claim 3,
The field programmable gate array (FPGA)
And receives a vehicle control signal from the Raspberry Pi module using a relay circuit.

Images