KR102041964B1 - Vehicle control device mounted on vehicle - Google Patents

Abstract

The present invention relates to a vehicle control apparatus provided in a vehicle. A vehicle control apparatus provided in a vehicle according to the present invention includes a vehicle control apparatus for controlling a vehicle including a sensing unit, the communication unit communicating with an external image photographing apparatus photographing a specific area; A controller configured to detect a specific object existing in the specific area based on at least one reference image photographed by the image photographing device; And a display unit configured to output information corresponding to the detected specific object to the captured image photographed by the sensing unit.

Description

Vehicle control device in vehicle {VEHICLE CONTROL DEVICE MOUNTED ON VEHICLE}

The present invention relates to a vehicle control apparatus provided in a vehicle.

The vehicle is a device capable of moving in the direction desired by the user on board. An example is a car.

On the other hand, for the convenience of a user using a vehicle, various sensors, electronic devices, and the like have been provided. In particular, research on the Advanced Driver Assistance System (ADAS) has been actively conducted for the convenience of the user's driving. In addition, development of autonomous vehicles is being actively conducted.

As such, as the development of the ADAS (Advanced Driving Assist System) is actively performed, the necessity of developing a technology capable of maximizing user convenience and safety in vehicle operation is emerging.

On the other hand, when the weather is not good, such as fog, heavy rain, heavy snow, there is a problem that the quality of the captured image is reduced. Accordingly, it is difficult to accurately identify the main information required for driving such as pedestrians, motorcycles, traffic signs, traffic lights, and the like.

It is an object of the present invention to solve the above and other problems. Another object is to provide a vehicle control apparatus for displaying information corresponding to a specific object on a captured image based on a reference image captured by an external image capturing apparatus.

According to an aspect of the present invention to achieve the above or another object, a vehicle control device for controlling a vehicle having a sensing unit, Communication unit for communicating with an external image pickup device for photographing a specific area; A controller configured to detect a specific object existing in the specific area based on at least one reference image photographed by the image photographing device; And a display unit configured to output information corresponding to the detected specific object to a captured image photographing the specific region by the sensing unit .

In at least one example embodiment, the at least one reference image may include a first reference image photographed at a first point in time and a second reference image photographed at a second point in time by the image capturing apparatus.

In this case, the captured image may be an image captured by the sensing unit at the second time point.

As another example, the first reference image may be an image having the least deterioration in image quality due to an external environment element among the images photographed by the image capturing apparatus.

In another embodiment, the specific object may be an object present in the driving route of the vehicle.

In another embodiment, the specific object may be an object present in an area of the image having a deterioration in image quality due to an external environment element or more.

As another embodiment, the specific object may be an object existing only in one of the first reference image and the second reference image.

As another embodiment, the specific object may include an object moving at a predetermined speed or more.

As another embodiment, the controller may calculate information corresponding to the specific object based on an image corresponding to the specific object included in at least one of the first reference image and the second reference image. .

As another embodiment, when the specific object exists only in the first reference image, the controller may control the display unit to output information corresponding to the change.

Referring to the effect of the vehicle control device provided in the vehicle according to the present invention.

According to at least one of the embodiments of the present invention, safe driving may be performed by displaying important specific objects such as pedestrians, motorcycles, signs, and the like on the captured image having low image quality.

In addition, a particular object may be output in various ways, such as an icon, augmented reality, or partly to improve image quality, thereby concentrating the driver's attention.

By displaying the changes on the road, the driver can easily check the updated information.

Further scope of the applicability of the present invention will become apparent from the following detailed description. However, various changes and modifications within the spirit and scope of the present invention can be clearly understood by those skilled in the art, and therefore, specific embodiments, such as the detailed description and the preferred embodiments of the present invention, should be understood as given by way of example only.

1 is a view showing the appearance of a vehicle according to an embodiment of the present invention.
2 is a view of the vehicle according to an embodiment of the present invention from various angles from the outside.
3 to 4 are views illustrating the interior of a vehicle according to an embodiment of the present invention.
5 to 6 are views referred to for describing an object according to an embodiment of the present invention.
7 is a block diagram referenced to describe a vehicle according to an embodiment of the present invention.
8 is a block diagram illustrating a vehicle control apparatus according to the present invention.
9 is a conceptual diagram illustrating an embodiment of a captured image and a reference image.
10 is a flowchart illustrating a control method of a vehicle control apparatus according to the present invention.
11 is a conceptual diagram illustrating an embodiment of outputting a pedestrian to a captured image.
12 is a conceptual diagram illustrating an embodiment of outputting a sign on a captured image.
FIG. 13 is a conceptual diagram illustrating an embodiment of outputting a two-wheeled vehicle to a captured image.
14 is a conceptual diagram illustrating an embodiment of outputting a change to a captured image.

Hereinafter, exemplary embodiments disclosed herein will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used in consideration of ease of specification, and do not have distinct meanings or roles from each other. In addition, in describing the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are intended to facilitate understanding of the embodiments disclosed herein, but are not limited to the technical spirit disclosed herein by the accompanying drawings, all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, the terms "comprises" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

The vehicle described herein may be a concept including an automobile and a motorcycle. In the following, a vehicle is mainly described for a vehicle.

The vehicle described herein may be a concept including both an internal combustion engine vehicle having an engine as a power source, a hybrid vehicle having an engine and an electric motor as a power source, an electric vehicle having an electric motor as a power source, and the like.

In the following description, the left side of the vehicle means the left side of the driving direction of the vehicle, and the right side of the vehicle means the right side of the driving direction of the vehicle.

1 is a view showing the appearance of a vehicle according to an embodiment of the present invention.

2 is a view of the vehicle according to an embodiment of the present invention from various angles from the outside.

3 to 4 are views illustrating the interior of a vehicle according to an embodiment of the present invention.

5 to 6 are views referred to for describing an object according to an embodiment of the present invention.

7 is a block diagram referenced to describe a vehicle according to an embodiment of the present invention.

1 to 7, the vehicle 100 may include a wheel that rotates by a power source and a steering input device 510 for adjusting a traveling direction of the vehicle 100.

The vehicle 100 may be an autonomous vehicle.

The vehicle 100 may be switched to an autonomous driving mode or a manual mode based on a user input.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the received user input through the user interface device 200.

The vehicle 100 may be switched to the autonomous driving mode or the manual mode based on the driving situation information. The driving situation information may be generated based on the object information provided by the object detecting apparatus 300.

For example, the vehicle 100 may be switched from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the driving situation information generated by the object detecting apparatus 300.

For example, the vehicle 100 may switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on the driving situation information received through the communication device 400.

The vehicle 100 may switch from the manual mode to the autonomous driving mode or from the autonomous driving mode to the manual mode based on information, data, and signals provided from an external device.

When the vehicle 100 is driven in the autonomous driving mode, the autonomous vehicle 100 may be driven based on the driving system 700.

For example, the autonomous vehicle 100 may be driven based on information, data, or signals generated by the driving system 710, the parking system 740, and the parking system 750.

When the vehicle 100 is driven in the manual mode, the autonomous vehicle 100 may receive a user input for driving through the driving manipulation apparatus 500. Based on a user input received through the driving manipulation apparatus 500, the vehicle 100 may be driven.

The overall length is the length from the front to the rear of the vehicle 100, the width is the width of the vehicle 100, and the height is the length from the bottom of the wheel to the roof. In the following description, the full length direction L is a direction in which the full length measurement of the vehicle 100 is a reference, the full width direction W is a direction in which the full width measurement of the vehicle 100 is a reference, and the total height direction H is a vehicle. It may mean the direction which is the reference of the height measurement of (100).

As illustrated in FIG. 7, the vehicle 100 includes a user interface device 200, an object detecting device 300, a communication device 400, a driving manipulation device 500, a vehicle driving device 600, and a traveling system. 700, a navigation system 770, a sensing unit 120, an interface unit 130, a memory 140, a control unit 170, and a power supply unit 190 may be included.

According to an embodiment, the vehicle 100 may further include other components in addition to the components described herein, or may not include some of the components described.

The user interface device 200 is a device for communicating with the vehicle 100 and a user. The user interface device 200 may receive a user input and provide the user with information generated in the vehicle 100. The vehicle 100 may implement user interfaces (UI) or user experience (UX) through the user interface device 200.

The user interface device 200 may include an input unit 210, an internal camera 220, a biometric detector 230, an output unit 250, and a processor 270.

According to an embodiment, the user interface device 200 may further include other components in addition to the described components, or may not include some of the described components.

The input unit 210 is for receiving information from a user, and the data collected by the input unit 210 may be analyzed by the processor 270 and processed as a user's control command.

The input unit 210 may be disposed in the vehicle. For example, the input unit 210 may include one region of a steering wheel, one region of an instrument panel, one region of a seat, one region of each pillar, and a door. one area of the door, one area of the center console, one area of the head lining, one area of the sun visor, one area of the windshield or of the window It may be disposed in one area or the like.

The input unit 210 may include a voice input unit 211, a gesture input unit 212, a touch input unit 213, and a mechanical input unit 214.

The voice input unit 211 may convert a user's voice input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170.

The voice input unit 211 may include one or more microphones.

The gesture input unit 212 may convert a user's gesture input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170.

The gesture input unit 212 may include at least one of an infrared sensor and an image sensor for detecting a user's gesture input.

According to an embodiment, the gesture input unit 212 may detect a 3D gesture input of the user. To this end, the gesture input unit 212 may include a light output unit or a plurality of image sensors for outputting a plurality of infrared light.

The gesture input unit 212 may detect a user's 3D gesture input through a time of flight (TOF) method, a structured light method, or a disparity method.

The touch input unit 213 may convert a user's touch input into an electrical signal. The converted electrical signal may be provided to the processor 270 or the controller 170.

The touch input unit 213 may include a touch sensor for detecting a user's touch input.

According to an embodiment, the touch input unit 213 may be integrally formed with the display unit 251 to implement a touch screen. Such a touch screen may provide an input interface and an output interface between the vehicle 100 and the user.

The mechanical input unit 214 may include at least one of a button, a dome switch, a jog wheel, and a jog switch. The electrical signal generated by the mechanical input unit 214 may be provided to the processor 270 or the controller 170.

The mechanical input unit 214 may be disposed on a steering wheel, a cente facia, a center console, a cockpit module, a door, or the like.

The internal camera 220 may acquire a vehicle interior image. The processor 270 may detect a state of the user based on the vehicle interior image. The processor 270 may acquire the gaze information of the user from the vehicle interior image. The processor 270 may detect a gesture of the user in the vehicle interior image.

The biometric detector 230 may acquire biometric information of the user. The biometric detector 230 may include a sensor for acquiring biometric information of the user, and may acquire fingerprint information, heartbeat information, etc. of the user using the sensor. Biometric information may be used for user authentication.

The output unit 250 is for generating output related to visual, auditory or tactile.

The output unit 250 may include at least one of the display unit 251, the audio output unit 252, and the haptic output unit 253.

The display unit 251 may display graphic objects corresponding to various pieces of information.

The display unit 251 is a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display (flexible) display, a 3D display, or an e-ink display.

The display unit 251 forms a layer structure or is integrally formed with the touch input unit 213 to implement a touch screen.

The display unit 251 may be implemented as a head up display (HUD). When the display unit 251 is implemented as a HUD, the display unit 251 may include a projection module to output information through an image projected on a wind shield or a window.

The display unit 251 may include a transparent display. The transparent display can be attached to the wind shield or window.

The transparent display may display a predetermined screen while having a predetermined transparency. Transparent display, in order to have transparency, transparent display is transparent thin film elecroluminescent (TFEL), transparent organic light-emitting diode (OLED), transparent liquid crystal display (LCD), transmissive transparent display, transparent light emitting diode (LED) display It may include at least one of. The transparency of the transparent display can be adjusted.

The user interface device 200 may include a plurality of display units 251a to 251g.

The display unit 251 includes one region of the steering wheel, one region 521a, 251b, and 251e of the instrument panel, one region 251d of the seat, one region 251f of each filler, and one region of the door ( 251g), one area of the center console, one area of the head lining, and one area of the sun visor, or may be implemented in one area 251c of the wind shield and one area 251h of the window.

The sound output unit 252 converts an electrical signal provided from the processor 270 or the controller 170 into an audio signal and outputs the audio signal. To this end, the sound output unit 252 may include one or more speakers.

The haptic output unit 253 generates a tactile output. For example, the haptic output unit 253 may vibrate the steering wheel, the seat belt, and the seats 110FL, 110FR, 110RL, and 110RR so that the user may recognize the output.

The processor 270 may control the overall operation of each unit of the user interface device 200.

According to an embodiment, the user interface device 200 may include a plurality of processors 270 or may not include the processor 270.

When the processor 270 is not included in the user interface device 200, the user interface device 200 may be operated under the control of the processor or the controller 170 of another device in the vehicle 100.

The user interface device 200 may be referred to as a vehicle display device.

The user interface device 200 may be operated under the control of the controller 170.

The object detecting apparatus 300 is a device for detecting an object located outside the vehicle 100.

The object may be various objects related to the driving of the vehicle 100.

5 to 6, the object O includes a lane OB10, another vehicle OB11, a pedestrian OB12, a two-wheeled vehicle OB13, traffic signals OB14, OB15, light, a road, a structure, Speed bumps, features, animals and the like can be included.

The lane OB10 may be a driving lane, a lane next to the driving lane, and a lane in which an opposite vehicle travels. The lane OB10 may be a concept including left and right lines forming a lane.

The other vehicle OB11 may be a vehicle that is driving around the vehicle 100. The other vehicle may be a vehicle located within a predetermined distance from the vehicle 100. For example, the other vehicle OB11 may be a vehicle that precedes or follows the vehicle 100.

The pedestrian OB12 may be a person located near the vehicle 100. The pedestrian OB12 may be a person located within a predetermined distance from the vehicle 100. For example, the pedestrian OB12 may be a person located on a sidewalk or a roadway.

The two-wheeled vehicle OB13 may be a vehicle that is positioned around the vehicle 100 and moves using two wheels. The motorcycle OB13 may be a vehicle having two wheels located within a predetermined distance from the vehicle 100. For example, the motorcycle OB13 may be a motorcycle or a bicycle located on sidewalks or roadways.

The traffic signal may include a traffic light OB15, a traffic sign OB14, a pattern or text drawn on a road surface.

The light may be light generated by a lamp provided in another vehicle, light generated by a street lamp, or sunlight.

The road may include a road surface, a curve, an uphill slope, a slope downhill, or the like.

The structure may be an object located around a road and fixed to the ground. For example, the structure may include a street lamp, a roadside tree, a building, a power pole, a traffic light, a bridge.

The features may include mountains, hills, and the like.

On the other hand, the object may be classified into a moving object and a fixed object. For example, the moving object may be a concept including another vehicle and a pedestrian. For example, the fixed object may be a concept including a traffic signal, a road, and a structure.

The object detecting apparatus 300 may include a camera 310, a radar 320, a lidar 330, an ultrasonic sensor 340, an infrared sensor 350, and a processor 370.

According to an embodiment, the object detecting apparatus 300 may further include other components in addition to the described components, or may not include some of the described components.

The camera 310 may be located at a suitable place outside the vehicle to acquire an image outside the vehicle. The camera 310 may be a mono camera, a stereo camera 310a, an around view monitoring (AVM) camera 310b, or a 360 degree camera.

For example, the camera 310 may be disposed in close proximity to the front windshield in the interior of the vehicle in order to acquire an image in front of the vehicle. Alternatively, the camera 310 may be disposed around the front bumper or the radiator grille.

For example, the camera 310 may be disposed in close proximity to the rear glass in the interior of the vehicle to acquire an image of the rear of the vehicle. Alternatively, the camera 310 may be disposed around the rear bumper, the trunk, or the tail gate.

For example, the camera 310 may be disposed in close proximity to at least one of the side windows in the interior of the vehicle to acquire an image of the vehicle side. Alternatively, the camera 310 may be arranged around the side mirror, fender or door.

The camera 310 may provide the obtained image to the processor 370.

The radar 320 may include an electromagnetic wave transmitter and a receiver. The radar 320 may be implemented in a pulse radar method or a continuous wave radar method in terms of radio wave firing principle. The radar 320 may be implemented by a frequency modulated continuous wave (FSCW) method or a frequency shift key (FSK) method according to a signal waveform among the continuous wave radar methods.

The radar 320 detects an object based on a time of flight (TOF) method or a phase-shift method based on an electromagnetic wave, and detects the position of the detected object, distance to the detected object, and relative velocity. Can be detected.

The radar 320 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The lidar 330 may include a laser transmitter and a receiver. The lidar 330 may be implemented in a time of flight (TOF) method or a phase-shift method.

The lidar 330 may be implemented as driven or non-driven.

When implemented in a driving manner, the lidar 330 may be rotated by a motor and detect an object around the vehicle 100.

When implemented in a non-driven manner, the lidar 330 may detect an object located within a predetermined range with respect to the vehicle 100 by optical steering. The vehicle 100 may include a plurality of non-driven lidars 330.

The lidar 330 detects an object based on a time of flight (TOF) method or a phase-shift method using laser light, and detects an object, a position of the detected object, a distance from the detected object, and Relative speed can be detected.

The lidar 330 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The ultrasonic sensor 340 may include an ultrasonic transmitter and a receiver. The ultrasonic sensor 340 may detect an object based on the ultrasonic wave, and detect a position of the detected object, a distance to the detected object, and a relative speed.

The ultrasonic sensor 340 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The infrared sensor 350 may include an infrared transmitter and a receiver. The infrared sensor 340 may detect an object based on infrared light, and detect a position of the detected object, a distance to the detected object, and a relative speed.

The infrared sensor 350 may be disposed at an appropriate position outside the vehicle to detect an object located in front, rear, or side of the vehicle.

The processor 370 may control overall operations of each unit of the object detecting apparatus 300.

The processor 370 may detect and track the object based on the obtained image. The processor 370 may perform operations such as calculating a distance to an object and calculating a relative speed with the object through an image processing algorithm.

The processor 370 may detect and track the object based on the reflected electromagnetic wave reflected by the transmitted electromagnetic wave to the object. The processor 370 may perform an operation such as calculating a distance from the object, calculating a relative speed with the object, and the like based on the electromagnetic waves.

The processor 370 may detect and track the object based on the reflected laser light reflected by the transmitted laser back to the object. The processor 370 may perform an operation such as calculating a distance from the object, calculating a relative speed with the object, and the like based on the laser light.

The processor 370 may detect and track the object based on the reflected ultrasound, in which the transmitted ultrasound is reflected by the object and returned. The processor 370 may perform an operation such as calculating a distance from the object, calculating a relative speed with the object, and the like based on the ultrasound.

The processor 370 may detect and track the object based on the reflected infrared light from which the transmitted infrared light is reflected back to the object. The processor 370 may perform an operation such as calculating a distance to the object, calculating a relative speed with the object, and the like based on the infrared light.

According to an embodiment, the object detecting apparatus 300 may include a plurality of processors 370 or may not include the processor 370. For example, each of the camera 310, the radar 320, the lidar 330, the ultrasonic sensor 340, and the infrared sensor 350 may individually include a processor.

When the processor 370 is not included in the object detecting apparatus 300, the object detecting apparatus 300 may be operated under the control of the processor or the controller 170 of the apparatus in the vehicle 100.

The object detecting apparatus 400 may be operated under the control of the controller 170.

The communication device 400 is a device for performing communication with an external device. Here, the external device may be another vehicle, a mobile terminal or a server.

The communication device 400 may include at least one of a transmit antenna, a receive antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication.

The communication device 400 may include a short range communication unit 410, a location information unit 420, a V2X communication unit 430, an optical communication unit 440, a broadcast transceiver 450, and a processor 470.

According to an embodiment, the communication device 400 may further include other components in addition to the described components, or may not include some of the described components.

The short range communication unit 410 is a unit for short range communication. The short range communication unit 410 may include Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), and Wi-Fi (Wireless). Local area communication may be supported using at least one of Fidelity, Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technologies.

The short range communication unit 410 may form short range wireless networks to perform short range communication between the vehicle 100 and at least one external device.

The location information unit 420 is a unit for obtaining location information of the vehicle 100. For example, the location information unit 420 may include a global positioning system (GPS) module or a differential global positioning system (DGPS) module.

The V2X communication unit 430 is a unit for performing wireless communication with a server (V2I: Vehicle to Infra), another vehicle (V2V: Vehicle to Vehicle), or a pedestrian (V2P: Vehicle to Pedestrian). The V2X communication unit 430 may include an RF circuit that can implement a communication with the infrastructure (V2I), an inter-vehicle communication (V2V), and a communication with the pedestrian (V2P).

The optical communication unit 440 is a unit for performing communication with an external device via light. The optical communication unit 440 may include an optical transmitter that converts an electrical signal into an optical signal and transmits the external signal to the outside, and an optical receiver that converts the received optical signal into an electrical signal.

According to an embodiment, the light emitting unit may be formed to be integrated with the lamp included in the vehicle 100.

The broadcast transceiver 450 is a unit for receiving a broadcast signal from an external broadcast management server or transmitting a broadcast signal to a broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal.

The processor 470 may control the overall operation of each unit of the communication device 400.

According to an embodiment, the communication device 400 may include a plurality of processors 470 or may not include the processor 470.

When the processor 470 is not included in the communication device 400, the communication device 400 may be operated under the control of the processor or the controller 170 of another device in the vehicle 100.

Meanwhile, the communication device 400 may implement a vehicle display device together with the user interface device 200. In this case, the vehicle display device may be called a telematics device or an AVN (Audio Video Navigation) device.

The communication device 400 may be operated under the control of the controller 170.

The driving operation apparatus 500 is a device that receives a user input for driving.

In the manual mode, the vehicle 100 may be driven based on a signal provided by the driving manipulation apparatus 500.

The driving manipulation apparatus 500 may include a steering input apparatus 510, an acceleration input apparatus 530, and a brake input apparatus 570.

The steering input device 510 may receive a driving direction input of the vehicle 100 from the user. The steering input device 510 is preferably formed in a wheel shape to enable steering input by rotation. According to an embodiment, the steering input device may be formed in the form of a touch screen, a touch pad, or a button.

The acceleration input device 530 may receive an input for accelerating the vehicle 100 from a user. The brake input device 570 may receive an input for deceleration of the vehicle 100 from a user. The acceleration input device 530 and the brake input device 570 are preferably formed in the form of a pedal. According to an embodiment, the acceleration input device or the brake input device may be formed in the form of a touch screen, a touch pad, or a button.

The driving manipulation apparatus 500 may be operated under the control of the controller 170.

The vehicle drive device 600 is a device that electrically controls the driving of various devices in the vehicle 100.

The vehicle driving apparatus 600 may include a power train driver 610, a chassis driver 620, a door / window driver 630, a safety device driver 640, a lamp driver 650, and an air conditioning driver 660. Can be.

According to an exemplary embodiment, the vehicle driving apparatus 600 may further include other components in addition to the described components, or may not include some of the described components.

On the other hand, the vehicle driving device 600 may include a processor. Each unit of the vehicle drive apparatus 600 may each include a processor individually.

The power train driver 610 may control the operation of the power train device.

The power train driver 610 may include a power source driver 611 and a transmission driver 612.

The power source driver 611 may control the power source of the vehicle 100.

For example, when the fossil fuel-based engine is a power source, the power source driver 610 may perform electronic control of the engine. Thereby, the output torque of an engine, etc. can be controlled. The power source drive unit 611 can adjust the engine output torque under the control of the control unit 170.

For example, when the electric energy based motor is a power source, the power source driver 610 may control the motor. The power source driver 610 may adjust the rotational speed, torque, and the like of the motor under the control of the controller 170.

The transmission driver 612 may control the transmission.

The transmission driver 612 can adjust the state of the transmission. The transmission drive part 612 can adjust the state of a transmission to forward D, backward R, neutral N, or parking P. FIG.

On the other hand, when the engine is a power source, the transmission drive unit 612 can adjust the bite state of the gear in the forward D state.

The chassis driver 620 may control the operation of the chassis device.

The chassis driver 620 may include a steering driver 621, a brake driver 622, and a suspension driver 623.

The steering driver 621 may perform electronic control of a steering apparatus in the vehicle 100. The steering driver 621 may change the traveling direction of the vehicle.

The brake driver 622 may perform electronic control of a brake apparatus in the vehicle 100. For example, the speed of the vehicle 100 may be reduced by controlling the operation of the brake disposed on the wheel.

On the other hand, the brake drive unit 622 can individually control each of the plurality of brakes. The brake driver 622 may control the braking force applied to the plurality of wheels differently.

The suspension driver 623 may perform electronic control of a suspension apparatus in the vehicle 100. For example, when there is a curvature on the road surface, the suspension driver 623 may control the suspension device to control the vibration of the vehicle 100 to be reduced.

Meanwhile, the suspension driver 623 may individually control each of the plurality of suspensions.

The door / window driver 630 may perform electronic control of a door apparatus or a window apparatus in the vehicle 100.

The door / window driver 630 may include a door driver 631 and a window driver 632.

The door driver 631 may control the door apparatus. The door driver 631 may control opening and closing of the plurality of doors included in the vehicle 100. The door driver 631 may control the opening or closing of a trunk or a tail gate. The door driver 631 may control the opening or closing of the sunroof.

The window driver 632 may perform electronic control of the window apparatus. The opening or closing of the plurality of windows included in the vehicle 100 may be controlled.

The safety device driver 640 may perform electronic control of various safety apparatuses in the vehicle 100.

The safety device driver 640 may include an airbag driver 641, a seat belt driver 642, and a pedestrian protection device driver 643.

The airbag driver 641 may perform electronic control of an airbag apparatus in the vehicle 100. For example, the airbag driver 641 may control the airbag to be deployed when the danger is detected.

The seat belt driver 642 may perform electronic control of a seatbelt appartus in the vehicle 100. For example, the seat belt driver 642 may control the passenger to be fixed to the seats 110FL, 110FR, 110RL, and 110RR by using the seat belt when detecting a danger.

The pedestrian protection device driver 643 may perform electronic control of the hood lift and the pedestrian airbag. For example, the pedestrian protection device driver 643 may control the hood lift up and the pedestrian air bag to be deployed when the collision with the pedestrian is detected.

The lamp driver 650 may perform electronic control of various lamp apparatuses in the vehicle 100.

The air conditioning driver 660 may perform electronic control of an air conditioner in the vehicle 100. For example, when the temperature inside the vehicle is high, the air conditioning driver 660 may control the air conditioning apparatus to operate to supply cool air to the inside of the vehicle.

The vehicle driving apparatus 600 may include a processor. Each unit of the vehicle drive apparatus 600 may each include a processor individually.

The vehicle driving apparatus 600 may be operated under the control of the controller 170.

The travel system 700 is a system for controlling various travels of the vehicle 100. The navigation system 700 can be operated in an autonomous driving mode.

The travel system 700 can include a travel system 710, a parking system 740, and a parking system 750.

In some embodiments, the navigation system 700 may further include other components in addition to the described components, or may not include some of the described components.

Meanwhile, the driving system 700 may include a processor. Each unit of the navigation system 700 may each include a processor individually.

In some embodiments, when the driving system 700 is implemented in software, the driving system 700 may be a lower concept of the controller 170.

According to an exemplary embodiment, the driving system 700 may include at least one of the user interface device 200, the object detecting device 300, the communication device 400, the vehicle driving device 600, and the controller 170. It may be a concept to include.

The traveling system 710 may perform driving of the vehicle 100.

The driving system 710 may receive navigation information from the navigation system 770, provide a control signal to the vehicle driving apparatus 600, and perform driving of the vehicle 100.

The driving system 710 may receive object information from the object detecting apparatus 300 and provide a control signal to the vehicle driving apparatus 600 to perform driving of the vehicle 100.

The driving system 710 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving device 600, and perform driving of the vehicle 100.

The taking-out system 740 may perform taking out of the vehicle 100.

The taking-out system 740 may receive navigation information from the navigation system 770, provide a control signal to the vehicle driving apparatus 600, and perform take-out of the vehicle 100.

The taking-out system 740 may receive the object information from the object detecting apparatus 300, provide a control signal to the vehicle driving apparatus 600, and perform take-out of the vehicle 100.

The taking-off system 740 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving apparatus 600, and perform take-out of the vehicle 100.

The parking system 750 may perform parking of the vehicle 100.

The parking system 750 may receive navigation information from the navigation system 770, provide a control signal to the vehicle driving apparatus 600, and perform parking of the vehicle 100.

The parking system 750 may receive the object information from the object detecting apparatus 300, provide a control signal to the vehicle driving apparatus 600, and perform parking of the vehicle 100.

The parking system 750 may receive a signal from an external device through the communication device 400, provide a control signal to the vehicle driving device 600, and perform parking of the vehicle 100.

The navigation system 770 can provide navigation information. The navigation information may include at least one of map information, set destination information, route information according to the destination setting, information on various objects on the route, lane information, and current location information of the vehicle.

The navigation system 770 may include a memory and a processor. The memory may store navigation information. The processor may control the operation of the navigation system 770.

According to an embodiment, the navigation system 770 may receive information from an external device through the communication device 400 and update the pre-stored information.

According to an embodiment, the navigation system 770 may be classified as a subcomponent of the user interface device 200.

The sensing unit 120 may sense a state of the vehicle. The sensing unit 120 may include an attitude sensor (for example, a yaw sensor, a roll sensor, a pitch sensor), a collision sensor, a wheel sensor, a speed sensor, and an inclination. Sensor, Weight Sensor, Heading Sensor, Gyro Sensor, Position Module, Vehicle Forward / Reverse Sensor, Battery Sensor, Fuel Sensor, Tire Sensor, Steering Sensor by Handle Rotation, Vehicle And an internal temperature sensor, an in-vehicle humidity sensor, an ultrasonic sensor, an illuminance sensor, an accelerator pedal position sensor, a brake pedal position sensor, and the like.

The sensing unit 120 includes vehicle attitude information, vehicle collision information, vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery Acquire sensing signals for information, fuel information, tire information, vehicle lamp information, vehicle internal temperature information, vehicle internal humidity information, steering wheel rotation angle, vehicle external illumination, pressure applied to the accelerator pedal, pressure applied to the brake pedal, and the like. can do.

The sensing unit 120 may further include an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake temperature sensor (ATS), a water temperature sensor (WTS), and a throttle position sensor. (TPS), TDC sensor, crank angle sensor (CAS), and the like.

The interface unit 130 may serve as a path to various types of external devices connected to the vehicle 100. For example, the interface unit 130 may include a port connectable with the mobile terminal, and may connect with the mobile terminal through the port. In this case, the interface unit 130 may exchange data with the mobile terminal.

Meanwhile, the interface unit 130 may serve as a path for supplying electrical energy to the connected mobile terminal. When the mobile terminal is electrically connected to the interface unit 130, under the control of the controller 170, the interface unit 130 may provide the mobile terminal with electrical energy supplied from the power supply unit 190.

The memory 140 is electrically connected to the controller 170. The memory 140 may store basic data for the unit, control data for controlling the operation of the unit, and input / output data. The memory 140 may be various storage devices such as a ROM, a RAM, an EPROM, a flash drive, a hard drive, and the like, in hardware. The memory 140 may store various data for overall operation of the vehicle 100, such as a program for processing or controlling the controller 170.

According to an embodiment, the memory 140 may be integrally formed with the controller 170 or may be implemented as a subcomponent of the controller 170.

The controller 170 may control the overall operation of each unit in the vehicle 100. The controller 170 may be referred to as an electronic control unit (ECU).

The power supply unit 190 may supply power required for the operation of each component under the control of the controller 170. In particular, the power supply unit 190 may receive power from a battery inside the vehicle.

One or more processors and controllers 170 included in the vehicle 100 may include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and FPGAs ( It may be implemented using at least one of field programmable gate arrays, processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions.

Meanwhile, the vehicle 100 related to the present invention may include a vehicle control device 800.

The vehicle control apparatus 800 may control at least one of the components described with reference to FIG. 7. From this point of view, the vehicle control device 800 may be a control unit 170.

Not limited to this, the vehicle control device 800 may be a separate configuration independent of the control unit 170. When the vehicle control device 800 is implemented as a component independent of the controller 170, the vehicle control device 800 may be provided in a portion of the vehicle 100.

Hereinafter, for convenience of description, the vehicle control apparatus 800 will be described as having a separate configuration independent of the controller 170. The function (operation) and control method described for the vehicle control apparatus 800 in the present specification may be performed by the controller 170 of the vehicle. That is, all contents described in relation to the vehicle control apparatus 800 may be analogously applied to the controller 170 in the same or similar manner.

In addition, the vehicle control apparatus 800 described herein may include some of the components described in FIG. 7 and various components provided in the vehicle. In the present specification, for convenience of description, the components described in FIG. 7 and various components provided in the vehicle will be described with separate names and reference numerals.

Hereinafter, a vehicle control apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings.

8 is a block diagram illustrating a vehicle control apparatus according to the present invention.

Referring to FIG. 8, the vehicle control apparatus 800 according to the present invention is a device for controlling the vehicle 100 including the sensing unit 120, and includes a communication unit 810, a control unit 820, and a display unit 830. It may include.

The communication unit 810 communicates with an external image capturing apparatus that captures a specific area.

The communication unit 810 is configured to communicate with various components described with reference to FIG. 7. For example, the communication unit 810 may receive various types of information provided through a controller are network (CAN).

As another example, the communication unit 810 may communicate with all devices that can communicate, such as a mobile terminal, a server, and another vehicle. This may be referred to as vehicle to everything (V2X) communication. V2X communication can be defined as a technology for exchanging or sharing information such as traffic conditions while communicating with road infrastructure and other vehicles while driving.

The specific area may include a point (area) photographed by a camera provided in the vehicle 100 at the position of the current vehicle 100. Therefore, the specific area may be changed according to the position, the moving speed, the moving direction, etc. of the current vehicle 100.

The external image capturing apparatus is a device capable of capturing the specific region, and may be installed in various external facilities. For example, it may include a CCTV installed on the road, a photographing apparatus attached to various servers or facilities.

As described above, as the vehicle 100 moves, the specific area photographed by the camera provided in the vehicle 100 may continuously change. Accordingly, the external image capturing apparatus capable of capturing the specific area may also be changed. Specific examples related to this will be described later.

The controller 820 detects a specific object existing in the specific area based on at least one reference image captured by the image capturing apparatus.

The reference image refers to an image obtained by capturing the specific area by the image capturing apparatus.

In at least one reference image, the at least one reference image may include a first reference image photographed at a first point in time and a second reference image photographed at a second point in time by the image capturing apparatus. have.

In this case, the first reference image may be an image having the least deterioration in image quality due to an external environment element among the images photographed by the image capturing apparatus.

As a specific example, the first reference image may be an image captured by the image capturing apparatus on a day in a favorable weather condition. Accordingly, the first reference image may be a clear image with the least deterioration in image quality due to an external environment element. The second reference image may be an image currently (second view) photographed by the image capturing apparatus.

The specific object may include an object to be considered when operating the vehicle 100 as an object existing in the specific area.

According to an embodiment, the specific object may be an object existing in a driving path of the vehicle 100, and may be a traffic sign, a traffic light, a pedestrian, a two-wheeled vehicle, etc. existing on a road on which the vehicle 100 travels. Alternatively, the object may be an object moving at a predetermined speed or more.

As a specific embodiment, the pedestrian passing through the driving path of the vehicle 100 may be detected as a specific object from the reference image (second reference image) currently photographed by the image capturing apparatus.

The display unit 830 outputs the information corresponding to the detected specific object to the captured image photographed by the sensing unit 120.

The display unit 830 may output various pieces of information under the control of the controller 820. The display unit 830 may be implemented as a navigation device, a head up display (HUD), or a transparent display. When implemented as the transparent display, the display unit 830 may be disposed in a windshield or a window of the vehicle.

The photographed image may include an image currently photographed by a camera provided in the vehicle 100. Information corresponding to the specific object may be output to the captured image in various ways.

In an embodiment, the icon representing the specific object may be displayed as augmented reality (AR). Alternatively, an image of the specific object (a portion of the reference image photographed from the specific object) may be output to the captured image.

As a specific example, an icon representing the position of a pedestrian may be displayed on the captured image having low image quality. As a result, an important specific object such as a pedestrian is displayed on the captured image having a low image quality, and safe driving can be achieved.

The at least one reference image may include a first reference image photographed at a first point in time and a second reference image photographed at a second point in time by the image capturing apparatus.

In this case, the photographed image may be an image photographed by the sensing unit 120 at the second time point. The first reference image may be an image having the least deterioration in image quality due to an external environment element among the images photographed by the image capturing apparatus.

In another embodiment, the specific object may be an object present in the driving path of the vehicle 100. In this case, the specific object may include a pedestrian passing through a road on which the vehicle 100 travels.

In another embodiment, the specific object may be an object present in an area of the image having a deterioration in image quality due to an external environment element or more.

As another embodiment, the specific object may include an object moving at a predetermined speed or more.

9 is a conceptual diagram illustrating an embodiment of a captured image and a reference image.

Referring to FIG. 9, a camera provided in the vehicle 100 may capture a first specific area 910 in front of the vehicle to obtain a first captured image. In addition, as the vehicle 100 travels, a camera provided in the vehicle 100 may capture the second specific region 920 in front of the vehicle 100 to obtain a second captured image.

In this case, a plurality of image capturing devices (CCTV, etc.) may be disposed along the driving road. Each of the image capturing apparatuses 1010 and 1020 includes the first reference image and the vehicle (the first reference image which is the image having the least deterioration in image quality due to an external environmental element) among the images photographed by the image capturing apparatuses 1010 and 1020. The second reference image captured at the same point in time as when the specific region is photographed may be generated (photographed).

In an embodiment, when the image quality of the first captured image is less than or equal to a predetermined level, the first-first reference image of the first image capturing apparatus 1010 photographing the first specific region 910 at different viewpoints (times) The image and the 1-2 reference image may be received.

In this case, the first-first reference image is an image having a quality of a predetermined degree or more previously photographed, and the first-second reference image is a time point when the vehicle 100 photographs the first specific region 910. The image may be taken at the same time.

Thereafter, based on the reference images, a specific object existing in a portion of low quality of the first captured image may be detected. For example, the pedestrian passing through the first specific area 910 may be blurred on the first captured image. In this case, the pedestrian may be detected based on the reference images, and the presence of the pedestrian may be displayed on the first captured image.

In the following embodiment, when the image quality of the second captured image is less than or equal to a preset level, the 2-1 reference image photographed from the second image capturing apparatus 1020 at a different time point may be used. The second reference image may be received.

In this case, the 2-1 reference image is an image having a quality of a predetermined degree or more previously photographed, and the 2-2 reference image is a time point when the vehicle 100 captures the second specific region 920. The image may be taken at the same time.

Thereafter, based on the reference images, a specific object existing in a portion of low quality of the second captured image may be detected. For example, an object moving at a speed higher than a predetermined speed existing in the second specific area 920 may be blurred in the second captured image. In this case, the moving object may be detected based on the reference images, and the moving object may be displayed on the second captured image.

As another embodiment, the specific regions corresponding to one viewpoint may be plural, and thus may communicate with the plurality of image capturing apparatuses, respectively.

As a specific example, the first captured image photographing the first specific region 910 forward and the third captured image photographing the lateral third specific region 930 may be obtained.

Similarly, when the image quality of the third captured image is less than or equal to a preset level, the 3-1th reference image and the 3-2 of the third specific region 930 captured by the image capturing apparatus from different viewpoints. The reference image may be received.

In this case, the image capturing apparatus is a device capable of capturing the third specific region 930 and may be the first image capturing apparatus 1010 or another image capturing apparatus.

In addition, the 3-1 reference image is an image having a quality of a predetermined degree or more previously taken, and the 3-2 reference image is a time point when the vehicle 100 captures the third specific region 930. The image may be taken at the same time. That is, the first captured image, the 1-2 reference image, the third captured image, and the third-2 reference image are images captured at the same time.

Subsequently, based on the reference images (the 3-1 reference image and the 3-2 reference image), a specific object existing in a portion of low quality of the third captured image may be detected.

For example, road facilities existing in the third specific area 930 may be blurred on the third captured image. In this case, the road facility may be detected based on the reference images, and the third facility image may indicate that the road facility exists.

10 is a flowchart illustrating a control method of a vehicle control apparatus according to the present invention.

Referring to FIG. 10, a step (S1010) of receiving a captured image of photographing a specific region from a camera provided in the vehicle 100 at a present time is performed.

In an embodiment, the front photographed image captured by the camera provided in the vehicle 100 may be received through a controller are network (CAN).

Subsequently, the step of comparing whether the image quality of the captured image is equal to or greater than a predetermined reference value is performed (S1020).

That is, the step of determining how much the image quality of the captured image is determined by external environmental factors such as fog, heavy rain, and heavy snow.

As a result, when the image quality of the captured image is equal to or greater than the reference value, outputting the captured image to the display unit 830 of the vehicle 100 is performed (S1030).

In contrast, if the quality of the captured image is less than the reference value, receiving the first reference image and the second reference image from an external image capturing apparatus (S1040) is performed.

In an embodiment, the first reference image may be an image having a good image quality when the weather condition is good. The second reference image may be an image captured by the same specific region (front of the vehicle) at the same point in time that the captured image is captured.

Subsequently, in operation S1050, a specific object is detected from the first reference image and the second reference image.

According to an embodiment, the specific object may be an object existing in an area of the image having a deterioration in image quality due to an external environment element or more.

Thereafter, the process of displaying the information corresponding to the specific object and outputting the captured image (S1060) is performed.

In an embodiment, the pedestrian may be blurred due to low image quality in the captured image. Accordingly, the pedestrian may be detected from the second reference image, and the presence of the pedestrian may be displayed on the captured image.

Hereinafter, embodiments of detecting and outputting a specific object in the captured image by the camera of the vehicle 100 based on the reference images photographed by the image capturing apparatus will be described.

In this case, in order to determine whether the image capturing apparatus photographs the same specific region as the vehicle 100, location information of the image capturing apparatus and the vehicle 100 may be used.

Meanwhile, the specific object may be an object existing in an area in which the deterioration of image quality due to an external environment element is greater than or equal to a predetermined level in the captured image.

Specifically, the specific object may be an object (pedestrian, sign, traffic light, etc.) existing in the portion having the lowest image quality of the captured image.

In another embodiment, the specific object may be an object present in the driving route of the vehicle. Specifically, the specific object may include a pedestrian, a road sign, a two-wheeled vehicle, other traffic facilities, and the like passing through a road on which the vehicle is driven.

11 is a conceptual diagram illustrating an embodiment of outputting a pedestrian to a captured image.

Referring to FIG. 11, the captured image 1110 photographing a specific region of the current vehicle 100 may be blurred with a quality of a preset level or less.

Accordingly, based on the second reference image 1120 of the current external image capturing apparatus, the specific object 1130 that is blurred in the captured image 1110 may be detected.

In an embodiment, the pedestrian 1130 passing through the specific area may be blurred in the captured image 1110 due to deterioration of weather such as fog, heavy rain, and heavy snow. In this case, the image capturing apparatus may capture an image closer to the specific region or enlarge the specific region than the vehicle 100.

Accordingly, the pedestrian 1130 may be detected from the second reference image 1120 currently photographing the specific region in the image capturing apparatus. Based on this, the image quality may be partially increased to display the pedestrian 1130 clearly on the captured image 1110 or the image 1140 may be output by adding an image effect to an edge.

To this end, an image of the second reference image 1120 may be partially used. That is, the portion of the photographed image 1110 where the pedestrian 1130 passes by may be changed to a portion where the pedestrian 1130 photographed by the second reference image 1120 passes.

12 is a conceptual diagram illustrating an embodiment of outputting a sign on a captured image.

Referring to FIG. 12, a captured image 1210 of photographing a specific region in the current vehicle 100 may be blurred with a quality of a preset level or less.

Accordingly, an external image capturing apparatus may detect a specific object 1230 that is blurred in the captured image 1210 based on the first reference image 1220 in which the specific region is captured with clear image quality.

According to an embodiment, the traffic sign 1230 present in the specific area may be blurred in the captured image 1210 due to deterioration of weather such as fog, heavy rain, heavy snow.

In this case, the image capturing apparatus may generate a second reference image which is closer to the specific area or enlarges the specific area than the vehicle 100. In addition, the first reference image 1220 photographing the specific region may be included (stored) when the weather condition is good.

Accordingly, the sign 1230 may be detected from the first reference image 1220 photographed by the image capturing apparatus.

Based on this, the image quality may be partially increased so that the sign 1230 is clearly displayed on the photographed image 1210, or an image 1240 may be output by adding an image effect to an edge.

Alternatively, the sign 1230 may be changed to an image icon and output. Specifically, the content (text) of the sign 1230 may be included in the sign shape and output.

To this end, an image of the first reference image 1220 may be partially used. That is, the part where the sign 1230 is located in the photographed image 1210 may be changed to the part where the sign 1230 photographed in the first reference image 1220 is located.

Meanwhile, the specific object may include an object moving at a predetermined speed or more.

In an embodiment, another vehicle or object approaching at a very high speed may be detected as the specific object although it is far from the driving road of the vehicle 100.

That is, at present, it does not directly affect the driving of the vehicle 100, but as it approaches at a very high speed (existing in or outside the driving path), an object that is at risk of collision, etc. may be detected as the specific object. have.

FIG. 13 is a conceptual diagram illustrating an embodiment of outputting a two-wheeled vehicle to a captured image.

Referring to FIG. 13, the captured image 1310 photographing a specific region of the current vehicle 100 may be blurred with a quality of a preset level or less.

Accordingly, based on the second reference image 1320 on which the specific region is currently captured by the external image capturing apparatus, the specific object 1330 that is blurred in the captured image 1310 may be detected.

In an embodiment, the motorcycle 1330 passing through the specific region may be blurred in the captured image 1310 due to deterioration of weather such as fog, heavy rain, and heavy snow. In this case, the image capturing apparatus may capture an image closer to the specific region or enlarge the specific region than the vehicle 100.

Accordingly, the two-wheeled vehicle 1330 may be detected from the second reference image 1320 at which the specific image capturing apparatus is currently photographed. Based on this, the image quality may be partially increased or the image effect may be added to an edge so that the two-wheeled vehicle 1330 is clearly seen in the captured image 1310.

To this end, an image of the second reference image 1320 may be partially used. That is, a portion of the photographed image 1310 through which the motorcycle 1330 passes by may be changed to a portion through which the motorcycle 1330 photographed by the second reference image 1320 passes.

In another embodiment, information 1340 related to the two-wheeled vehicle 1330 may be output to the captured image 1310. Specifically, the type of the two-wheeled vehicle 1330 or the distance from the upper vehicle 100 may be output.

11 to 13 illustrate the case where only one of the first reference image and the second reference image is used. However, both reference images may be used to detect a specific object.

The controller 830 may calculate information corresponding to the specific object based on an image corresponding to the specific object included in at least one of the first reference image and the second reference image.

As described above with reference to FIGS. 11 to 13, an image of the second reference image may be partially used. That is, the output portion of the specific object in the captured image may be changed to the output portion of the specific object photographed in the second reference image.

As another embodiment, the specific object may be an object existing only in one of the first reference image and the second reference image.

When the specific object exists only in the first reference image, the controller 830 may control the display unit 830 to output information corresponding to the change.

14 is a conceptual diagram illustrating an embodiment of outputting a change to a captured image.

Referring to FIG. 14, a specific object 1440 exists in the first reference image 1410 captured by the image capturing apparatus at the first time point, but the specific object is included in the second reference image 1420 captured at the second time point. 1440 may not exist.

In an embodiment, the sign 1440 is under construction in the first reference image 1410, but the sign 1440 may not exist in the second reference image 1420.

Accordingly, a message 1450 indicating that construction of the corresponding area is completed may be added to the photographed image 1430 and output. As a result, it is easy for the driver to check changes that are different from the prior art.

The embodiments described above have been described as having two reference images, but the present invention is not limited thereto. That is, two or more reference images may exist. For example, a specific object may be detected based on a plurality of reference images photographing a specific area according to a date.

Referring to the effect of the vehicle control device provided in the vehicle according to the present invention.

According to at least one of the embodiments of the present invention, safe driving may be performed by displaying important specific objects such as pedestrians, motorcycles, signs, and the like on the captured image having low image quality.

In addition, a particular object may be output in various ways, such as an icon, augmented reality, or partly to improve image quality, thereby concentrating the driver's attention.

By displaying the changes on the road, the driver can easily check the updated information.

The present invention described above can be embodied as computer readable codes on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. This also includes implementations in the form of carrier waves (eg, transmission over the Internet). The computer may also include a processor or a controller. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

Claims (10)

A vehicle control apparatus for controlling a vehicle having a sensing unit,
A communication unit receiving a captured image of photographing a specific region from the sensing unit at a first time point, and communicating with an external image photographing apparatus photographing the specific region;
A display unit for outputting the captured image; And
The communication unit may receive the first reference image photographed at the first time point and the second reference image photographed at a second time point different from the first time point when the image quality of the captured image is lower than a reference. And a controller configured to detect a specific object existing in the specific region based on the first and second reference images.
The control unit
Extracting a partial region including the specific object from the second reference image, and controlling the display unit to display the partial region overlaid on the captured image,
The partial region is selectively superimposed on the photographed image according to the image quality of the photographed image. delete delete The method of claim 1,
The second reference image,
Vehicle image control device, characterized in that the least image quality degradation due to an external environmental element of the image photographed by the image capturing device. The method of claim 1,
The specific object is,
Vehicle control apparatus, characterized in that the object present in the driving path of the vehicle. The method of claim 1,
The specific object is,
Vehicle control apparatus, characterized in that the object present in the area of the image quality degradation due to an external environment element in the captured image or more. delete The method of claim 1,
The specific object is,
Vehicle control apparatus comprising an object moving at a predetermined speed or more. The method of claim 1,
The control unit,
And calculating information corresponding to the specific object based on an image corresponding to the specific object included in at least one of the first reference image and the second reference image. delete

Images