KR20200139312A - Vehicle and control method thereof - Google Patents

Abstract

The present technology provides a vehicle and a control method thereof, wherein an image around a vehicle is obtained before opening a vehicle door and is provided to a user after opening the vehicle door so that the user can safely get off the vehicle by recognizing an environment around the vehicle even when the vehicle door is opened. The vehicle according to an embodiment includes: a plurality of cameras provided in the vehicle; an output unit; and a control unit for outputting a synthesized image obtained by synthesizing images around the vehicle each obtained by the plurality of cameras to the output unit when a vehicle speed is less than a predetermined speed.

Description

Vehicle and its control method {VEHICLE AND CONTROL METHOD THEREOF}

The present invention is a technology related to a vehicle and a control method thereof that provides information for a user to safely get off the vehicle.

Surround View Monitor is a function that creates a top view screen on the navigation screen that looks as if the car was taken from above by using four cameras, one at the front and rear of the vehicle and at the bottom of the left and right side mirrors. Using such a surround view monitoring system, the driver can grasp the situation around the vehicle at a glance and park safely or traverse a narrow road.

Meanwhile, the vehicle may acquire an image around the vehicle used for the above-described surround view, output it on a display, and provide it to a user.

In particular, when a display is provided in the rear seat of the vehicle, the vehicle may output the acquired image on the display provided in the rear seat. However, when the door on the side of the vehicle is opened, the viewing angle of the camera located on the side door of the vehicle is changed, so that it is difficult to easily acquire an image from the side of the vehicle.

The present technology provides a vehicle and a control method for the vehicle by acquiring an image around the vehicle before opening the vehicle door and providing it to the user after opening the vehicle door so that the user can safely get off the vehicle by recognizing the environment around the vehicle even when the vehicle door is opened.

A vehicle according to an embodiment includes a plurality of cameras provided in the vehicle; Output section; And a controller configured to output, to the output unit, a composite image obtained by synthesizing the images around the vehicle each obtained by the plurality of cameras when the vehicle speed is less than a predetermined speed.

The vehicle according to an embodiment further includes at least one door provided in the vehicle, wherein the control unit includes an image acquired at a predetermined time before opening of the at least one door and the plurality of cameras respectively. Images around the vehicle may be synthesized and output to the output unit.

When the side door of the vehicle is opened, the control unit combines the image of the rear camera provided in the vehicle and the side image of the vehicle acquired at a time prior to opening of the at least one door and outputs it to the output unit. I can.

The plurality of cameras acquire location information of at least one object located around the vehicle,

The controller may display the location information on the synthesized image and output it to the output unit.

When it is determined that the at least one door is opened and the at least one object approaches the vehicle, the controller may output a warning signal to at least a part of the composite image.

The controller may display a reference line including location information of the vehicle on the composite image.

The controller may derive a top view image of the vehicle including an opening trajectory of at least one door provided in the vehicle and output it to the output unit.

A vehicle control method according to an exemplary embodiment acquires a vehicle speed, and when the vehicle speed is less than a predetermined speed, synthesizes images around the vehicle each obtained by the plurality of cameras, and outputs the synthesized image. Includes printing to the negative.

Synthesizing the image around the vehicle may include synthesizing an image acquired at a time before the opening of the at least one door and an image around the vehicle each acquired by the plurality of cameras.

Synthesizing the image around the vehicle includes: when the side door of the vehicle is opened, an image of a rear camera provided in the vehicle and a side image of the vehicle acquired at a predetermined time before opening of the at least one door are synthesized. May include.

The vehicle control method according to an embodiment further includes obtaining location information of at least one object located around the vehicle,

Outputting the synthesized image to an output unit may include displaying the location information on the synthesized image and outputting the position information to the output unit.

Outputting the composite image to an output unit includes opening the at least one door,

When it is determined that the at least one object approaches the vehicle, outputting a warning signal to at least a part of the composite image may be included.

The vehicle control method according to an embodiment may further include displaying a reference line including location information of the vehicle on the synthesized image.

The vehicle control method according to an exemplary embodiment may further include deriving a top view image of the vehicle including an opening trajectory of at least one door provided in the vehicle and outputting it to the output unit.

In the vehicle and its control method according to an exemplary embodiment, an image around a vehicle is acquired before the vehicle door is opened, and is provided to the user after the door is opened, so that the user can safely get off the vehicle by recognizing the environment around the vehicle even when the vehicle door is opened.

1 is an exterior view of a vehicle equipped with a camera according to an exemplary embodiment.
2 is a diagram showing a control block diagram according to an embodiment.
3 is a diagram for explaining a composite image output when a door is not opened according to an exemplary embodiment.
4 is a diagram illustrating a composite image output when a door is opened, according to an exemplary embodiment.
5 is a graph illustrating an operation of determining a time before opening of at least one predetermined door according to an exemplary embodiment.
6 is a view for explaining an operation of outputting a top view image of the vehicle including an opening trajectory of at least one door provided in the vehicle.
7 is a diagram showing a flow chart according to an embodiment.

The same reference numerals refer to the same elements throughout the specification. The present specification does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the present invention belongs will be omitted. The term'unit, module, member, block' used in the specification may be implemented as software or hardware, and according to embodiments, a plurality of'units, modules, members, blocks' may be implemented as one component, It is also possible for one'unit, module, member, block' to include a plurality of components.

Throughout the specification, when a part is said to be "connected" with another part, this includes not only the case of being directly connected, but also the case of indirect connection, and the indirect connection includes connection through a wireless communication network. do.

In addition, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

Throughout the specification, when a member is said to be positioned "on" another member, this includes not only the case where a member is in contact with another member, but also the case where another member exists between the two members.

Terms such as first and second are used to distinguish one component from other components, and the component is not limited by the above-described terms.

Singular expressions include plural expressions, unless the context clearly has exceptions.

In each step, the identification code is used for convenience of explanation, and the identification code does not describe the order of each step, and each step may be implemented differently from the specified order unless a specific sequence is clearly stated in the context. have.

Hereinafter, the operating principle and embodiments of the present invention will be described with reference to the accompanying drawings.

1 is an exterior view of a vehicle equipped with a camera according to an exemplary embodiment.

Referring to Figure 1, the exterior of the vehicle 1 according to an embodiment is a wheel for moving the vehicle 1, a door 15L shielding the interior of the vehicle 1 from the outside, and the driver inside the vehicle 1 It includes a windshield 16 providing a view of the front of the vehicle 1 and side mirrors 14L and 14R providing a view of the rear of the vehicle 1 to the driver.

The wheel includes a front wheel provided in the front of the vehicle and a rear wheel provided in the rear of the vehicle, and the driving device (not shown) provided inside the vehicle 1 is a front wheel or a rear wheel so that the vehicle 1 moves forward or backward. Provides rotational force to Such a driving device may employ an engine that generates rotational force by burning fossil fuels or a motor that generates rotational force by receiving power from a capacitor.

The door is rotatably provided on the left and right sides of the vehicle 1 so that a driver or a passenger can ride inside the vehicle 1 when opened, and shields the interior of the vehicle 1 from the outside when closed. In addition, a handle capable of opening and closing the doors 15L and 15R may be provided outside the vehicle 1, and an LF antenna (not shown) capable of transmitting a low frequency (LF) signal may be mounted on the handle. .

The windshield 16 is provided on the upper front side of the main body to enable the driver inside the vehicle 1 to acquire visual information in front of the vehicle 1, and is also referred to as windshield glass.

In addition, the side mirrors 14L and 14R include a left side mirror 14L provided on the left side of the vehicle 1 and a right side mirror 14R provided on the right side, and the driver inside the vehicle 1 1) It is possible to acquire visual information of the side and rear.

In addition, the vehicle 1 may include a sensing device such as a proximity sensor that detects obstacles at the rear or side or another vehicle, and a rain sensor that detects whether or not precipitation and precipitation.

The proximity sensor may transmit a detection signal to the side or rear of the vehicle and receive a reflected signal reflected from an obstacle such as another vehicle. Based on the waveform of the received reflected signal, the presence or absence of an obstacle on the side or rear of the vehicle 1 may be detected, and the position of the obstacle may be detected. As an example of such a proximity sensor, a method of transmitting ultrasonic waves or infrared rays and detecting a distance to an obstacle using ultrasonic waves or infrared rays reflected by the obstacle may be employed.

In addition, the vehicle may include cameras 101-1, 101-2, 101-3, and 101-4. Cameras 101-1, 101-2, 101-3, and 101-4 that acquire images around the vehicle may be provided in the front, rear, and sides of the vehicle to acquire images, and each camera 101-1, 101 The images acquired by -2, 101-3, 101-4) may be derived as images around the vehicle through a method described later.

2 is a diagram showing a control block diagram according to an embodiment.

A vehicle according to an embodiment includes a door 104, a plurality of cameras 101, an output unit 103, and a control unit 102.

The door 104 is provided in the vehicle as described above and is rotatably provided on the left and right sides of the vehicle 1 so that a driver or a passenger can ride inside the vehicle 1 when opened, and when closed, the vehicle ( Shield the inside of 1) from the outside. In addition, a handle capable of opening and closing the door 104 may be provided outside the vehicle 1, and an LF antenna (not shown) capable of transmitting a low frequency (LF) signal may be mounted on the handle.

The plurality of cameras 101 may capture front, side, and rear images of the vehicle and transmit the captured images to the controller 102. According to an embodiment, the camera 101 may be provided in the side mirrors 14R and 14L provided in the vehicle 1 to acquire an image around the vehicle 1.

The camera 101 installed in the vehicle 1 may include a Charge-Coupled Device (CCD) camera 101 or a CMOS color image sensor. Here, both CCD and CMOS refer to sensors that convert light that has entered through the lens of the camera 101 into electrical signals and store them. Specifically, the CCD (Charge-Coupled Device) camera 101 is a device that converts an image into an electric signal using a charge-coupled device. In addition, the CIS (CMOS Image Sensor) refers to a low-consumption, low-power type imaging device having a CMOS structure, and serves as an electronic film for digital devices. In general, CCDs have higher sensitivity than CIS and are widely used in vehicle 1, but are not limited thereto.

The output unit 103 may be implemented in the form of a display provided in the rear seat of the vehicle.

The output unit 103 is a cathode ray tube (CRT), a digital light processing (DLP) panel, a plasma display panel, a liquid crystal display (LCD) panel, and electroluminescence. (Electro Luminescence: EL) panel, Electrophoretic Display (EPD) panel, Electrochromic Display (ECD) panel, Light Emitting Diode (LED) panel, or Organic Light Emitting Diode: OLED) panel, etc. may be provided, but is not limited thereto.

If the vehicle speed is less than a predetermined speed, the controller 102 may output a composite image obtained by synthesizing the images around the vehicle each obtained by a plurality of cameras to the output unit 103.

Specifically, if the vehicle is less than a predetermined speed, such as when the vehicle is stopped, the vehicle may synthesize the image around the vehicle obtained by the rear camera and the side image obtained by the side camera and output it to the output unit 103 provided in the rear seat of the vehicle.

The controller 102 may synthesize an image acquired at a time before opening of the at least one door and an image around the vehicle each acquired by a plurality of cameras, and output to the output unit 103.

That is, when the door of the vehicle is opened, it is difficult for the camera provided on the side of the vehicle to acquire the vehicle side image, so that the vehicle side image acquired at the time before the door is opened and the vehicle rear image acquired in real time can be synthesized and output. have.

When the side door of the vehicle is opened, the controller 102 synthesizes the image of the rear camera provided on the vehicle and the side image of the vehicle acquired at a time before opening of the at least one door, and outputs it to the output unit 103. I can.

A plurality of cameras in the vehicle may acquire location information of at least one object located around the vehicle. That is, the controller 102 may acquire location information of an object around the vehicle based on the image acquired by the camera 101, but may obtain location information of the object around the vehicle based on a sensor such as an ultrasonic sensor. .

The controller 102 may display the location information on the synthesized image and output it to the output unit 103. In particular, the controller 102 may display the location information of the composite image when the vehicle cannot acquire the side image because the side door of the vehicle is opened, and the vehicle approaching the vehicle disappears from the rear image of the vehicle. Meanwhile, the location information may include information including a positional relationship between an object and a vehicle, but there is no restriction on location information such as outputting a warning message when an object approaches the vehicle other than specific information.

Specifically, when it is determined that at least one door is opened and at least one object approaches the vehicle, the controller 102 may output a warning signal to at least a part of the composite image.

The controller 102 may display a reference line including vehicle location information on the composite image. The reference line may indicate a ground reference distance value where the vehicle is currently located. In one embodiment, the reference line may mean an extension line of a point where the wheel of the vehicle and the ground contact each other.

The controller 102 may derive a top view image of the vehicle including an opening trajectory of at least one door 104 provided in the vehicle and output it to the output unit 103.

The top view image of the vehicle 102 may be derived by synthesizing the vehicle surrounding images acquired by a plurality of cameras provided around the vehicle.

The door trajectory may mean a motion trajectory of a side door provided in the vehicle.

The control unit 102 includes a memory (not shown) that stores data for an algorithm for controlling the operation of components in the vehicle or a program that reproduces the algorithm, and a processor that performs the above-described operation using data stored in the memory ( (Not shown) can be implemented. In this case, the memory and the processor may be implemented as separate chips, respectively. Alternatively, the memory and processor may be implemented as a single chip.

At least one component may be added or deleted corresponding to the performance of the components of the vehicle illustrated in FIG. 2. In addition, it will be readily understood by those of ordinary skill in the art that the mutual positions of the components may be changed corresponding to the performance or structure of the system.

Meanwhile, each of the components shown in FIG. 2 refers to software and/or hardware components such as Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC).

3 is a diagram for explaining a composite image output when a door is not opened according to an exemplary embodiment.

Referring to FIG. 3, when the speed of the vehicle is less than a predetermined speed, a composite image obtained by synthesizing the images around the vehicle each obtained by the plurality of cameras may be output to an output unit.

Specifically, the camera provided in the vehicle may obtain a side image VS3 of the vehicle and a rear image VB3 of the vehicle, and output a synthesized image derived by synthesizing the images respectively.

In synthesizing each image, image conversion and synthesis techniques can be applied.

Specifically, the controller may define a region of interest (ROI) region for displaying by minimizing distortion of an obstacle in the rear region and the front region. In addition, the control unit may secure the verticality of the distortion-converted vertical object and the amount of horizon view so that a certain area after the horizon can be displayed from the bottom of the rear image. The controller may output a synthesized image based on a gear signal input by the user for low speed. In addition, if the door has not yet been opened, the camera provided on the side of the vehicle can acquire a side image of the vehicle, so the vehicle can synthesize real-time images of the side and rear of the vehicle and output it to the output unit provided in the rear seat of the vehicle. .

In FIG. 3, when the door is not opened, the controller may output the side image VS3 and the object O3 included in the rear image VB3 of the vehicle in real time.

Meanwhile, the operation of synthesizing and outputting the side image of the vehicle and the rear image of the vehicle described in FIG. 3 is only one embodiment of the present invention, and the operation of synthesizing and outputting the side and rear images when the vehicle door is not opened. There is no limit.

4 is a diagram illustrating a composite image output when a door is opened, according to an exemplary embodiment.

Referring to FIG. 4, the controller may combine an image acquired at a time before opening of at least one predetermined door and an image around the vehicle each acquired by a plurality of cameras and output to the output unit.

Specifically, when the side door of the vehicle is opened, the image VB4 of the rear camera provided in the vehicle and the lateral image VS4 of the vehicle acquired at a time before opening of the at least one door are combined, and the output unit 103 ) Can be printed. A detailed description of the predetermined time before opening will be described later.

When the door is opened, the lateral camera provided in the vehicle cannot acquire the lateral image of the vehicle. Therefore, the control unit uses the vehicle surrounding image (VS4) acquired before the vehicle door is opened and the vehicle rear image (VB4) acquired in real time. A composite image may be generated based on) and output to the output unit 103.

Meanwhile, in this case, the controller may obtain location information of at least one object located around the vehicle, display the location information on the composite image, and output it to the output unit 103.

4 shows an operation of outputting the location information M4 around the vehicle to the output unit 103 in the form of a warning signal such as "an obstacle is approaching".

Referring to FIG. 4, an object O4 located at the rear of the vehicle and around the approaching vehicle is shown. Meanwhile, when the object O4 around the vehicle is included in the rear image of the vehicle, the vehicle may include the object in the composite image and output it. On the other hand, when an object is not included in the rear area of the vehicle and the door is opened, the shape of the object cannot be output to the composite image in real time, but position information of the object can be output as shown in FIG. 4.

In addition, the reference line including the location information of the vehicle of the approaching obstacle in the image is applied to the pixel position value (image Coordinates) and output to the output unit.

5 is a graph illustrating an operation of determining a time before opening of at least one predetermined door according to an exemplary embodiment.

Referring to FIG. 5, the x-axis represents time and the y-axis represents a location where an image acquired by a vehicle is stored.

t1 means the time when the vehicle starts to drive at low speed, t2 means the time when the vehicle stops, and t3 means the time when the door is opened.

According to an exemplary embodiment, from t1 to t2, it may mean a situation in which a Surround View Monitor (SVM) is turned on and the vehicle is running at 15 kph or less.

Meanwhile, e1 means that the SVM is turned on and the image is stored in the buffer.

e2 means that when the vehicle is stopped, the first image is stored in the memory at t2.

e3 is a point in time after the door is opened and indicates an operation to determine the output of the composite image. Specifically, when the difference between t3 and t2 does not exceed the storage range of the above-described buffer, the controller may derive a composite image based on the first image stored in the memory. Meanwhile, when the difference between t3 and t2 exceeds the storage range of the above-described buffer, a composite image may be derived based on the image at the time of closing the door.

Meanwhile, the operation of determining a time before opening of at least one predetermined door described in FIG. 5 is only an embodiment of the present invention, and a time before opening of at least one predetermined door is determined based on factors other than the size of the memory. Of course you can.

6 is a view for explaining an operation of outputting a top view image of the vehicle including an opening trajectory of at least one door provided in the vehicle.

The controller may derive a top view image of the vehicle including an opening trajectory of at least one door provided in the vehicle and output it to the output unit.

As described above, the control unit can derive a top view image (VT6) based on the surrounding image acquired by the camera provided in the vehicle, and the door opening trajectory in consideration of the predetermined size of the vehicle door and the rotation angle of the door. (Z6L, Z6R) can be determined.

Meanwhile, the user can prevent damage to the vehicle when getting off the vehicle based on the door opening trajectories Z6L and Z6R displayed on the output unit.

7 is a diagram showing a flow chart according to an embodiment.

Referring to FIG. 7, if the vehicle speed is less than a predetermined speed 1001, the vehicle may acquire 1002 lateral and rear images acquired by a camera provided in the vehicle, and output a composite image based thereon (1003). ).

However, when the door of the vehicle is opened (1004), an image synthesized based on the image acquired before the opening time may be output to the output unit (1005).

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing instructions executable by a computer. The instruction may be stored in the form of a program code, and when executed by a processor, a program module may be generated to perform the operation of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all kinds of recording media in which instructions that can be read by a computer are stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like.

As described above, the disclosed embodiments have been described with reference to the accompanying drawings. A person of ordinary skill in the art to which the present invention pertains, without changing the technical spirit or essential features of the present invention, has different forms from the disclosed embodiments. It will be appreciated that the present invention may be practiced. The disclosed embodiments are illustrative and should not be construed as limiting.

1: vehicle
101: camera
102: control unit
103: output unit
104: door

Claims (14)

A plurality of cameras provided in the vehicle;
Output section; And
And a controller configured to output a synthesized image obtained by synthesizing images around the vehicle each obtained by the plurality of cameras to the output unit when the vehicle speed is less than a predetermined speed.
The method of claim 1,
At least one door provided in the vehicle; further includes,
The control unit,
A vehicle that synthesizes an image acquired at a time before opening of the at least one door and an image around the vehicle each acquired by the plurality of cameras, and outputs it to the output unit.
The method of claim 2,
The control unit,
When the side door of the vehicle is opened,
A vehicle for synthesizing an image of a rear camera provided in the vehicle and a side image of the vehicle acquired at a predetermined time before opening of the at least one door, and outputting it to the output unit.
The method of claim 2,
The plurality of cameras,
Obtaining location information of at least one object located around the vehicle,
The control unit,
A vehicle that displays the location information on the composite image and outputs it to the output unit.
The method of claim 4,
The control unit,
The at least one door is open,
When it is determined that the at least one object approaches the vehicle,
A vehicle that outputs a warning signal to at least a part of the composite image.
The method of claim 2,
The control unit,
A vehicle displaying a reference line including location information of the vehicle in the composite image.
The method of claim 2,
The control unit,
A vehicle that derives a top view image of the vehicle including an opening trajectory of at least one door provided in the vehicle and outputs it to the output unit.
Get the vehicle's speed,
If the speed of the vehicle is less than the predetermined speed, the plurality of cameras synthesize images around the vehicle each obtained,
And outputting the synthesized image to an output unit.
The method of claim 8,
Synthesizing the image around the vehicle,
And synthesizing an image acquired at a time before opening of the at least one door and an image around the vehicle each acquired by the plurality of cameras.
The method of claim 9,
Synthesizing the image around the vehicle,
When the side door of the vehicle is opened,
And synthesizing an image of a rear camera provided in the vehicle and a side image of the vehicle acquired at a predetermined time before opening of the at least one door.
The method of claim 9,
Further comprising obtaining location information of at least one object located around the vehicle,
Outputting the composite image to an output unit,
And displaying the location information on the synthesized image and outputting the position information to the output unit.
The method of claim 11,
Outputting the composite image to an output unit,
The at least one door is open,
When it is determined that the at least one object approaches the vehicle,
And outputting a warning signal to at least a part of the synthesized image.
The method of claim 9,
The vehicle control method further comprising displaying a reference line including location information of the vehicle on the composite image.
The method of claim 9,
The vehicle control method further comprising deriving a top view image of the vehicle including an opening trajectory of at least one door provided in the vehicle and outputting it to the output unit.

Images