WO2014142367A1

WO2014142367A1 - Electronic device and method for controlling same

Info

Publication number: WO2014142367A1
Application number: PCT/KR2013/001999
Authority: WO
Inventors: 김현수; 이중재; 박영경; 박준오; 진주경; 한종우; 이남승; 박덕기; 이제훈; 박수범
Original assignee: 엘지전자 주식회사
Priority date: 2013-03-13
Filing date: 2013-03-13
Publication date: 2014-09-18

Abstract

The present invention relates to an electronic device capable of acquiring an image during driving and a method for controlling the same. The electronic device according to an embodiment of the present invention comprises: a main body; a camera for acquiring an image from the outside during driving of the main body; a memory for storing a first camera parameter as a camera parameter; and a control unit for predicting a vanishing line using a vanishing point detected in the acquired image, for generating a second camera parameter on the basis of the predicted vanishing line, and for comparing the first camera parameter and the second camera parameter, thereby determining whether to update the camera parameter.

Description

Electronic device and its control method

The present invention relates to an electronic device capable of acquiring an image while driving and a control method thereof.

With the rapid development of the information age, the importance of the ability to input and output information and to store data in electronic devices is emphasized. Electronic devices that can have such functions can be divided into portable electronic devices such as terminals, and fixed electronic devices such as video display devices and refrigerators. The portable electronic device may be mounted on a vehicle and display driving information of the vehicle.

The vehicle may be equipped with a camera. The camera may be mounted on at least one of the front part and the rear part of the vehicle, and may acquire an image from the outside. The controller may detect a roadway outer line indicating a lane mark, a shoulder, and the like, an obstacle, another vehicle, a pedestrian, and the like, from the acquired image, and display the detection result on the display unit. The controller may calculate distance information between the front vehicle and the rear vehicle using the acquired image.

However, as the vehicle is inclined, the camera may be inclined, and thus the obtained image may be distorted. For example, as the vehicle is tilted forward, the camera pitches down, and the camera acquires an image in which lanes converge. On the other hand, as the vehicle is inclined backward, the camera pitches up, and the camera acquires an image in which the lane diverges.

An object of the present invention is to provide an electronic device and a control method thereof capable of acquiring an undistorted image even when the vehicle is inclined.

An electronic device according to an embodiment of the present disclosure includes a main body; A camera that acquires an image from the outside while the main body is driving; A memory for storing the first camera parameter as a camera parameter; And predicting a vanishing line using the vanishing point detected in the obtained image, generating a second camera parameter based on the predicted vanishing line, and comparing the first and second camera parameters to update the camera parameter. It includes a control unit for determining whether or not.

In example embodiments, the controller may correct the pitch motion of the camera using the updated camera parameter.

The camera parameter may include an internal parameter and an external parameter, the internal parameter may include a focal length value of the camera, and the external parameter may be an angle on x, y, and z axes of the camera. It may include.

The control unit may detect a vanishing point in the obtained image, predict the vanishing line using the detected vanishing point, and verify the reliability of the predicted vanishing line.

The control unit may extract an image related to the inclination of the lane from the obtained image, extract intersection points of lane candidates from the extracted image, and detect any one of the extracted intersection points as a vanishing point. Can be.

In an embodiment, the controller may correct the detected vanishing point based on statistical information of the extracted intersection points.

The control unit may include: a short term filter for predicting a vanishing line using a vanishing point of each frame; And a long term filter that predicts a vanishing line by using an average of vanishing points of each of a predetermined number of frames.

In example embodiments, the control unit may generate reliability information related to the degree of parallelism of the lane candidates and determine whether to correct the predicted vanishing line based on the reliability information.

In example embodiments, the controller may determine whether the convergence condition of the second camera parameter is satisfied based on at least one of a travel distance, a travel time, and a travel speed.

The control unit may update the second camera parameter to a new camera parameter when the convergence condition is satisfied, and maintain the first camera parameter as the camera parameter when the convergence condition is not satisfied. Can be.

The control unit may store the new camera parameter in the memory when the start of the main body is stopped when the convergence condition is satisfied.

One embodiment of the present invention relates to a control method of an electronic device including a main body. The control method of the electronic device may include loading a first camera parameter stored in a memory as a camera parameter; Acquiring an image from the outside using a camera while driving the main body; Predicting a vanishing line using the vanishing point detected in the acquired image, and generating a second camera parameter based on the predicted vanishing line; And comparing the first and second camera parameters to determine whether to update the camera parameters.

The control method of the electronic device may further include correcting a pitch motion of the camera by using the updated camera parameter.

The method may further include estimating a vanishing line using the vanishing point detected in the acquired image, and generating a second camera parameter based on the predicted vanishing line, detecting the vanishing point in the obtained image. ; Predicting a vanishing line using the detected vanishing point; And verifying reliability of the predicted vanishing line.

The detecting of the vanishing point in the acquired image may include: extracting an image related to a slope of a lane from the obtained image; Extracting intersections of lane candidates from the extracted image; And detecting any one of the extracted intersection points as a vanishing point.

The present invention may predict the vanishing line using the vanishing point of the acquired image and generate a new camera parameter corresponding to the predicted vanishing line. The present invention can correct the pitch motion of a camera using the generated new camera parameter when the generated new camera parameter satisfies the update condition. As a result, the present invention can automatically obtain the undistorted image by automatically correcting the pitch motion of the camera even when the vehicle is inclined.

1 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

2 is a conceptual diagram illustrating a visible region according to a pitch motion of a camera provided in a vehicle.

3 is a conceptual diagram illustrating lanes in world coordinates generated by pitch motion.

4 is a flowchart illustrating a control method of an electronic device according to an embodiment of the present disclosure.

5 is a flowchart illustrating a method of predicting a vanishing line in an acquired image.

6 is a flowchart illustrating a method of detecting a vanishing point in an acquired image.

7 is a conceptual diagram illustrating an algorithm in which a short term filter predicts a disappearance line.

8 is a conceptual diagram illustrating an algorithm in which a long term filter predicts a vanishing line.

9 is a conceptual diagram illustrating an embodiment in which bright points are classified before and after correction of pitch motion.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

A vehicle terminal described herein may mean a terminal installed in a vehicle. For example, a vehicle terminal is not only a fixed terminal embedded in a vehicle, but also a mobile phone, a smartphone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), and a portable multimedia player that can be attached to and detached from a cradle. ), A navigator, etc. may be included.

Referring to FIG. 1, the vehicle terminal 100 may include a wireless communication unit 110, an output unit 120, a user input unit 130, a memory 140, a power supply unit 150, a microphone 170, and a controller 160. It may include. Since the components shown in FIG. 1 are not essential, the vehicle terminal 100 having more or fewer components may be implemented.

Hereinafter, the components 110 to 170 of the vehicle terminal 100 will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the vehicle terminal 100 and the external server 200 or between the vehicle terminal 100 and a network in which the vehicle terminal 100 is located. . For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short range communication module 114, and a GPS receiver 115.

The broadcast receiving module 111 receives a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast related information means information related to a broadcast channel, a broadcast program or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, the broadcast related information may be received by the mobile communication module 112. The broadcast signal and the broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives a wireless signal with at least one of a base station, an external terminal, and an external server 200 on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of voice call signals, video call signals, text messages or multimedia messages. In addition, the mobile communication module 112 may receive information on a predetermined region and content corresponding to the predetermined region from an external server.

The wireless internet module 113 is a module for wireless internet access and may be built in or external to the vehicle terminal 100. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication unit 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, etc. may be used. The vehicle terminal 100 may wirelessly connect a mobile terminal (not shown) located near the vehicle by using the short-range communication unit 114, and perform a call using the mobile terminal (not shown).

The GPS receiver 115 is a module for obtaining a location of the vehicle terminal 100, and a representative example thereof is a satellite navigation device (GPS). The GPS receiver 115 may calculate location information of the vehicle in which the vehicle terminal 100 is installed.

The output unit 120 generates an output related to visual, auditory, tactile, and the like. The output unit 120 may include a display unit 121, a sound output module 122, and an alarm unit 123.

The display unit 121 displays (outputs) information processed by the vehicle terminal 100. For example, when the vehicle terminal 100 operates in the navigation mode, the display 121 displays a user interface (UI) or a graphic user interface (GUI) related to the navigation. When the vehicle terminal 100 operates in the broadcast reception mode, the display 121 displays the received image, the UI, or the GUI.

The display unit 121 may be a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic light emitting diode (OLED), or a flexible display (flexible). The display may include at least one of a 3D display and an e-ink display.

At least one display (or display element) included in the display unit 121 may be configured to be transparent or light transmissive so that the outside can be seen therethrough. This may be referred to as a transparent display. A representative example of such a transparent display is TOLED (Transparant OLED). The display unit 151 may have a light transmissive structure on the windshield of the vehicle. With this structure, the object located in front of the vehicle can be seen through the area occupied by the display 121 in the vehicle. For example, a front glass window for checking the front of the vehicle may be used as the display unit 121.

There may be two or more display units 121 according to the implementation form of the vehicle terminal 100. For example, in the vehicle terminal 100, a plurality of display units may be spaced apart from or integrally located on one surface, or may be located on different surfaces.

The display unit 121 may output a map according to the navigation mode. The GPS receiver 115 may map location information of the vehicle calculated on the map. In addition, the driving information of the vehicle calculated by the controller 160 may be output to the display 121. The driving information may include information related to the driving of the vehicle, such as the speed of the vehicle, the position of the vehicle, the residual amount of oil, and the direction in which the vehicle is headed.

In this case, mapping refers to outputting by overlaying a graphic image or an object on a predetermined position of a 2D or 3D image called a map. For example, the output of a new pop-up window on the map may be regarded as mapping of the pop-up window on the map. Here, the object refers to graphic elements such as an application icon, a widget, a thumbnail image, and the like.

The touch sensor may have the form of a touch film, a touch sheet, a touch pad, or the like. The touch sensor may be configured to convert a pressure applied to a specific portion of the display 121 or a change in capacitance generated at a specific portion of the display 121 into an electrical input signal. The touch sensor may be configured to detect a touch pressure as well as a touched position and area.

When the touch sensor and the display unit 121 have a mutual layer structure, the display unit 121 may be used as an input device in addition to the output device. The display unit 121 may be referred to as a “touch screen”.

If there is a touch input via the touch screen, signals corresponding thereto are sent to a touch controller (not shown). The touch controller processes the signals transmitted from the touch sensor and then transmits data corresponding to the processed signals to the controller 160. As a result, the controller 160 can determine which area of the display 121 is touched.

When the touch screen is capacitive, the touch screen may be configured to detect the proximity of the sensing object by the change of the electric field according to the proximity of the sensing object. The touch screen may be classified as the proximity sensor 141.

The sound output module 122 may output audio data received from the wireless communication unit 110 or stored in the memory 140 in a call signal reception, a call mode or a recording mode, a voice selection mode, a broadcast reception mode, and the like. The sound output module 122 may output a sound signal related to a function (for example, a call signal reception sound, a message reception sound, a navigation alarm, etc.) performed in the vehicle terminal 100. The sound output module 122 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 123 outputs a signal for notifying occurrence of an event of the vehicle terminal 100. Examples of events generated in the vehicle terminal 100 include call signal reception, message reception, key signal input, and touch input. The alarm unit 123 may output a signal for notifying occurrence of an event by vibration, in addition to a video signal or an audio signal. Since the video signal or the audio signal may be output through the display 121 or the sound output module 122, the display 121 and the sound output module 122 may be classified as part of the alarm unit 123.

The user input unit 130 generates input data for the user to control the operation of the vehicle terminal 100. The user input unit 130 may be composed of a key pad, a dome switch, a touch pad (static and electrostatic), a jog wheel, a jog switch, and the like. In addition, the user input unit 130 may detect a touch input for at least one predetermined area output to the display unit 121.

The memory 140 may store a program for the operation of the controller 160 and may temporarily store input and output data (for example, a predetermined area, a still image, a video, etc.). The memory 140 may store data relating to various patterns of vibration and sound output when a touch is input on the touch screen.

The memory 140 may be a flash memory, a hard disk, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), random access memory. RAM, static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk It may include at least one storage medium. The vehicle terminal 100 may operate in association with a web storage that performs a storage function of the memory 140 on the Internet.

The power supply unit 150 receives external power and internal power under the control of the controller 160 to supply power required for the operation of each component.

The microphone 170 processes the sound signal input from the outside in the call mode, the recording mode, the voice selection mode, and the like as electrical voice data. The voice data processed by the microphone 170 in the call mode may be converted into a form transmittable to a mobile terminal wirelessly connected through the local area communication unit 114 and output. The microphone 170 may implement various noise removing algorithms for removing noise generated in the process of inputting an external sound signal.

The controller 160 controls the overall operation of the vehicle terminal 100. For example, control and processing related to receiving a predetermined area, calculating position information, and determining whether to enter a predetermined area is performed. In particular, the controller 160 may store one or more predetermined areas received from the external server 200 in the memory 140. The controller 160 may determine whether the vehicle enters the at least one predetermined region by using the location information calculated by the GPS receiver. As a result of the determination, when entering the predetermined area, content corresponding to the predetermined area may be played.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using software, hardware or a combination thereof.

According to a hardware implementation, the embodiments described herein include Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs). It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. In some cases, the embodiments described herein may be implemented by the controller 160 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code can be implemented in software applications written in the appropriate programming languages. Such software code may be stored in the memory 140 and executed by the controller 160.

2, a vehicle may include an electronic device 100. As illustrated in FIG. 1, the electronic device 100 may include a camera 180, a memory 140, a controller 160, and a display 121.

The camera 180 may be mounted on at least one of the front part and the rear part of the vehicle, and may acquire an image from the outside. The controller 160 may detect a roadway outer line indicating a lane mark, a shoulder, and the like, an obstacle, another vehicle, a pedestrian, and the like, from the acquired image, and display the detection result on the display 121. . In addition, the controller 160 may calculate distance information between the front vehicle and the rear vehicle using the acquired image.

As shown in (a) of FIG. 2, an image acquired by the camera 180 in a state where the vehicle is not inclined has a general viewable area.

On the other hand, as shown in (b) and (c) of Figure 2, the vehicle can be inclined due to the inclination of the road or the weight of the occupants. As the vehicle is tilted, the camera 180 is also tilted, and accordingly, the visible area of the image acquired by the camera 180 may be changed.

For example, as shown in FIG. 2B, as the vehicle is inclined forward, the camera 180 pitches down and the camera 180 acquires an image in which lanes converge. do. On the other hand, as shown in (c) of FIG. 2, as the vehicle is inclined backward, the camera 180 pitches up and the camera 180 acquires an image in which the lane diverges.

That is, the inclination of the vehicle due to the inclination of the roadway or the weight of the occupant generates a pitch motion of the camera 180, thereby causing a change in the visible area.

3A shows a lane image. 3B shows a lane on the world coordinate in a situation where the vehicle is not inclined, that is, there is no pitch motion. 3C shows a lane on the world coordinate in a situation in which the vehicle is inclined forward, that is, in a pitch down state. 3D shows a lane on the world coordinate in a situation in which the vehicle is inclined backward, that is, in a pitched up state.

That is, the information of the world coordination is distorted by the pitch motion. Parallelism of lanes is not guaranteed in the acquired image, and distance information between different vehicles is distorted. Accordingly, when the pitch motion is not corrected, an error may occur in the lane detection or the algorithm for estimating the distance between other vehicles or the pedestrian.

On the other hand, according to the conventional technology of estimating the vanishing point to correct the pitch motion, there is a problem that the performance of correcting the pitch motion may be degraded due to an error of the vanishing point estimation due to noise.

Accordingly, hereinafter, the electronic device 100 stably estimates the vanishing point and automatically corrects the pitch motion of the camera 180 according to the driving distance, the traveling speed, and the traveling time at the time of driving of the vehicle without additional driver's operation. It will be described with reference to the accompanying drawings.

4 is a flowchart illustrating a control method of the electronic device 100 according to an embodiment of the present disclosure. As illustrated in FIG. 1, the electronic device 100 may include a camera 180, a memory 140, a controller 160, and a display 121.

Referring to FIG. 4, first, a step (S110) of loading a first camera parameter stored in the memory 140 as a camera parameter is performed.

In detail, the controller 160 may load the first camera parameter as the initial camera parameter when the vehicle starts up. In this case, the first camera parameter means the most recently updated camera parameter. The controller 160 may initialize camera parameters.

The camera parameter refers to a conversion constant for acquiring world coordinate information from an image acquired by the camera 180. The camera parameter may include at least one of an internal parameter and an external parameter. Herein, the internal parameter includes a focal length value and a center value of the camera 180, and the external parameter includes an angle on the x, y and z axes of the camera 180.

Next, in operation S120, an image is acquired from the outside using the camera 180 while the vehicle is driving.

As described above, the camera 180 may be mounted on at least one of the front part and the rear part of the vehicle, and may acquire an image from the outside.

Thereafter, the vanishing line is detected using the vanishing point detected in the acquired image, and a second camera parameter is generated based on the predicted vanishing line (S130).

In detail, the controller 160 may extract an input value for performing online calibration based on the acquired image. The controller 160 may perform online calibration to predict the vanishing line and generate a second camera parameter based on the predicted vanishing line.

Thereafter, step S140 is performed in which it is determined whether to update the camera parameters by comparing the first and second camera parameters.

The controller 160 may inspect the update condition of the second camera parameter based on at least one of the travel distance, the travel time, and the travel speed. Here, when the second camera parameter satisfies the convergence condition, the controller 160 may update the second camera parameter with the new camera parameter. In this case, the controller 160 may store the new camera parameter in the memory 140 when the vehicle is stopped. In addition, the controller 160 may correct the pitch motion of the camera 180 by using the new camera parameter.

On the other hand, when the second camera parameter does not satisfy the convergence condition, the controller 160 may continuously maintain the first camera parameter as the camera parameter.

As described above, the present invention may predict the vanishing line by using the vanishing point of the acquired image, and generate a new camera parameter corresponding to the predicted vanishing line. The present invention can correct the pitch motion of the camera 180 by using the generated new camera parameter when the generated new camera parameter satisfies the update condition. As a result, according to the present invention, even when the vehicle is inclined, the pitch motion of the camera 180 is automatically corrected, thereby obtaining an undistorted image.

5 is a flowchart illustrating a method of predicting a vanishing line in an acquired image. 6 is a flowchart illustrating a method of detecting a vanishing point in an acquired image. As illustrated in FIG. 1, the electronic device 100 may include a camera 180, a memory 140, a controller 160, and a display 121.

Referring to FIG. 5, first, a vanishing point is detected in an image acquired by the camera 180 (S210). This will be described in more detail with reference to FIG. 6.

Referring to FIG. 6, the controller 160 may extract an image related to the inclination of the lane from the obtained image (S310). The controller 160 may find lane candidates in the extracted image (S320), extract intersections of the lane candidates (S330), and detect any one of the extracted intersection points as a vanishing point (S340).

For example, the controller 160 may display points on a portion corresponding to the lane in the acquired image. The controller 160 may extract an intersection point when a straight line connecting the points is drawn. The controller 160 may detect, as the vanishing point, the point at which the most intersection points are displayed based on the extracted statistical information of the intersection points.

Returning to FIG. 5, step S220 is followed in which the vanishing point is tracked.

The controller 160 may acquire the statistics of the vanishing point for a predetermined period while tracking the vanishing point. By continuing to track the vanishing point, the controller 160 can correct the vanishing point found in the wrong instant. Accordingly, the controller 160 may robustly process noise.

Subsequently, a step (S230) in which a vanishing line is predicted proceeds.

The controller 160 includes a short term filter and a long term filter. The short term filter may predict the vanishing line using the vanishing point of each frame. Meanwhile, the long term filter may calculate an average of vanishing points of each of a predetermined number of frames, and predict the vanishing line using the average.

The short term filter may be used to correct for pitch motion that is changed instantaneously by curvature of the roadway. In addition, the long term filter may be used to correct the pitch motion changed as the vehicle body is tilted by the occupant.

Thereafter, step S240 is performed in which reliability of the vanishing line is verified.

The controller 160 may generate reliability information related to the degree of parallelism of the lane candidates, and determine whether to correct the predicted vanishing line based on the reliability information.

Reliability information may be defined to the degree of parallelism of the lane candidates. The reliability information can be calculated using the vector dot product. The controller 160 may correct the predicted vanishing line based on the reliability information, and may update camera parameters.

As described above, the present invention may predict the vanishing line using the corrected vanishing point instead of directly using the vanishing point to predict the vanishing line. In addition, the present invention can predict the vanishing line using both the short term filter and the long term filter.

In the algorithm of FIG. 7, the first portion P1 refers to a portion that predicts the vanishing line with the detected vanishing point, and the second portion P2 converges to the reference vanishing line VL _REF when the detection of the vanishing point fails. It means the part to do.

7 (a) is y (VP), which means the y coordinate of the vanishing point. FIG. 7B is a flow weight, and means the influence of the currently detected vanishing point on predicting the short term vanishing line VL _SHORT . 7 (c) is an inertia weight, and means the influence of the update amount ΔVL _SHORT of the short term vanishing line predicted in the previous frame.

Referring to the first portion P1 of FIG. 7, an average value of the flow weight and the inertial weight may be used to predict the short term vanishing line VL _SHORT . That is, the flow weight and the inertial weight can buffer each other.

FIG. 7D is a compensation weight, which means the speed of convergence to the reference vanishing line VL _REF when the detection of the vanishing point fails. Here, the reference vanishing line VL _REF is a value associated with the first camera parameter in FIG. 4. Accordingly, when the detection of the vanishing point fails, the controller 160 can use the first camera parameter, which is the most recent camera parameter.

FIG. 7E means the update amount ΔVL _{SHORT of} the short term vanishing line predicted by the short term filter. Specifically, the update amount ΔVL _SHORT of the short term vanishing line means a difference value between the most recent vanishing line and the vanishing line predicted by the current short term filter. The update amount ΔVL _{SHORT of the} short term vanishing line may be delayed D to the previous frame and then multiplied by the inertia weight.

7 (f) means the reference vanishing line VL _REF . As described above, the reference vanishing line VL _REF is a value associated with the first camera parameter in FIG. 4.

FIG. 7G illustrates a final short term vanishing line VL _SHORT as a final value.

Referring to FIG. 8, the long term filter obtains an average of vanishing points of each of the latest n frames (eg, 300 frames), and predicts a vanishing line using the average. In this case, vanishing point data of each of the last n frames may be stored in the memory 140.

Referring to FIG. 9A, objects are not easily classified by bumps or the like before the pitch motion is corrected. As shown, since the control unit 160 only uses color information and brightness information for classification of objects, the

objects

302 and 303 are overlapped based on the vanishing line (horizontal line) 301. 302, 303) was not easy to distinguish between the front vehicle and other objects.

However, referring to FIG. 9B, after the pitch motion is corrected, the

objects

305 and 306 may be easily classified based on the vanishing line 304. As shown, the controller 160 controls the front vehicle 305 located above the vanishing line 304 and the streetlight 306 located above the front lane 305 based on the vanishing line 304. It can be detected easily.

Meanwhile, when calculating a vehicle, the total calculation time may be determined by setting a region of interest (ROI) for vehicle search. At this time, according to the present invention, it is possible to reduce the calculation time by setting the ROI to be adjustable by using the updated camera parameter (homography) through the predicted vanishing line.

In addition, the lane detection algorithm using the camera homography performance is determined by the accuracy of the camera homography. Therefore, according to the present invention which compensates for the inaccuracy of camera parameters due to the pitch motion change of the camera 180, the accuracy of the algorithm can be increased.

According to one embodiment disclosed in the present specification, the above-described method may be implemented in processor-readable code on a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may be implemented in the form of a carrier wave (for example, transmission over the Internet). Include.

In the apparatus for measuring a biosignal of a vehicle driver disclosed herein, the configuration and method of the above-described embodiments may not be limitedly applied, but all or a part of the embodiments may be selectively combined so that various modifications may be made. It may be.

Claims

main body;

A camera that acquires an image from the outside while the main body is driving;

A memory for storing the first camera parameter as a camera parameter; And

A vanishing line is predicted using vanishing points detected in the acquired image, and a second camera parameter is generated based on the predicted vanishing line.

And a controller configured to compare the first and second camera parameters to determine whether to update the camera parameters.
The method of claim 1,

The control unit,

And correct the pitch motion of the camera by using the updated camera parameter.
The method of claim 2,

The camera parameter is,

Including internal and external parameters,

The internal parameter is

A focal length value of the camera,

The external parameter is,

And an x, y and z axis angle of the camera.
The method of claim 1,

The control unit,

Detecting a vanishing point from the acquired image,

Using the detected vanishing point to predict a vanishing line,

And verifying the reliability of the predicted vanishing line.
The method of claim 4, wherein

The control unit,

And extracting an image related to the inclination of the lane from the obtained image, extracting intersections of lane candidates from the extracted image, and detecting any one of the extracted intersections as a vanishing point.
The method of claim 5,

The control unit,

And correct the detected vanishing point based on statistical information of the extracted intersection points.
The method of claim 6,

The control unit,

A short term filter for predicting a vanishing line using the vanishing point of each frame; And

And a long term filter for predicting a vanishing line by using an average of vanishing points of each of a predetermined number of frames.
The method of claim 7, wherein

The control unit,

And generating reliability information related to the degree of parallelism of the lane candidates and determining whether to correct the predicted vanishing line based on the reliability information.
The method of claim 1,

The control unit,

And checking whether the convergence condition of the second camera parameter is satisfied based on at least one of a travel distance, a travel time, and a travel speed.
The method of claim 9,

The control unit,

If the convergence condition is satisfied, updating the second camera parameter with a new camera parameter,

And when the convergence condition is not satisfied, maintaining the first camera parameter as the camera parameter.
The method of claim 10,

The control unit,

And when the convergence condition is satisfied, storing the new camera parameter in the memory when the start of the main body is stopped.
In the control method of an electronic device including a main body:

Loading a first camera parameter stored in a memory as a camera parameter;

Acquiring an image from the outside using a camera while driving the main body;

Predicting a vanishing line using the vanishing point detected in the acquired image, and generating a second camera parameter based on the predicted vanishing line; And

Comparing the first and second camera parameters to determine whether to update the camera parameters.
The method of claim 12,

And correcting the pitch motion of the camera using the updated camera parameter.
The method of claim 12,

A method of predicting a vanishing line using the vanishing point detected in the acquired image, and generating a second camera parameter based on the predicted vanishing line,

Detecting a vanishing point in the acquired image;

Predicting a vanishing line using the detected vanishing point; And

And verifying reliability of the predicted vanishing line.
The method of claim 14,

Detecting a vanishing point in the obtained image,

Extracting an image related to the inclination of the lane from the obtained image;

Extracting intersections of lane candidates from the extracted image; And

And detecting any one of the extracted intersection points as a vanishing point.