KR20200084470A - Intelligent side view camera system - Google Patents

Abstract

Disclosed is an intelligent side view camera system for a vehicle. The intelligent side view camera system for a vehicle comprises: a left image acquisition unit which supports real-time fast image processing and ultralow-light high-definition image acquisition and is installed on the left side of a vehicle to acquire a left rear image of the vehicle; a right image acquisition unit which supports real-time fast image processing and ultralow-light high-definition image acquisition and is installed on the right side of the vehicle to acquire a right rear image of the vehicle; an image recognition and processing unit to perform ultralow-light environment image noise removal of the left rear image and the right rear image of the vehicle and lens distortion correction; a control unit to control optical band backlight correction when a backlight condition is detected, control image quality improvement in accordance with a lighting or a climate condition, and generate an event signal in accordance with vehicle surrounding detection; an event processing unit to process an occurrence of an event based on an event signal in accordance with control of the control unit; and a display unit to display a left image and a right image outputted by the image recognition and processing unit.

Description

Vehicle intelligent side view camera system {INTELLIGENT SIDE VIEW CAMERA SYSTEM}

The present invention relates to a side view camera system for a vehicle that can replace the side mirror of a vehicle, and to an intelligent side view camera system that can be applied to an autonomous vehicle that is a core technology of the fourth industrial revolution.

Both sides of the vehicle are equipped with side view mirrors so that the driver can recognize the rear side of the vehicle.

The side-view mirror has a problem that a blind spot may exist, and is expected to be replaced by a camera and a video display device having various advantages and purposes for providing a better view to the driver.

The vehicle side view system, which consists of a camera and a video display device, aims to prevent accidents by securing safety by providing a better view to the driver, and additionally provides energy savings at the same time, and is required for the Advanced Driver Assistance System (ADAS). It can be used as a camera system.

Prior art 1 to prior art 5 provide various techniques for providing blind spot information using a camera and a video display device or replacing the existing side view mirror.

However, the apparatus and method disclosed in prior art 1 to prior art 5 simply present a technique for replacing the existing side view mirror, and are an intelligent system that can be applied to autonomous vehicles or smart cars, which are the core technologies of the fourth industrial revolution. However, it does not provide convenience and does not provide a configuration to operate intelligently and adaptively in various driving environments.

Prior Patent 1: Korean Patent Registration No. 10-0650126 (Invention name: side mirror device using digital camera) Prior Patent 2: Korean Registered Patent No. 10-1486670 (Invention name: digital camera side mirror) Prior Patent 3: Korean Registered Patent No. 10-1828154 (Invention name: Camera type side mirror of a vehicle) Prior Patent 4: Korean Registered Patent No. 10-1449160 (Invention name: Vehicle blind spot information providing device and method) Prior Patent 5: Korean Registered Patent No. 10-1353541 (Invention name: blind spot information providing device and information providing method)

The embodiments of the present invention aim to overcome the problems of the prior arts and provide an intelligent system or convenience that can be applied to autonomous vehicles or smart cars, which are the core technologies of the fourth industrial revolution.

The embodiments of the present invention aim to provide an intelligent side view camera system that supports high-quality real-time high-speed image processing and provides excellent image quality even in an ultra-low light environment.

In addition, embodiments of the present invention are to provide an intelligent side view camera system that can operate intelligently and adaptively to various driving environments.

In addition, embodiments of the present invention is to provide an intelligent side view camera system capable of real-time image processing that satisfies the criteria provided by international standards through distortion correction, broadband backlight correction, and image quality improvement according to an enlarged viewing angle.

In addition, embodiments of the present invention is to provide an intelligent side view camera system that can automatically detect nearby vehicles and objects using a camera image and provide appropriate notification to a user.

Other problems and specific objects to be solved of the present invention will be described in more detail through specific contents for carrying out the invention.

An intelligent side view camera system for a vehicle according to an embodiment supports real-time high-speed image processing and ultra-low-resolution high-resolution image acquisition, and is installed on the left side of the vehicle to obtain a left rear image of the vehicle, and a real-time high-speed image processing and second An image that supports low-light high-resolution image acquisition and is installed on the right side of the vehicle to acquire the right rear image of the vehicle, and the ultra-low-environment environment image of the vehicle's left and right rear images to remove noise and correct lens distortion. A recognition and processing unit, a control unit for controlling broadband backlight correction when detecting backlight conditions, controlling image quality improvement according to lighting or climatic conditions, and a control unit for generating an event signal according to detection of surroundings of the vehicle, and an event signal according to the control of the control unit. On the basis of the event processing unit for processing the occurrence of an event and a display unit for displaying the left image and the right image output from the image recognition and processing unit.

According to an embodiment of the present invention, it is possible to overcome the problems of the prior arts and provide an intelligent system or convenience that can be applied to autonomous vehicles or smart cars, which are core technologies of the fourth industrial revolution.

According to an embodiment of the present invention, it is possible to provide an intelligent side view camera system that supports high-quality real-time high-speed image processing and provides excellent image quality even in an ultra-low light environment.

Further, according to an embodiment of the present invention, it is possible to provide an intelligent side view camera system capable of operating intelligently and adaptively in various driving environments.

In addition, according to an embodiment of the present invention, it is possible to provide an intelligent side-view camera system capable of real-time image processing that satisfies the criteria provided by international standards through distortion correction, broadband backlight correction, and image quality improvement according to an enlarged viewing angle.

In addition, according to an embodiment of the present invention, it is possible to provide an intelligent side view camera system capable of automatically detecting surrounding vehicles and objects using a camera image and providing appropriate notification to a user.

1 is a view showing the configuration of an intelligent side view camera system for a vehicle according to an embodiment of the present invention.
2 is an exemplary view illustrating an example in which a side view camera system according to an embodiment is installed inside a vehicle.
3 is an exemplary view for explaining an installation example of a side view camera system according to an embodiment from outside the vehicle.
4 and 5 are exemplary views for explaining the configuration of a side view frame according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, but the present invention is not limited or limited by the embodiments.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, "comprises" and/or "comprising" refers to the components, steps, operations and/or elements mentioned above, the presence of one or more other components, steps, operations and/or elements. Or do not exclude additions.

As used herein, "example", "example", "side", "example", etc. should be construed as any aspect or design described being better or more advantageous than another aspect or designs. It is not done.

Also, the term'or' means an inclusive OR'inclusive or' rather than an exclusive OR. That is, unless stated otherwise or unclear from the context, the expression'x uses a or b'means any of the natural inclusive permutations.

Also, the singular expression (“a” or “an”) used in the specification and claims generally means “one or more” unless the context clearly indicates that it is of the singular form or unless otherwise stated. It should be interpreted as.

Further, terms such as first and second used in the specification and claims may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless specifically defined.

On the other hand, in the description of the present invention, when it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms used in the present specification (terminology) are terms used to properly represent an embodiment of the present invention, which may vary depending on a user, an operator's intention, or a custom in a field to which the present invention pertains. Therefore, definitions of these terms should be made based on the contents throughout the present specification.

1 is a view showing the configuration of an intelligent side view camera system for a vehicle according to an embodiment of the present invention.

Referring to FIG. 1, an intelligent side view camera system 100 for a vehicle according to an embodiment includes a left image acquisition unit 10, a right image acquisition unit 20, an image recognition and processing unit 110, a control unit 120, and events It includes a processing unit 130 and a display unit 140. The vehicle intelligent side view camera system 100 may further include a storage unit 150 and a sensor unit 160.

The left image acquisition unit 10 supports real-time high-speed image processing and ultra-low-illumination high-resolution image acquisition, and is installed on the left side of the vehicle to acquire the left rear image of the vehicle.

The right image acquisition unit 20 supports real-time high-speed image processing and ultra-low-illumination high-resolution image acquisition and is installed on the right side of the vehicle to acquire the right rear image of the vehicle.

The left image acquisition unit 10 and the right image acquisition unit 20 may include real-time high-speed image processing and an image sensor for acquiring an ultra-low-illumination high-resolution image.

For the image sensor, for example, a specification capable of supporting a high resolution of FHD (1080 pixels) and supporting at least 60 frames or more may be applied.

For the viewing angle of the image sensor, for example, a specification capable of supporting a wide angle of 60 degrees or more may be applied.

The image sensor may, for example, support a wide viewing angle of 30-80 degrees per side.

When the viewing angle of the image sensor is wide, it is easier to observe the side and rear of the vehicle while driving, and by installing two display monitors that output the left and right images at a location close to the driver, the driver can disparate the left and right images. You can observe without feeling.

The left image acquisition unit 10 and the right image acquisition unit 20 may be embedded in the side view frames 1 and 2 protruding out of the vehicle when driving, respectively.

At this time, the side view frames 1 and 2 may be designed to be inserted into the vehicle during parking and protrude to the outside at the same time as the start is stopped, unlike the existing side view mirrors.

Here, since the side view frames 1 and 2 occupy a very small area compared to a conventional side view mirror even in a protruding state while driving, there is an effect of increasing fuel efficiency in high-speed driving aerodynamically, and noise separating wind Removed, the wind noise can be greatly reduced.

The image recognition and processing unit 110 performs noise reduction and lens distortion correction in a very low-light environment image of a left rear image and a right rear image of the vehicle.

The image recognition and processing unit 110 may improve performance of a displayed image by applying frequency characteristic modeling and color correction of an ultra-low-light image sensor, noise modeling and de-noising of an ultra-low-light image sensor image, and color contrast techniques.

The image recognition and processing unit 110 may improve performance of a displayed image by applying a filtering technique in a time domain and a filtering method in a spatial domain.

When a lens having a wide viewing angle is applied to the image acquisition unit, the image recognition and processing unit 110 may correct the image distortion caused by the use of the lens having the wide viewing angle and output the corrected image distortion to the display unit.

The image recognition and processing unit 110 may correct the captured image to a normal image by applying software correction technology. When correcting the radial distortion of the lens, the image recognition and processing unit 110 compares the image with the distortion and the normal image, checks the distortion coefficient, and substitutes the opposite distortion of the distortion to correct and correct the spatial position of the three points By using the software technology to check the distortion rate and correct the image by applying the confirmed distortion rate to the distorted image.

The image recognition and processing unit 110 applies a broadband backlight correction algorithm to process all the images in a clean manner when a bright portion and a dark portion of the image are simultaneously input due to light reflection of a surrounding vehicle, a backlight, and the like. Broadband backlight correction can be performed for condition observation and image enhancement.

The image recognition and processing unit 110 adjusts the shutter speed of the sensor in order to minimize light blur when the bright light such as a headlight shines directly on the camera. To minimize light blur, the shutter speed is controlled to minimize light blur. Since there is a problem that the field of view is not secured, shutter speed control for minimizing blurring while maintaining proper brightness can be applied in consideration of both day and night situations.

The image recognition and processing unit 110 is a method for securing a clear image when information about a fog situation, a snowy situation, or a rainy situation is input through the controller 120, optimization of the AE algorithm, gamma correction technique, etc. Image quality can be improved by applying.

On the other hand, the image recognition and processing unit 110, the left image acquisition unit 10 and the right image acquisition unit 20 is installed in the vehicle when a low-spec image sensor capable of obtaining a high-resolution image in an ultra-low light environment is installed in the vehicle It is also possible to receive characteristic information from an image obtained from another image sensor (for example, a high specification front/rear-looking black box image sensor) and improve image quality by referring to the received characteristic information.

In addition, since the image sensor for obtaining real-time high-speed image processing and ultra-low-resolution high-resolution images is an expensive equipment, it may be mounted on only one of the left image acquisition unit 10 and the right image acquisition unit 20.

For example, the left image acquisition unit 10 is equipped with an image sensor for real-time high-speed image processing and obtaining ultra-low-resolution high-resolution images, and the right image acquisition unit 20 cannot acquire high-resolution images in an ultra-low-light environment. A low specification image sensor can be fitted.

In this case, the image recognition and processing unit 110 refers to the frequency characteristics, modeling and color correction, noise removal characteristics, etc., based on the image acquired by the left image acquisition unit 10, and then obtains the image acquired by the right image acquisition unit 20 Image quality improvement, illumination improvement, and the like.

When the backlight condition is detected, the controller 120 controls broadband backlight correction, controls image quality improvement according to lighting or climatic conditions, and generates an event signal according to vehicle surrounding detection.

The control unit 120 may control the image recognition and processing unit 110 to perform backlight correction when a backlight condition is detected in the image output from the image recognition and processing unit 110, and the image quality may be improved according to lighting or climatic conditions. The image recognition and processing unit 110 may be controlled by monitoring whether it is a necessary situation.

The control unit 120 may control the event processing unit 130 to detect a vehicle by analyzing an image output from the image recognition and processing unit 110 and to output a notification according to a state by calculating a vehicle speed and distance.

In order to provide an intelligent operation, the controller 120 may perform an operation of matching the left/right image acquisition device and the left/right display or the left/right area of the display during the initial calibration operation after the system is mounted.

Further, the controller 120 stores pixel information, brightness information, and flashing cycle information when a lighting state such as a direction indication is detected through a left/right image, even when separate information is not provided from a vehicle ECU or the like. You can check the lighting status of the direction indicator through the video. Through this, the control unit 120 may detect a situation in which the direction indicator light is lit, control the type of screen displayed on the display according to the lighting situation, control the steering angle of the image sensor, or adjust the sliding of the side view frame.

In addition, the controller 120 may detect the steering angle of the steering wheel through image analysis, and may provide a function of moving the view center or adjusting in a specific direction according to the detected steering angle of the steering wheel.

In addition, the control unit 120 may support the parking mode by adjusting the view center of the image sensor downward in the parking mode.

In addition, the controller 120 may calculate the speed of the trailing vehicle and the distance to the vehicle through image analysis, and may adjust zoom in and zoom out of the image sensor based on this.

The event processing unit 130 processes the occurrence of an event based on the event signal under the control of the control unit 120.

The event processing unit 130 may display the generated event on the display unit 140 or output a sound notification.

The display 140 may divide a region into a single screen to display a left image and a right image, or may include two displays 141 and 143.

The storage 150 may store output images of the left image acquisition unit 10, the right image acquisition unit 20, and the image recognition and processing unit 110.

The output image of each of the left image acquisition unit 10, the right image acquisition unit 20, and the image recognition and processing unit 110 stored in the storage unit 150 may be transmitted to a server at a remote location, and transmitted to a server at a remote location. The images may be used for image recognition and development of an image enhancement algorithm of the processor 110 through a higher performance image enhancement algorithm.

At this time, the output image itself of each of the left image acquisition unit 10, the right image acquisition unit 20, and the image recognition and processing unit 110 stored in the storage unit 150 may be difficult to be directly transmitted to the server. Accordingly, the control unit 120 may include a difference image and a characteristic difference between output images of the left image acquisition unit 10, the right image acquisition unit 20, and the image recognition and processing unit 110 stored in the storage unit 150. It is also possible to transmit only information and pixel difference information to the server.

The sensor unit 160 may include sensors that sense a driving environment.

For example, the sensor unit 160 may include at least one of an impact sensor, a vehicle speed sensor, a temperature sensor, a rain sensor, and a driver's eye sensor.

The controller 120 may control the operation of the intelligent side view camera system according to the driving environment based on the information input through the sensor unit 160.

For example, when a user's gaze is directed to the display unit 140 through a driver's gaze detection sensor, an image with improved image quality is provided, distance information from a trailing vehicle is displayed, or an edge portion of a trailing vehicle displayed on the screen is emphasized. The image recognition and processing unit 110 may be controlled to display.

Accordingly, the controller 120 may control an image output to the display unit 140 based on the user's gaze.

2 is an exemplary view illustrating an example in which a side view camera system according to an embodiment is installed inside a vehicle.

Referring to FIG. 2, an example in which a left image acquisition unit 10 and a display 1 (141, left display) are provided on the left side of a vehicle based on a driver is illustrated.

In FIG. 2, the side view frame 1 is installed near the left door 13.

On the other hand, if the user's gaze is not fixed to the left display 141 through the driver's gaze detection sensor, the controller 120 is displayed on the side of the driver's seat or through the windshield glass 11 or 12 through the mirror. It can also drive a head-up display system (HUD).

When the head-up display system (HUD) is implemented, the vehicle driving state of the rear room is displayed on the driver's side or on the front windshield glass (11, 12), so it is possible to monitor the rear side while keeping the driver's gaze forward. It can support safe driving.

3 is an exemplary view for explaining an installation example of a side view camera system according to an embodiment from outside the vehicle.

Referring to FIG. 3, the installation positions of the side view frames 1 and 2 in which the left image acquisition unit 10 and the right image acquisition unit 20 are installed are installed in a position where the existing side mirrors are installed. It may be, or may be set to allow the user to adjust the position through the sliding path (301, 302).

In addition, it can be set to automatically slide back and forth along the sliding path when a specific event occurs.

For example, a specific event that is sliding receives information only when a specific path change occurs (unturned left turn, unturned right turn on a narrow road, right turn, etc.) from the navigation device when navigation is operating, and turns right or left through location information When it is necessary to do so, when the direction indicator light corresponding to each direction is on, it is slid in the forward direction, and the angle of the image sensor may be set to adjust the viewing angle in the opposite direction of the direction indicator light to the maximum allowable range. In other words, if you need to turn right and enter the street side, when the right indicator light is on, the angle of the image sensor of the left image acquisition unit 10 widens the viewing angle to the left and the side view frame 1 follows the sliding path 301 in front of the vehicle. Can be sliding. This allows the driver to more safely and visually confirm the presence of the vehicle.

4 and 5 are exemplary views for explaining the configuration of a side view frame according to an embodiment.

4 and 5, the left and right side view frames are formed with air passages 510 through which air flows inside, and air flow is concentrated in the lens direction 401 of the camera on the outlet side of the air passages. The guide portion 410 may be formed so as to be possible.

When the flow path through which the air can be moved is formed in the side-view camera housing and the outlet of the flow path is formed toward the camera, water from the camera lens is automatically removed by wind around the camera when rain or snow falls. Can.

The device described above may be implemented with hardware components, software components, and/or combinations of hardware components and software components. For example, the devices and components described in the embodiments include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors (micro signal processors), microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed on networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and constructed for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes made by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (4)

A left image acquisition unit that supports real-time high-speed image processing and ultra-low-resolution high-resolution image acquisition and is installed on the left side of the vehicle to acquire a left rear image of the vehicle;
A right image acquisition unit that supports real-time high-speed image processing and ultra-low-resolution high-resolution image acquisition and is installed on the right side of the vehicle to acquire a right rear image of the vehicle;
An image recognition and processing unit that performs noise reduction and lens distortion correction in a very low-light environment image of a left rear image and a right rear image of a vehicle;
A control unit for controlling broadband backlight correction when detecting backlight conditions, controlling image quality improvement according to lighting or climatic conditions, and generating an event signal according to vehicle surrounding detection;
An event processing unit which processes an event occurrence based on an event signal under the control of the control unit; And
And a display unit displaying a left image and a right image output from the image recognition and processing unit.
Vehicle intelligent side view camera system. According to claim 1,
Further comprising a sensor unit including a gaze detection sensor for tracking the driver's gaze, the control unit to control the image output to the display unit based on the user's gaze
Vehicle intelligent side view camera system. According to claim 1,
The left image acquisition unit and the right image acquisition unit are respectively installed on the left and right side view frames, an air passage through which air is formed is formed, and a guide for concentrating air flow toward the camera lens at the outlet of the air passage Rich
Vehicle intelligent side view camera system. According to claim 1,
The control unit detects a vehicle through image analysis and calculates the speed and distance of the vehicle to control the event processing unit, a function to control a screen type displayed on the display based on a lighting state such as a direction indication, and a steering wheel steering angle. Provides at least one of the functions of detecting and controlling the steering angle of the image sensor according to the steering angle
Vehicle intelligent side view camera system.

Images