KR20210054094A - System and method for display control using camera image - Google Patents

Abstract

The present invention relates to a display control system using a camera image and, more specifically, to a system capable of controlling at least one of a display in a vehicle and a windshield glass display by using image information acquired through a camera. According to the present invention, the display control system includes: an input part receiving an image from a camera acquiring an image of surroundings of a vehicle; a memory storing a display control program for at least one of a display in a vehicle and a wind glass display by using the image; and a processor executing the display control program. The processor generates driving support information through analysis on the image, and performs display control corresponding to the driving support information to control the display in the vehicle or the windshield glass display.

Description

Display control system and method using camera image {SYSTEM AND METHOD FOR DISPLAY CONTROL USING CAMERA IMAGE}

The present invention relates to a display control system and method using a camera image, and more particularly, a system for controlling at least one of an in-vehicle display and a windshield glass display using image information acquired by a camera, and It's about how.

According to the prior art, an ADAS technology has been proposed for use as a black box using a camera that photographs the front/rear of a vehicle, and further discriminating against a signal change of a traffic light and preventing lane departure.

However, according to the prior art, since the driver does not control the in-vehicle display or the windshield glass display that the driver is watching while driving, there is a limitation in that a service capable of more intuitive recognition by the driver cannot be provided.

The present invention has been proposed to solve the above-described problem, and a display control that provides a service capable of intuitive recognition of the driver by performing mapping between an image acquired through a camera and an AVN/HUD screen or a windshield viewing area. Its purpose is to provide a system and method.

The display control system using a camera image according to the present invention includes an input unit for receiving an image from a camera that acquires an image around a vehicle, and at least one of an in-vehicle display and a windshield glass display using the image. It includes a processor that executes a memory and a display control program, wherein the processor analyzes the image to generate driving assistance information, and controls the in-vehicle display or the windshield glass display by performing display control corresponding to the driving assistance information. It is done.

The camera is a zoom camera that acquires an image outside the vehicle.

The input unit receives gaze information of the occupant from the in-vehicle image acquisition unit, and the processor displays driving assistance information classified as high risk in a preset area according to the gaze tracking result using the gaze information.

The processor performs image processing on the object area outside the vehicle in consideration of gaze information, and performs different image processing on the gaze area of the driver and passengers according to the classification of the object outside the vehicle.

The processor determines a display area and a display priority according to the importance classification of the driving assistance information.

The processor performs display control by determining whether information is necessary for the driver in relation to the object in the image.

The display control method using a camera image according to the present invention includes (a) acquiring an image around the vehicle, (b) analyzing the image to classify the risk of the object in the image, and (c) the risk classifying result. It characterized in that it comprises the step of determining the display area and priority according to the display, and performing display control.

In step (a), an image is acquired using a zoom camera disposed on the vehicle.

In step (b), driving risk information, auxiliary information, and driving convenience information are classified according to the degree of risk of the object.

In step (c), the information is displayed in the area where the gaze is located in consideration of the gaze information of the occupant or a separate display for the actual photograph is performed.

According to an embodiment of the present invention, the driver is more intuitive by controlling to display an image on road conditions and milestones while driving acquired through a camera image, or to display a separate display (highlight) on the actual picture beyond the windshield glass. It has the effect of providing services that can be perceived as.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram showing a display control system using a camera image according to an embodiment of the present invention.
2 illustrates display control according to a driver's gaze area of a display control system using a camera image according to an exemplary embodiment of the present invention.
FIG. 3 is a view showing highlight display of a real picture beyond a windshield glass of a display control system using a camera image according to an exemplary embodiment of the present invention.
4A to 4C are diagrams illustrating display control for a live-action picture beyond a windshield glass of a display control system using a camera image according to an exemplary embodiment of the present invention.
5 is a flowchart illustrating a display control method using a camera image according to an embodiment of the present invention.

The above-described and other objects, advantages, and features of the present invention, and methods of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the following embodiments are the object of the invention to those of ordinary skill in the art to which the present invention pertains. It is only provided to easily inform the composition and effect, and the scope of the present invention is defined by the description of the claims.

Meanwhile, terms used in the present specification are for explaining embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, actions and/or elements in which the recited component, step, operation and/or element is Or does not preclude additions.

1 is a block diagram showing a display control system using a camera image according to an embodiment of the present invention.

The display control system 100 using a camera image according to the present invention includes an input unit 11 receiving an image from a camera that acquires an image around a vehicle, and at least one of an in-vehicle display and a windshield glass display using the image. It includes a memory 120 in which a display control program is embedded and a processor 130 that executes the display control program, and the processor 130 analyzes the image to generate driving assistance information, and performs display control corresponding to the driving assistance information. To control the in-vehicle display or the windshield glass display.

The camera is a zoom camera that acquires an image outside the vehicle.

The processor 130 performs display control with a display (image) for information such as road conditions and milestones while driving and highlights for a windshield glass (real photo).

The processor 130 displays guide signs and information, provides a navigation path more accurately, and performs display control to highlight and show the corresponding text when a sign matching the navigation path is seen.

The processor 130 performs display control to coat color on the AR HUD and windshield glass using image information and text recognition results obtained using a zoom camera.

The input unit 110 receives gaze information of the occupant from the in-vehicle image acquisition unit, and the processor 130 displays driving assistance information classified as high risk in a preset area according to the gaze tracking result using the gaze information.

According to an embodiment of the present invention, driving assistance information related to an image acquired through a camera is displayed in different areas in consideration of the driver's gaze and the importance of the information.

For example, a display control is performed on the upper part of the windshield to allow the driver to intuitively recognize information located on the upper part such as driving milestones, etc., and display control is performed on the road situation and the HUD screen at the lower part of the windshield.

The processor 130 performs image processing on the object area outside the vehicle in consideration of gaze information, and performs different image processing on the gaze area of the driver and passengers according to the classification of the object outside the vehicle.

The processor 130 determines a display area and a display priority according to the classification of the importance of the driving assistance information.

In the case of driving hazard information (accident, road kill, road damage), the information is controlled to be displayed/highlighted in the area where the driver's gaze is located.

The processor 130 interlocks with the cloud and the server, transmits the result of recognizing the situation or object in front of the driving vehicle with the zoom camera, and provides detailed information on the type of the object (box, animal) to the occupant.

In the case of driving assistance information (mileposts, speeding cameras, etc.), it is controlled to display/highlight in a preset area (general area).

In the case of driving convenience information (gas station, etc.), whether or not to be displayed is determined according to whether it is necessary to provide it to the driver, and when it is displayed, it is controlled to be displayed in the area in front of the cluster or the passenger seat.

The processor 130 performs display control by determining whether information is necessary for the driver in relation to the object in the image.

The processor 130 determines whether the information recognized by the camera is information necessary for the driver, and performs display control such as an image display or a live-action highlight according to a result of determining whether to display the corresponding information on the screen.

Even when it is determined not to display the corresponding information on the screen, the processor 130 does not provide a separate warning to the driver, but provides a simple explanation (text, voice, etc.) according to the setting.

The processor 130 uses the image acquired by the zoom camera to change the lane or reduce the speed according to danger information (a puddle, a stepped portion on the road) of the route.

When there is an overloaded freight vehicle in front of the vehicle by using the image acquired by the zoom camera, the processor 130 controls the vehicle to travel while avoiding it.

The processor 130 guides a lane having a relatively smooth traffic flow when there is a congestion section in front using the image acquired by the zoom camera.

The processor 130 acquires traffic light information located at a distance by using the image acquired by the zoom camera and controls the speed of the driving vehicle accordingly.

The processor 130 acquires information on the driving speed of a vehicle located at a distance and a nearby vehicle by using the image acquired by the zoom camera, and proposes an optimal speed (ASCC).

When the ambulance approaches, the processor 130 displays the image on the cluster or HUD to yield by using the image acquired by the rear zoom camera.

According to an embodiment of the present invention, an image is acquired and stored for a POI in a driving route, a route is set in a video, and an address is provided to the image.

According to an embodiment of the present invention, voice data acquired from a passenger (for example, "Shall we eat lunch later") and a route viewed by the zoom camera are set to provide not only route information to the driver, but also information about the relevant building (restaurant). to provide.

2 illustrates display control according to a driver's gaze area of a display control system using a camera image according to an exemplary embodiment of the present invention.

The processor 130 obtains the occupant's gaze direction information D1 and D2 from a camera photographing the occupant in the vehicle, and controls to display driving assistance information in an area where the occupant's gaze is located.

In the case of driving risk information (accident, road kill, road damage), the processor 130 controls to display/highlight the corresponding information in the area A1 where the driver's gaze is located.

The processor 130 controls to display/highlight the driving assistance information (milestone, speeding camera, etc.) in a preset area (A2, general area).

In the case of the driving convenience information (gas station, etc.), the processor 130 determines whether to display the driving convenience information (gas station, etc.) according to whether or not the driver needs to be provided, and controls the display to be displayed in the cluster or front area A3 of the passenger seat.

When the processor 130 is classified as high risk and it is determined that it is necessary to continuously inform the driver, the area where the driver's gaze is located is changed from A1 to A2 as the driver's gaze information is changed from D1 to D2. It is determined that it is, and the information is controlled to be continuously displayed by tracking the driver's gaze.

FIG. 3 is a view showing highlight display of a real picture beyond a windshield glass of a display control system using a camera image according to an exemplary embodiment of the present invention.

The processor 130 checks whether there is information matching the navigation path information among the front images acquired using the zoom camera.

When the route is set in the navigation, the processor 130 can retrieve the route information and check whether it matches the front image, and even when the navigation route is not set, the driver is expected to drive using the driving history information. It is possible to check whether the forward image is matched according to the expected path.

For example, since the real image viewed by the driver through the windshield has a limit on the recognition distance, the zoom camera recognizes the sign information of a remote location and transmits it to the processor 130.

The processor 130 performs display control to perform a small highlight on the entire sign when a sign matching the path of the navigation is seen with respect to sign information at a remote location, and highlights and displays text within the sign as the relative distance is narrowed. , Display control is performed to highlight and display only a portion of the sign that matches the path.

3, the driving vehicle is driving on a road in which lanes 1, 2, and 3 are in the direction of Daejeon, lanes 4 are in Gangneung, and lanes 5 are in Incheon. It can be seen that the vehicle must drive in the direction of Incheon, that is, along five lanes.

The processor 130 performs display control to highlight some areas of signs (areas corresponding to signs corresponding to a preset driving path) among signs matched with navigation path information, so that the driver can perform the actual signs displayed over the windshield. By recognizing the highlight display, it is possible to smoothly travel according to a preset route.

4A to 4C are diagrams illustrating display control for a live-action picture beyond a windshield glass of a display control system using a camera image according to an exemplary embodiment of the present invention.

As shown in FIG. 4A, it is assumed that there is a carcass of an animal that has been roadkilled across the lane in front of the driving vehicle.

In addition to the warning signs of wildlife, it is proposed to guide the road kill section through the navigation, but the carcasses of animals that have already occurred are not only a risk to safe driving, but also cause psychological discomfort to drivers and passengers who witnessed them. .

As shown in FIG. 4B, when there is a load-killed animal carcass in the front image acquired by using the zoom camera, the processor 130 performs a blur process on the real windshield (animal carcass), By performing display control to blur the corresponding area, the driver and passengers are supported so that they do not see the dead animal as it is.

The processor 130 blurs the real windshield (carcasses of animals), but highlights the edges of the windshield so that the carcasses are not seen as they are, but provides a mark so that driving avoiding them is smooth. .

As shown in FIG. 4C, the processor 130 blurs the real windshield (carcasses of animals), but traffic information on roads covered by the carcasses of animals (eg, lanes, center lines, direction guidance information, speed guidance information) Is displayed as a highlight in the blurred area.

The processor 130 acquires gaze direction information of the driver and passengers, and accordingly performs display control to perform blur processing and the above-described highlight processing for the area where the driver's gaze is located, and the area where the gaze of the passenger in the passenger seat is located. The display control is performed differently for only the blur process.

5 is a flowchart illustrating a display control method using a camera image according to an embodiment of the present invention.

The display control method using a camera image according to the present invention includes obtaining an image around the vehicle (S510), analyzing the image to classify the risk of an object in the image (S520), and displaying a display according to the result of classifying the risk. Determining an area and priority, and performing display control (S530).

In step S510, an image is acquired using a zoom camera disposed on the vehicle.

In step S520, driving risk information, auxiliary information, and driving convenience information are classified according to the degree of risk of the object.

The driving risk information relates to an accident, a road kill, a road break, and the like, and in step S530, the corresponding information is controlled to be displayed/highlighted in an area where the driver's gaze is located.

The driving assistance information relates to a milestone, a speeding camera, and the like, and in step S530, a control is performed to display/highlight a preset area (general area).

The driving convenience information relates to a gas station, etc., in step S520, whether to be displayed is determined according to whether or not it is necessary to provide the driver, and in step S530, the cluster or the front area of the passenger seat is controlled to be displayed.

In step S530, information is displayed in an area where the gaze is located in consideration of the gaze information of the occupant or a separate display (highlight) for the actual photograph is performed.

In step S530, in the case of important information that needs to be continuously provided to the driver, the driver's gaze is tracked and the important information is continuously displayed in the area where the gaze is located.

In step S530, when the display target information occurs at the same time, display control is performed in the order of driving risk information, auxiliary information, and convenience information according to the priority of the risk.

Meanwhile, the display control method using a camera image according to an embodiment of the present invention may be implemented in a computer system or recorded on a recording medium. The computer system may include at least one processor, memory, user input device, data communication bus, user output device, and storage. Each of the above-described components performs data communication through a data communication bus.

The computer system may further include a network interface coupled to the network. The processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in a memory and/or storage.

The memory and storage may include various types of volatile or nonvolatile storage media. For example, the memory may include ROM and RAM.

Accordingly, a method for controlling a display using a camera image according to an exemplary embodiment of the present invention may be implemented in a computer-executable method. When a display control method using a camera image according to an embodiment of the present invention is performed in a computer device, commands readable by a computer may perform the control method according to the present invention.

Meanwhile, the display control method using a camera image according to the present invention described above may be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording media in which data that can be decoded by a computer system is stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like. In addition, the computer-readable recording medium can be distributed in a computer system connected through a computer communication network, and stored and executed as codes that can be read in a distributed manner.

So far, we have looked at the center of the embodiments of the present invention. Those of ordinary skill in the art to which the present invention pertains will be able to understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the above description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (12)

An input unit for receiving an image from a camera for obtaining an image around the vehicle;
A memory in which at least one display control program of at least one of an in-vehicle display and a windshield glass display is embedded using the image; And
Including a processor for executing the display control program,
The processor analyzes the image to generate driving assistance information, and controls the in-vehicle display or windshield glass display by performing display control corresponding to the driving assistance information.
Display control system using in-camera image.
The method of claim 1,
The camera is a zoom camera that acquires an image outside the vehicle
Display control system using in-camera image.
The method of claim 1,
The input unit receives gaze information of the occupant from an image acquisition unit in the vehicle.
Display control system using in-camera image. The method of claim 3,
The processor is configured to display, in a preset area, driving assistance information classified as high risk according to a gaze tracking result using the gaze information.
Display control system using in-camera image.
The method of claim 3,
The processor performs image processing on an object area outside the vehicle in consideration of the gaze information
Display control system using in-camera image.
The method of claim 5,
The processor performs different image processing on the gaze areas of the driver and passengers according to the classification of objects outside the vehicle.
Display control system using in-camera image.
The method of claim 1,
The processor determines a display area and a display priority according to the importance classification of the driving assistance information.
Display control system using in-camera image.
The method of claim 1,
The processor performs the display control by determining whether information is necessary for the driver in relation to the object in the image
Display control system using in-camera image.
(a) obtaining an image around the vehicle;
(b) analyzing the image to classify the risk of objects in the image; And
(c) determining a display display area and priority according to the result of classification of the risk, and performing display control
Display control method using a camera image comprising a.
The method of claim 9,
The step (a) is to acquire the image using a zoom camera disposed on the vehicle.
Display control method using an in-camera image.
The method of claim 9,
The step (b) is to classify driving risk information, auxiliary information, and driving convenience information according to the risk of the object.
Display control method using an in-camera image.
The method of claim 9,
The step (c) is to display the information in the area where the gaze is located in consideration of the gaze information of the occupant or to perform a separate display for the actual photograph.
Display control method using an in-camera image.

Images