KR20120066194A - System for providing image around vehicle with portable terminal and method thereof - Google Patents

Abstract

PURPOSE: A system for providing a vehicle surrounding image is provided to enable for user to easily park and to simplify the structure of a device for a vehicle. CONSTITUTION: A system for providing a vehicle surrounding image comprises: cameras(10) mounted on a vehicle and photographs an image surrounding the vehicle; an image processing unit(20) which processes the images according to previously selected view and the posture of a portable terminal(100); and a control unit(130) of the portable terminal for displaying surrounding image on a display panel(110).

Description

Vehicle surrounding image providing system and method using a mobile terminal {SYSTEM FOR PROVIDING IMAGE AROUND VEHICLE WITH PORTABLE TERMINAL AND METHOD THEREOF}

The present invention relates to a system for providing a vehicle surrounding image using a mobile terminal and a method thereof, and more particularly, to correct and provide a surrounding image of a vehicle according to the horizontal and vertical arrangement of a display panel of the portable terminal, It provides a peripheral image including a vehicle auxiliary line for guiding parking and prevents a cost increase due to the use of expensive equipment, promotes safe driving of the user, and records the driving state of the user. The present invention relates to a vehicle surrounding image providing system using a mobile terminal and a method thereof.

In general, a system for capturing and monitoring a vehicle's surroundings is not universally installed in a vehicle, so a driver may check the front through his own eyes, or the rear through a side mirror or a room mirror, and move the vehicle forward or I have been going backwards.

However, depending on the structure of the car, it may be easy to check the surrounding environment of the front and rear sitting in the driver's seat, or there may be blind spots that cannot be seen through the naked eye or each mirror. Especially in the case of a large car, there are many parts that cannot be identified by the side mirror or the room mirror alone. Before starting the car, it is necessary to look around the car and check whether there are any obstacles with the naked eye. Can be prevented.

In addition, since the driver who wants to park the car cannot check at a glance the left and right and rear, the inexperienced novice driver has a risk of contact accident with a car or a parking pillar that is parked around. In addition, even an obstacle located in front of the vehicle may be obscured by a frame positioned between the windshield and the door of the vehicle, and may lead to a human accident if the infants sitting and playing in the front or rear of the vehicle are not found. have.

Therefore, recently, an apparatus for photographing a surrounding environment through a camera attached to front, rear, left and right sides of a vehicle, and combining the captured images and displaying them on a navigator screen provided in the driver's seat has been developed. However, when the separate navigator is not provided, it is inconvenient to purchase a navigator or another device in order to receive a service that receives the surrounding image of the vehicle. In addition, in recent years, automotive multi-camera technology such as wire-based AVM has been developed and used, but the configuration of the equipment is complicated and expensive, it is difficult to be widely used.

On the other hand, portable terminals such as smartphones, cellular phones, PDAs, and tablet PCs, which are widely used in recent years, have various functions including a navigator function and are easily connected with portable terminals according to the development of a local area network. There is a need to find a method for providing to a portable terminal. In this way, when the user uses the portable terminal while changing the posture of the portable terminal in the horizontal or vertical direction, it is determined that the correction of the image according to the change in the posture of the portable terminal is necessary.

On the other hand, even if a novice driver is provided with a rear view image at the time of rear parking, the rear view image may not know in which direction the car must be reversed in order to prevent a contact accident with a car or a parking post parked nearby. You will need additional ways to guide parking.

The present invention provides a vehicle surrounding image providing system using a mobile terminal to correct the surrounding image of the vehicle according to the posture of the portable terminal and provide it to the portable terminal, and synthesize the vehicle auxiliary line to the surrounding image when displaying it on the portable terminal. And a method thereof.

In addition, the present invention is to provide a vehicle surrounding image providing system and a method using a portable terminal to prevent the cost increase due to the use of expensive equipment according to the use of a high specification operation function and the display panel of the portable terminal The purpose.

In addition, the present invention, the vehicle information provided from the vehicle in real time to display on the portable terminal or to the external black box server to promote the safe driving of the user, at any time to check the user's driving state can be used as evidence later An object of the present invention is to provide a vehicle surrounding image providing system and a method using a portable terminal.

According to an aspect of the present invention, there is provided a vehicle surrounding image providing system using a portable terminal, the camera being mounted on a vehicle and photographing an image around the vehicle; And an image processing apparatus configured to process the images photographed and processed by the camera so as to correspond to the posture of the portable terminal and a preselected viewpoint.

In a vehicle surrounding image providing system using a mobile terminal according to another embodiment of the present invention, an image of processing the vehicle surrounding image photographed by a camera mounted on the vehicle so as to correspond to the posture of the portable terminal and a pre-selected viewpoint to generate an image. And a portable terminal including a processing device.

The vehicle surrounding image providing system may further include an auxiliary line generator configured to generate a virtual parking auxiliary line that guides parking according to the movement of the vehicle when the vehicle is parked.

The image processing apparatus may generate a surrounding image having a parking assistance line by superimposing the parking assistance line generated by the auxiliary line generator on an image photographed by the camera.

The portable terminal may further include a position sensor for detecting a vertical and horizontal arrangement of the portable terminal, and the image processing apparatus is configured to display the display according to the horizontal and vertical arrangement of the display panel of the portable terminal detected by the position sensor. The surrounding image or the parking assistance line may be sized to correspond to the width or height of the panel.

The image processing apparatus may superimpose and display a message indicating that the image displayed on the display panel is different from the actual image according to the size change of the surrounding image and the parking auxiliary line.

A plurality of cameras are provided, and the image processing apparatus provides an image captured by the camera as an original image to a display panel of the portable terminal, or processes the image to have a viewpoint of one of a top view image and a 3D image. It can be provided as a display panel.

The portable terminal can communicate with the image processing apparatus by wire or wirelessly.

A vehicle information collecting unit for collecting information on an operating state of the vehicle from electronic devices and sensors of the vehicle; Information collected by the vehicle information collecting unit may be provided to the portable terminal and displayed.

The portable terminal may be equipped with a program such as a lane departure warning system (LDWS), a forward collision warning system (FCWS), a driver driving record apparatus, and the like.

It is connectable to the portable terminal through a communication network, and provides a peripheral image of the vehicle processed by the image processing apparatus, information collected by the vehicle information collecting unit, and information from the LDWS, FCWS, driver driving recorder, and the like. It may further include a black box server for receiving and storing.

According to still another aspect of the present invention, there is provided a method for providing a vehicle surrounding image using a portable terminal, the method including photographing an image of a vehicle surrounding using a camera; Generating surrounding images by processing the images photographed by the camera to correspond to a posture of a mobile terminal and a pre-selected viewpoint; And displaying the processed image captured by the camera on the portable terminal.

The method includes generating a virtual parking assistance line for guiding parking according to the movement of the vehicle when the vehicle is parked; And generating a surrounding image having the parking assistance line by overlapping the parking assistance line with the image photographed by the camera.

The method includes detecting a change in the width and width of the mobile terminal; Correcting the surrounding image or the parking assistance line according to the horizontal and vertical arrangement of the portable terminal; And displaying the corrected surrounding image or parking assistance line on the display panel of the portable terminal.

The method may include superposing a message indicating that the image displayed on the display panel is different from the actual image according to the size change of the surrounding image or the parking assistance line; As shown in FIG.

The method includes providing a plurality of cameras; And displaying the images captured by the plurality of cameras on the display panel of the portable terminal as it is, or by the image processing apparatus to have one of a top view image and a 3D image. It may further comprise the step of displaying.

The method includes collecting information about an operating state of a vehicle from electronic devices and sensors of the vehicle; And displaying the information on a display panel of the portable terminal.

The method includes collecting information from a lane departure warning system (LDWS), a forward collision warning system (FCWS), a driver driving record apparatus, and the like mounted on the portable terminal; Synthesizing the information into the surrounding image; And displaying the surrounding image synthesized with the information on the display panel of the portable terminal.

The method may include surrounding images processed by the image processing apparatus by an external black box server through a communication network, information collected from electronic devices and sensors of the vehicle, and information collected from the LDWS, FCWS, driver driving recorder, and the like. Sending a; may further include.

The vehicle surrounding image providing system and method using the portable terminal can provide image information of the vehicle to the portable terminal, and in particular, corrects and provides the surrounding image according to the width and width of the display panel of the portable terminal. Accordingly, since the user does not have to have a separate navigator or equipment, the user can reduce the economic burden and promote user satisfaction. In addition, by providing an auxiliary parking line for guiding parking when the vehicle is superimposed on the surrounding image to the portable terminal, the user can easily park the vehicle. In particular, when processing the image information in the portable terminal, by utilizing the high specification operation function of the portable terminal, and by using the display panel, it is possible to reduce the cost by simplifying the structure of the vehicle device.

In addition, the vehicle information provided from the vehicle can be displayed on the portable terminal in real time, thereby helping the user to drive safely. In addition, by uploading the user's driving record, surrounding images, vehicle information, etc. to the external black box server in real time, the user can later check the driving status of the user and use it as evidence of contact accidents.

1 is a schematic block diagram of a vehicle surrounding image providing system using a portable terminal according to an embodiment of the present invention;
2A and 2B are front views of a mobile terminal before and after an image is processed in the image providing system of FIG. 1;
3 is a block diagram illustrating an image processing apparatus of FIG. 1;
4 is a flowchart illustrating a process of providing an image to a mobile terminal in the image providing system of FIG. 1;
5 is a schematic block diagram of a vehicle surrounding image providing system using a mobile terminal according to another embodiment of the present invention;
6 is a flowchart illustrating a process of processing an image provided by the image providing system of FIG. 5 in a portable terminal;
7 is a block diagram of an image providing system of the present invention having a black box function.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

1 is a schematic block diagram of a vehicle surrounding image providing system using a portable terminal according to an embodiment of the present invention, and FIGS. 2A and 2B are front views of a mobile terminal before and after an image is processed in the image providing system of FIG. to be.

The image providing system captures an image of a vehicle surrounding and processes the image at various viewpoints and provides the image to the portable terminal 100. In this case, the image is corrected to conform to the display panel according to the horizontal and vertical directions of the portable terminal 100. Provided to the terminal 100. In addition, the present image providing system superimposes the parking assist line when the vehicle is parked on an image around the vehicle and provides the image to the portable terminal 100.

To this end, the present image providing system may include a vehicle device 1 installed in a vehicle for capturing and processing a vehicle surrounding image and a mobile terminal 100 provided with an image provided from the vehicle device 1.

The portable terminal 100 may be a memory such as a notebook computer, a palmtop computer, an ultra mobile personal computer (UMPC), a tablet PC, a personal digital assistant (PDA), a web pad, a mobile phone, a smartphone, or the like. It may be made of a terminal having a computing power with a means and a microprocessor.

The portable terminal 100 may include a display panel 110, a position sensor 115, a terminal side communication unit 120, and a terminal side control unit 130.

The display panel 110 is mounted on one side of the mobile terminal 100 to display an image of a vehicle surrounding provided from a vehicle apparatus, and generally has a width and width. The display panel 110 may be manufactured in various sizes and materials according to the type of the mobile terminal 100. For example, materials such as LCD, LED, TFT LCD (thin-film transistor liquid crystal display), AMOLED (active organic light emitting diode) and the like can be used.

The position sensor 115 is a sensor for detecting horizontal and vertical position changes of the mobile terminal 100, and may use a gravity sensor, an acceleration sensor, a gyro sensor, or the like.

The terminal side communication unit 120 supports communication with a vehicle device, and supports wireless or wired communication. Accordingly, the portable terminal 100 can communicate with the vehicle device 1 using a Bluetooth wireless communication method or a wired communication method using a USB port.

The terminal controller 130 may determine whether the display panel 110 is disposed vertically or horizontally according to the position change of the mobile terminal 100 detected by the position sensor 115. The terminal control unit 130 provides the determined horizontal and vertical arrangement information of the display panel 110 to the vehicle apparatus 1 through the terminal communication unit 120. In addition, the terminal-side control unit 130 displays the surrounding image provided from the vehicle apparatus 1 or the surrounding image having the auxiliary line on the display panel 110.

The on-vehicle device 1 processes a photographed image mounted on a vehicle and provides the image to the mobile terminal 100, and includes a plurality of cameras 10, an image processing device 20, an auxiliary line generator 50, and a vehicle side. The communication unit 70 and the vehicle-side control unit 30 may be included.

The plurality of cameras 10 may be installed at the front, rear, and both sides of the vehicle, respectively, and may use a lens having a large angle of view, such as a wide angle lens or a fisheye lens, and a pinhole camera 10 may also be used. Each camera 10 captures a three-dimensional object as a two-dimensional image through a lens having a wide angle of view of 170 ° or more, and the captured image is processed through four channels (ch1, ch2, ch3, and ch4), respectively. Delivered to device 20.

On the other hand, the position and the shooting angle each camera 10 is installed may be variously determined according to the conditions, such as the imageable range of the camera 10, the size of the vehicle, the desired shooting range. For example, in the case where the camera 10 can be photographed in a wide range, the installation interval of the camera 10 can be widened. In the case where the size of the vehicle is large, the installation number of the camera 10 can be increased. If the user wants to widen the range of photographing along the periphery of the vehicle, the installation position of the camera 10 may be disposed on the upper portion of the vehicle, or a plurality of cameras 10 may be installed in the vertical direction of the vehicle. Accordingly, the plurality of cameras 10 can be mounted on the front, rear, side, side mirror areas, and the like of the vehicle. In addition, the plurality of cameras 10 may be mounted on the vehicle roof, and when mounted on the vehicle roof, the cameras 10 may be mounted on the front, rear, and both sides of the vehicle roof.

The images captured by the cameras 10 are transferred to the image processing apparatus 20 in real time.

The image processing apparatus 20 receives an image captured by each camera 10 and corrects the image, and a point in time at which the image captured by each camera 10 is set by a user, for example, a top view image and an original image. Convert to 3D video. In addition, the image processing apparatus 20 synthesizes the parking auxiliary line generated by the auxiliary line generator 50 to the surrounding image, and displays the surrounding image according to the horizontal and horizontal arrangement of the display panel 110 of the mobile terminal 100. Correct.

Here, the top view image is an image viewed from the top of the vehicle, and the image processing apparatus 20 composes the top view image using the images captured by the cameras 10. The original image refers to the image taken by each camera 10, and in the case of the original image, the image processing apparatus 20 performs simple processing such as removing noise or amplifying a signal. The 3D image refers to an image obtained by reconstructing an image photographed by each camera 10 in 3D, and the image processing apparatus 20 forms a 3D image by combining images captured by each camera 10.

As shown in FIG. 3, the image processing apparatus 20 includes an image input unit 21, an image processing unit 23, a first image combining unit 25, a second image combining unit 26, and an image correcting unit. (29).

The image input unit 21 receives an image captured by the plurality of cameras 10 through each channel ch1, ch2, ch3, and ch4 and performs a preprocessing process of removing or amplifying noise of the received image. . The image preprocessed by the image input unit 21 is provided to the vehicle-side control unit 30 or the image processing unit 23 according to the type of the output image. For example, in the case of the original image, the images photographed by the plurality of cameras 10 are removed from the image input unit 21 and transmitted to the second image synthesizing unit 26 or the image correcting unit 29 for processing. . On the other hand, the top view image or the 3D image is provided to the image processor 23.

The image processor 23 processes the captured image received from the image input unit 21 through a look up table 27 to generate and output a top view image or a 3D image corrected image from each captured image. do. Here, the lookup table 27 may be generated by applying a distortion correction algorithm, an affine transformation algorithm, and a viewpoint transformation algorithm.

The lookup table 27 used for image processing in the image processor 23 is divided into a top view image lookup table 27a and a 3D image lookup table 27b according to the viewpoint of the final output image. Accordingly, when one of the top view and the 3D is selected by the driver or the designer in advance, the image processor 23 selects the lookup table 27 corresponding to the selected viewpoint and generates and outputs a corrected image.

The first image synthesizer 25 receives the corrected image corrected through the image processor 23, and synthesizes the received image by an overlay method to overlap the received corrected image. The first image synthesizing unit 25 performs an overlay composition process using a mask image. The mask image includes weights of the region information for each channel (ch1, ch2, ch3, ch4) and pixels constituting the corrected image. ) Has information.

The first image synthesizer 25 adjusts the weights of the pixels included in the overlapping regions between the corrected images so that the overlapping regions are naturally displayed. That is, the first image synthesizing unit 25 synthesizes the four correction images in an overlay manner to generate a peripheral image that can see the surrounding 360 ° of the vehicle at a glance. In this case, the surrounding image is generated as one of the top view image and the 3D image.

The second image synthesizing unit 26 converts the auxiliary line generated by the auxiliary line generating unit 50 into a top view image or a 3D image generated by the first image synthesizing unit 25 and an original image processed by the image input unit 21. The image is superimposed on the formed surrounding image to generate a 'surrounding image having auxiliary lines'. In addition, the second image synthesizing unit 26 receives the message generated by the vehicle-side control unit 30 and superimposes the surrounding image.

The auxiliary line generating unit 50 generates a parking auxiliary line which is a road that the vehicle should go backward when the vehicle is parked by using the auxiliary line generating program prepared in advance. First, the auxiliary line generator 50 selects an auxiliary line generation program corresponding to the parallel parking line or the vertical parking line, calculates the turning angle of the vehicle using the minimum turning radius and the angular velocity of the vehicle. Create a matching parking assist line. The parking assistance line is generated as a pair of lines indicating the wheel of the vehicle or the trajectory of the vehicle itself. The method of generating the parking auxiliary line is a known technique, and since the present invention does not claim to generate the auxiliary line as a right range, a detailed description thereof will be omitted.

The minimum turning radius of the vehicle required for generating the parking auxiliary line is calculated and stored in advance by the vehicle designer, and the angular velocity may be calculated using the turning radius calculated by the angle of the wheel and the speed of the vehicle using the speedometer.

The auxiliary line generating unit 50 may be manually operated after a predetermined input action is performed by the user, or may be automatically operated when the vehicle control unit 30 determines that the vehicle is parked. .

The parking auxiliary line generated by the auxiliary line generator 50 is provided to the second image synthesizing unit 26 of the image processing apparatus 20 and overlaps with the surrounding image to synthesize the 'ambient image having the auxiliary line'.

The image corrector 29 synthesizes the surrounding image or the second image synthesizer 26 generated by the first image synthesizer 25 according to the horizontal and vertical directions of the portable terminal 100 provided from the portable terminal 100. The surrounding image having the auxiliary line is corrected. As illustrated in FIG. 2, when the mobile terminal 100 is disposed horizontally, the vertical width of the display panel 110 is smaller than the horizontal width. On the other hand, when the mobile terminal 100 is disposed vertically, the vertical width of the display panel 110 is narrower than the horizontal width.

Accordingly, the image corrector 29 may correct the surrounding image in two ways. One is a method of cutting and displaying both sides of the surrounding image by an area cut by the vertical arrangement when the mobile terminal 100 is rotated to switch from the horizontal arrangement to the vertical arrangement. On the contrary, when switching from the vertical arrangement to the horizontal arrangement, the upper and lower portions of the surrounding image may be cut and displayed as much as the area cut by the horizontal arrangement. Another method is to resize the surrounding image to reduce the aspect ratio and reduce the aspect ratio when the mobile terminal 100 is rotated to switch from the horizontal arrangement to the vertical arrangement. On the contrary, when switching from the vertical arrangement to the horizontal arrangement, the aspect ratio of the surrounding image may be increased and the size may be resized to decrease the aspect ratio. In this case, the aspect ratio and the aspect ratio to be resized may be determined according to the aspect ratio of the display panel 110, respectively.

Meanwhile, one of the vertical arrangement and the horizontal arrangement of the mobile terminal 100 may be set as a basic arrangement, and the image correction unit 29 may operate only when the basic arrangement is changed from the basic arrangement to another arrangement. That is, when the basic arrangement of the portable terminal 100 is set to the horizontal arrangement, when the portable terminal 100 is arranged horizontally, the display panel 110 is displayed as it is without correcting the surrounding image. On the other hand, when the mobile terminal 100 is disposed vertically, the image correction unit 29 may be operated to resize the surrounding image by using one of the two methods described above to display the surrounding image on the display panel 110. .

The vehicle side communication unit 70 supports communication with the mobile terminal 100, and may support a wireless or wired communication method. In the case of supporting wireless communication, the vehicle-side communication unit 70 may support Bluetooth communication with the mobile terminal 100, and in case of supporting wired communication, the mobile terminal 100 may be communicatively connected through a USB port. Can support the method.

The vehicle control unit 30 is processed by the image input unit 21 according to the vertical or horizontal arrangement of the mobile terminal 100 detected by the position sensor 115 of the mobile terminal 100 when the vehicle is in normal driving. The original image, the top view image or the 3D image generated by the first image synthesizer 25 is provided to the image corrector 29 to be corrected. Meanwhile, when the vehicle is parked, the vehicle control unit 30 operates the auxiliary line generator 50 to generate a parking auxiliary line, and transfers the generated parking auxiliary line to the second image synthesizing unit 25. 'Nearby image with auxiliary line' is generated. In addition, the vehicle-side control unit 30 uses the auxiliary line generated by the second image synthesizing unit 26 according to the vertical or horizontal arrangement of the mobile terminal 100 detected by the position sensor 115 of the mobile terminal 100. The surrounding image is provided to the image corrector 29 and corrected. When the correction is completed, the vehicle-side controller 30 transmits the corrected surrounding image or the surrounding image having the auxiliary line to the mobile terminal 100 through the vehicle-side communication unit 70.

On the other hand, when the vehicle side control unit 30 needs to be corrected by the image corrector 29, if the horizontal or vertical width of the surrounding image is different from the actual one by the correction, the horizontal width is narrower than the actual width. The vertical width is wider than it is actually provided to the second image synthesizing unit 26 so as to be synthesized with the surrounding images.

Referring to FIG. 4, a process of obtaining a surrounding image and providing the surrounding image to the mobile terminal 100 using the image providing system is described below.

Each camera 10 mounted on the vehicle photographs the surroundings of the vehicle in real time (S400), and the captured image is transmitted to the image processing apparatus 20 through each channel. When the output form of the image is an original image, the image from each camera 10 is removed and amplified by the image input unit 21 to generate a peripheral image in the original format. When the output form of the image is a top view image or a 3D image, the captured image is removed and amplified by the image input unit 21 and processed and synthesized through the image processing unit 23 and the first image synthesizing unit 25. A peripheral image in 3D format is generated (S405).

While the surrounding image is processed as described above, the vehicle-side controller 30 determines whether to operate the image corrector 29 based on the horizontal and vertical arrangement information of the portable terminal 100 provided from the portable terminal 100 (S410). ). When the horizontal and vertical arrangement of the portable terminal 100 is changed or is not a basic arrangement, the vehicle side controller 30 firstly uses the first image synthesizing unit 25 and the image input unit 21 as the second image synthesizing unit 26. The peripheral image from the image is transferred to the message that the image is different from the actual width or the vertical width to form a peripheral image having the message (S420).

Then, the vehicle control unit 30 operates the image correction unit 29 to correct the surrounding image having a message to match the display arrangement of the mobile terminal 100 (S425). The controller transmits the corrected peripheral image to the mobile terminal 100 through the communication unit 70 (S430), and the terminal side controller 130 of the mobile terminal 100 displays the corrected peripheral image of the mobile terminal 100. Displayed on the panel 110.

On the other hand, when the image correction is not necessary, the vehicle-side control unit 30 transmits the surrounding images from the first image synthesizing unit 25 and the image input unit 21 to the portable terminal 100 (S415).

When the vehicle attempts to park while displaying the surrounding image (S435), the vehicle-side control unit 30 operates the auxiliary line generator 50 to generate the parking auxiliary line (S440). The auxiliary line is transferred to the second image synthesizing unit 26. The second image synthesizing unit 26 forms a peripheral image having the auxiliary line by superimposing the parking auxiliary line on the original, top view, and 3D format surrounding images output from the image input unit 21 or the first image synthesizing unit 25. (S445).

Even while forming the surrounding image having the auxiliary line, the vehicle-side control unit 30 checks whether the display panel 110 provided from the portable terminal 100 is disposed in the horizontal and vertical directions (S450). If correction is required by changing the horizontal and vertical arrangement of the display panel 110, the vehicle-side controller 30 first sends a message to the second image synthesizing unit 26 that the image is different from the actual width or height. Transferring forms a peripheral image having an auxiliary line and a message (S455). Then, the vehicle controller 30 operates the image corrector 29 and transmits the surrounding image having the auxiliary line to the image corrector 29. The image corrector 29 corrects the surrounding image having the auxiliary line to match the arrangement of the display panel 110 (S460).

When the correction is completed, the vehicle-side control unit 30 transmits the surrounding image having the corrected auxiliary line to the mobile terminal 100 using the communication unit (S465), and displays the display panel 110 of the mobile terminal 100. do.

On the other hand, when the correction of the peripheral image having the auxiliary line is not necessary, the vehicle-side control unit 30 transmits the peripheral image having the auxiliary line generated by the second image synthesizing unit 26 to the mobile terminal 100 (S470). ).

The vehicle control unit 30 continuously monitors the horizontal and vertical arrangement of the display panel 110 of the mobile terminal 100, and when a change occurs in the vertical and horizontal arrangement, the image correction unit 29 is operated to display the surrounding image. Correct to match the arrangement of 110.

5 is a schematic block diagram of a vehicle surrounding image providing system using a mobile terminal according to another embodiment of the present invention.

The vehicle surrounding image providing system using the portable terminal 1100 of the present embodiment is similar to the vehicle surrounding image providing system of the embodiment of FIG. 1 in that the vehicle apparatus 1001 and the portable terminal 1100 are provided.

However, the vehicle surrounding image providing system of the present embodiment, the vehicle device 1001 is to take the image of the vehicle surroundings and transmit to the mobile terminal 1100, the mobile terminal 1100 to process the vehicle surrounding image in the display panel 1110 It is different from the vehicle surrounding image providing system of the embodiment shown in FIG.

As illustrated in FIG. 5, the portable terminal 1100 includes a display panel 1110, a position sensor 1115, a terminal side communication unit 1120, a terminal side control unit 1130, an image processing apparatus 1020, and an auxiliary line. The generator 1050 may include the LDWS 1140, the FCWS 1145, and the driver driving recorder 1150.

The on-vehicle device 1001 processes a photographed image mounted on a vehicle and provides the image to the mobile terminal 1100, and collects a plurality of cameras 1010, a vehicle side communication unit 1070, a vehicle side control unit 1030, and vehicle information collection. It may include a unit 1080.

Compared to the embodiment of FIG. 1, the portable terminal 1100 and the vehicle apparatus 1001 include the image processing apparatus 1020 and the auxiliary line generator 1050 among the components in the vehicle apparatus 1001. It is different that it was transferred to 1100. In addition, a vehicle information collecting unit 1080 is added to the vehicle apparatus 1001, and an LDWS 1140, an FCWS 1145, and a driver driving recorder 1150 are added to the portable terminal 1100.

Thus, the newly added components will be described, and the role of the other components is the same, and thus repeated descriptions for each component will be omitted.

However, the terminal side controller 1130 of the portable terminal 1100 and the vehicle side controller 1030 of the vehicle apparatus 1001 may have a role according to the transfer of the image processing apparatus 1020 and the auxiliary line generator 1050. Since it is changed, the terminal side control unit 1130 and the vehicle side control unit 1030 will be described again.

When an image photographed from each camera 1010 is input, the vehicle side controller 1030 of the vehicle apparatus 1001 transmits the image to the portable terminal 1100 through the vehicle side communication unit 1070. In this case, the vehicle-side control unit 1030 may sequentially transmit the images from each camera 1010 to the mobile terminal 1100, respectively, or combine the images from each camera 1010 into one image to carry the mobile terminal 1100. Can also be sent. That is, the vehicle side controller may combine the images received from the four channels connected to each camera 1010 and transmit the images through one channel. In this case, the transmission speed of the image may be improved than when the images received through the four channels are respectively transmitted to the mobile terminal 1100. The vehicle controller 1030 may transmit only the image input from one or three or less interest channels selected by the user through the portable terminal 1100 to the portable terminal 1100.

The vehicle information collecting unit 1080 may receive vehicle information from various electronic devices and sensors installed in the vehicle through a CAN (Control Area Network). CAN is the most common protocol used in vehicle communication. Vehicle information collected through CAN may include speed, steering angle, gear, ultrasonic sensor information, brake operation, Excel operation, and the like. The vehicle control unit 1030 transmits the vehicle information collected through the CAN to the mobile terminal 1100 to be displayed on the display panel 1110 of the mobile terminal 1100.

The terminal side controller 1130 processes the image provided from the vehicle apparatus 1001 in the image processing apparatus 1020 and operates the auxiliary line generator 1050 as needed to generate a vehicle auxiliary line to generate an image processing apparatus ( The image is superimposed on the surrounding image processed at 1020. In addition, the terminal controller 1130 operates the image corrector 1029 according to the horizontal and vertical arrangement of the portable terminal 1100 detected by the position sensor 1115 to match the horizontal and horizontal arrangement of the display panel 1110. Allow the image to be corrected.

Looking at how the terminal control unit 1130 controls the image processing device 1020 in more detail as follows.

The terminal control unit 1130 may process the original image and the first image processed by the image input unit 1021 according to the vertical or horizontal arrangement of the portable terminal 1100 detected by the position sensor 1115 when the vehicle is in normal driving. The top view image or the 3D image generated by the synthesis unit 1025 is provided to the image correction unit 1029 to be corrected.

Meanwhile, when the vehicle is parked, the terminal controller 1130 operates the auxiliary line generator 1050 to generate a parking auxiliary line, and transfers the generated parking auxiliary line to the second image synthesizing unit 1026. 'Nearby image with auxiliary line' is generated. In addition, the terminal-side control unit 1130 may display the surrounding image having the auxiliary line generated by the second image synthesizing unit 1026 according to the vertical or horizontal arrangement of the portable terminal 1100 detected by the position sensor 1115. Provided to the government 1029 and corrected. When the correction is completed, the terminal side controller 1130 displays the corrected peripheral image or the peripheral image having the auxiliary line on the display panel 1110.

Meanwhile, when the terminal side controller 1130 needs to be corrected by the image corrector 1029, when the horizontal or vertical width of the surrounding image is different from the actual image by the correction, the horizontal width is narrower than the actual width. The vertical width is wider than it is actually provided to the second image synthesizing unit 1026 so as to be synthesized with the surrounding images.

In addition, the terminal-side control unit 1130 is provided with the vehicle information on the speed, steering angle, gear, ultrasonic sensor information, brake operation, Excel operation, etc. provided from the vehicle device 1001 as the display panel 1110 Can be marked on. In addition, the terminal controller 1130 may display information on the vehicle state obtained by analyzing the vehicle information. For example, a warning signal may be displayed during sudden acceleration or sudden deceleration, which may be determined according to a brake or Excel operation.

Recently, programs such as Lane Departure Warning System (LDWS) 1140, which is a lane departure warning system, a Forward Collision Warning System (FCWS) 1145, and a driver driving recorder 1150 have been developed. In the present invention, such a program is installed in the mobile terminal 1100. The terminal controller 1130 may receive information from the LDWS 1140, the FCWS 1145, the driver driving recorder 1150, and display the information on the display panel 1110.

In addition, the terminal-side control unit 1130 may receive a lane departure warning, a front collision warning, and the like from the vehicle apparatus 1001 in real time and output the same through the display panel 1110 and the speaker.

On the other hand, the terminal side control unit 1130 for displaying the vehicle information from the vehicle apparatus 1001 and the information from the LDWS 1140, the FCWS 1145, the driver driving recorder 1150, and the like on the display panel 1110. ) Transmits vehicle information provided in real time to the second image synthesizing unit 1026 so as to overlap the surrounding image. Then, the surrounding image in which the vehicle information and the like are superimposed is displayed on the display panel 1110.

Referring to FIG. 6, a process of displaying and processing an image captured by the camera 1010 by the portable terminal 1100 using the image providing system having the above configuration will be described below.

Each camera 1010 mounted on the vehicle photographs the surroundings of the vehicle in real time, and the captured image is transmitted to the vehicle side controller 1030 through each channel. The vehicle control unit 1030 combines four channels of images input from each camera 1010 into one channel, or transmits only the image of the channel selected through the portable terminal 1100 to the portable terminal 1100 (S600). The vehicle side controller 1030 may also transmit the collected vehicle information, lane departure warning, forward collision warning, driver driving record, and the like to the mobile terminal 1100 (S605).

When the image is received by the portable terminal 1100, the terminal side controller 1130 transmits the image to the image input unit 1021 of the image processing apparatus 1020. When the output form of the image selected through the mobile terminal 1100 is the original image, the image input unit 1021 removes and amplifies the image provided from the in-vehicle device 1001 to generate the surrounding image in the original format. If the output form of the image selected through the mobile terminal 1100 is a top view image or a 3D image, the image removed and amplified by the image input unit 1021 may be an image processor 1023 and a first image synthesizer 1025. The image is processed and synthesized to generate a top view or a surrounding image in 3D format (S610). The generated surrounding image is transmitted to the second image synthesizing unit 1026 to superimpose vehicle information provided from the vehicle apparatus 1001 on the surrounding image (S615).

While the surrounding image is being processed, the terminal-side controller 1130 determines whether to operate the image corrector 1029 based on the horizontal and horizontal arrangement information of the mobile terminal 1100 detected by the position sensor 1115 ( S620). In addition, when the horizontal and vertical arrangement of the portable terminal 1100 is changed or is not a basic arrangement, the terminal-side controller 1130 may firstly, as the second image synthesizing unit 1026, the first image synthesizing unit 1025 and the image input unit ( A peripheral image from 1021 is provided, and together with this, a message that the image is different from the actual width or height is transmitted, thereby forming a 'ambient image having a message' (S625).

Thereafter, the terminal side controller 1130 operates the image corrector 1029 to correct the surrounding image having a message so as to match the display arrangement of the mobile terminal 1100 (S630). The terminal side controller 1130 displays the corrected surrounding image on the display panel 1110 (S635).

On the other hand, when the image correction is not necessary, the terminal-side control unit 1130 displays the peripheral image from the first image synthesizing unit 1025 and the image input unit 1021 on the display panel 1110.

When the vehicle attempts to park while displaying the surrounding image as described above, the terminal-side controller 1130 may determine that the vehicle attempts parking based on the vehicle information provided from the vehicle (S640). Then, the terminal side controller 1130 operates the auxiliary line generator 1050 to generate the parking auxiliary line (S645), and transmits the generated parking auxiliary line to the second image synthesizing unit 1026. The second image synthesizing unit 1026 forms a peripheral image having an auxiliary line by superimposing a parking auxiliary line on an original image, a top view, and a 3D format peripheral image output from the image input unit 1021 or the first image synthesizing unit 1025. (S650)

Even while forming the surrounding image having the auxiliary line in this way, the terminal-side control unit 1130 checks whether the display panel 1110 provided from the position sensor 1115 is disposed in the horizontal and vertical directions (S655). If correction is required by changing the horizontal and vertical arrangement of the display panel 1110, the terminal-side controller 1130 first sends a message to the second image synthesizing unit 1026 that the image is different from the actual width or height. Transferring forms a peripheral image having an auxiliary line and a message (S660). Then, the terminal controller 1130 operates the image corrector 1029 and transmits the surrounding image having the auxiliary line to the image corrector 1029. The image corrector 1029 corrects the surrounding image having the auxiliary line to match the arrangement of the display panel 1110 (S665). When the correction is completed, the terminal side controller 1130 displays the surrounding image having the corrected auxiliary line on the display panel 1110 (S670).

Meanwhile, when the correction of the surrounding image having the auxiliary line is not necessary, the terminal side controller 1130 transmits the surrounding image having the auxiliary line generated by the second image synthesizing unit 1026 to the mobile terminal 1100.

While the surrounding image is displayed on the display panel 1110 of the portable terminal 1100, the terminal controller 1130 provides various vehicle information provided from the on-vehicle device 1001 to the second image superimposing unit to overlap the surrounding image. The vehicle information may be displayed together on the display panel 1110.

7 is a configuration diagram of an image providing system employing a black box function according to the present invention.

The image providing system according to an exemplary embodiment disclosed in FIGS. 1 to 4 and the image providing system according to another exemplary embodiment disclosed in FIGS. 5 and 6 may include a black capable of uploading information to an external black box server 500. Has a box function.

The black box function may be performed by the in-vehicle devices 1 and 1001 or may be performed by the portable terminals 100 and 1100. In the image providing system disclosed in FIGS. 1 to 4, the image processing apparatus 1020 is provided in the vehicle apparatus 1 to provide the processed peripheral image to the mobile terminal 100. That is, since both the vehicle apparatus 1 and the portable terminal 100 have both the surrounding image and the vehicle information, one of the vehicle apparatus 1 and the portable terminal 100 communicates with an external black box server 500. Various vehicle information collected from the surrounding image and the vehicle can be transmitted. On the other hand, in the image providing system disclosed in FIGS. 5 and 6, the portable terminal 1100 processes and displays an image, and thus, the portable terminal 1100 transmits the surrounding image and vehicle information to the black box server 500. .

Therefore, in the image providing system disclosed in FIGS. 1 to 4, the vehicle-side communication unit 70 and / or the terminal-side communication unit 120 should support a communication method for communicating with an external black box server 500. . On the other hand, in the image providing system disclosed in FIGS. 5 and 6, only the terminal communication unit 1120 needs to support a communication method for communicating with an external black box server 500.

Accordingly, the image providing system of the present invention is capable of communicating with the external black box server 500, and when the real time or the communication with the server in the black box server 500 can be captured by the camera 1010 and processed the surrounding image and Vehicle information can be uploaded.

The image providing system according to the present invention may provide image information of a vehicle to the portable terminals 100 and 1100, and particularly, to fit the width and width of the display panels 110 and 1110 of the portable terminals 100 and 1100. The surrounding image can be corrected and displayed. Accordingly, since the user does not need to provide a separate navigation or equipment, the user can reduce the economic burden and promote user satisfaction. In addition, by providing an auxiliary parking line for guiding parking when the vehicle is superimposed on the surrounding image to the portable terminals 100 and 1100, the user can easily park the vehicle. In particular, when processing the image information in the mobile terminal 1100, by utilizing the high specification calculation function of the mobile terminal 1100, and by using the display panel 1110, the structure of the vehicle device 1001 can be simplified to reduce costs can do.

In addition, the vehicle information provided from the vehicle can be displayed on the portable terminal 1100 in real time, thereby helping the user to drive safely. In addition, by uploading the user's driving record, surrounding images, vehicle information, etc. to the external black box server 500 in real time, the user's driving status can be confirmed later and used as evidence of a contact accident.

Although the preferred embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

1, 1001: Vehicle device 10, 1010: Camera
20, 1020: image processing apparatus 21, 1021: image input unit
23, 1023: image processing unit 25, 1025: first image synthesizing unit
26, 1026: second image synthesizer 29, 1029: image correction unit
30, 1030: control unit 50, 1050: auxiliary line generating unit
100 and 1100: mobile terminal 110, 1110: display panel

Claims (18)

A camera mounted on the vehicle and configured to capture an image around the vehicle; And
And an image processing apparatus configured to process the images photographed and processed by the camera so as to correspond to a posture of a mobile terminal and a pre-selected viewpoint, to generate an image. And a portable terminal including an image processing apparatus configured to process an image of the vehicle surroundings taken by a camera mounted on the vehicle so as to correspond to the posture of the portable terminal and a preselected viewpoint. Provide system. The method according to claim 1 or 2,
An auxiliary line generator configured to generate a virtual parking auxiliary line for guiding parking according to the movement of the vehicle when the vehicle is parked;
And the image processing apparatus is configured to generate a surrounding image having a parking assistance line by superimposing the parking assistance line generated by the auxiliary line generator on an image photographed by the camera. The method of claim 3, wherein
The portable terminal further includes a position sensor for detecting the horizontal and vertical arrangement of the portable terminal;
The image processing apparatus is configured to resize the surrounding image or the parking assist line to correspond to the width or height of the display panel according to the width and height of the display panel of the portable terminal detected by the position sensor. Vehicle surrounding image providing system, characterized in that. The method of claim 3, wherein
The image processing apparatus,
And displaying a message indicating that the image displayed on the display panel is different from the actual image is superimposed on the image according to the size change of the surrounding image and the parking auxiliary line. The method of claim 3, wherein
The camera is provided with a plurality,
The image processing apparatus may provide an image captured by the camera to a display panel of the portable terminal as it is, or to provide a view of one of a top view image and a 3D image to a display panel of the portable terminal. A vehicle surrounding image providing system using a portable terminal. The method of claim 1,
The portable terminal is a vehicle surrounding image providing system using a portable terminal, characterized in that the wired or wireless communication with the image processing apparatus. The method according to claim 1 or 2,
A vehicle information collecting unit for collecting information on an operating state of the vehicle from electronic devices and sensors of the vehicle;
The vehicle surrounding image providing system using the mobile terminal, wherein the information collected by the vehicle information collecting unit is provided and displayed on the mobile terminal. The method of claim 2,
The portable terminal includes a vehicle such as a lane departure warning system (LDWS) that is a lane departure warning system, a forward collision warning system (FCWS) that is a forward collision warning device, and a driver driving recorder. Peripheral image providing system. The method of claim 9,
It is connectable to the portable terminal through a communication network, and provides a peripheral image of the vehicle processed by the image processing apparatus, information collected by the vehicle information collecting unit, and information from the LDWS, FCWS, driver driving recorder, and the like. Vehicle surrounding image providing system using a mobile terminal, characterized in that it further comprises a black box server for receiving and storing. Capturing an image around the vehicle using a camera;
Generating surrounding images by processing the images photographed by the camera to correspond to a posture of a mobile terminal and a pre-selected viewpoint; And
And displaying the image captured and processed by the camera on the portable terminal. The method of claim 11,
Generating a virtual parking assistance line for guiding parking according to the movement of the vehicle when the vehicle is parked; And
And superimposing the parking assistance line on the image photographed by the camera to generate a peripheral image having the parking assistance line. The method according to claim 11 or 12,
Detecting a change in the horizontal and vertical arrangement of the portable terminal;
Correcting the surrounding image or the parking assistance line according to the horizontal and vertical arrangement of the portable terminal; And
And displaying the corrected surrounding image or parking assistance line on a display panel of the portable terminal. The method of claim 13,
Superimposing a message indicating that the image displayed on the display panel is different from the actual image according to the size change of the surrounding image or the parking auxiliary line; Vehicle surrounding image providing method using a mobile terminal, characterized in that it further comprises. The method of claim 11,
Providing a plurality of cameras; And
Displaying images captured by the plurality of cameras on the display panel of the portable terminal as it is, or processing the image processing apparatus to have one of a top view image and a 3D image on the display panel of the portable terminal. Displaying a vehicle surrounding image using a mobile terminal, characterized in that it further comprises the step of displaying. The method of claim 11,
Collecting information about an operating state of the vehicle from electronic devices and sensors of the vehicle; And
And displaying the information on a display panel of the portable terminal. The method of claim 11,
Collecting information from a lane departure warning system (LDWS), a forward collision warning system (FCWS), a driver driving recorder, and the like mounted on the portable terminal;
Synthesizing the information into the surrounding image;
And displaying the surrounding image synthesized with the information on a display panel of the portable terminal. The method of claim 11,
Transmitting surrounding images processed by the image processing apparatus, information collected from electronic devices and sensors of the vehicle, and information collected from the LDWS, FCWS, driver driving recorder, etc. to an external black box server through a communication network; Vehicle surrounding image providing method using a mobile terminal, characterized in that it further comprises.

Images