JP2010183234A - Drawing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a drawing apparatus which enables a user to recognize an obstacle regardless of a displayed frame graphic. <P>SOLUTION: The drawing apparatus for processing the peripheral image of a vehicle captured by an imaging apparatus and preparing an image to be displayed on a display device includes: a view point conversion processing section for converting the image acquired from the imaging apparatus to an image viewed from a virtual view point at the upper part of the vehicle; a parking frame recognition section for detecting a parking frame from the image to which view point conversion processing is performed by image recognition; a recognized result evaluation section for calculating a score indicating the likelihood of the detected parking frame; a frame graphic preparation section for preparing the frame graphic indicating the parking frame on the basis of the calculated score; and an image composition section for combining the prepared frame graphic with the view-point-converted image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a drawing apparatus, and more particularly, to a drawing apparatus that processes a surrounding image of a vehicle captured by an imaging device and creates an image displayed on a display device.

  As one of parking assist devices incorporating a conventional drawing device, there has been proposed a parking assist device that creates an overhead image viewed from above the vehicle using one or more cameras provided around the vehicle. (For example, refer to Patent Document 1).

In the invention described in Patent Document 1, a figure indicating a parking frame is superimposed and displayed on the parking frame position of the overhead image so that the user can intuitively specify the position and direction of the target parking space. The target parking space is presented to the user.
JP 2007-230371 A (page 5-7, FIG. 1)

  In the conventional parking assistance device, a method of detecting a parking frame by image recognition is also proposed, but it is difficult to obtain a parking frame completely correctly by image recognition. If the frame figure is superimposed when the parking frame is not detected correctly, for example, if there is an obstacle such as a flap board in a pay parking lot, the user will be caught by the displayed frame figure. There was a problem that these obstacles could not be recognized.

  The present invention has been made to solve the conventional problems, and when the recognition rate of the currently displayed parking frame is low and a correct frame figure cannot be displayed, the current recognition rate is low for the user. Accordingly, an object of the present invention is to provide a drawing apparatus that can recognize an obstacle without being caught by a displayed frame figure.

  In order to solve the above-described conventional problems, the present invention is a drawing device that processes a surrounding image of a vehicle captured by an imaging device and creates an image displayed on the display device. A viewpoint conversion processing unit that converts an image viewed from a virtual viewpoint above the vehicle, a parking frame recognition unit that detects a parking frame by image recognition from the viewpoint-converted image, and a probability of the detected parking frame. A recognition result evaluation unit that calculates a score to be represented, a frame graphic generation unit that generates a frame graphic indicating a parking frame based on the calculated score, and an image that combines the generated frame graphic with a viewpoint-converted image And a synthesis unit.

  In addition, the frame graphic creating unit creates a frame graphic composed of a frame line with high transmittance when the score calculated by the recognition result evaluating unit is lower than a predetermined threshold.

  In addition, the frame graphic creating unit creates a frame graphic in which the area is filled by increasing the transmittance when the score calculated by the recognition result evaluating unit is lower than a predetermined threshold.

  In addition, the frame figure creation unit increases the transmittance for a region including a portion where the score calculated by the recognition result evaluation unit is lower than a predetermined threshold, and the transmittance for a region including a portion whose score is higher than the predetermined threshold. Create a frame figure that fills the area with a low.

  According to the drawing apparatus of the present invention, it is possible to convey the accuracy of the recognition rate of the parking frame to the user by evaluating the image recognition result of the parking frame and displaying the evaluation result in an intuitively understandable image. In addition, even when the recognition rate of the parking frame is low and a correct frame figure cannot be displayed, the user can recognize an obstacle without being caught by the displayed frame figure.

  Hereinafter, a drawing apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a hardware configuration diagram of a drawing apparatus Urnd according to an embodiment of the present invention. In FIG. 1, the drawing device Urnd includes a processor 1, a program memory 2, and a working area 3. The program memory 2 is typically a ROM (Read Only Memory). The program memory 2 stores a program PGa that defines the processing procedure of the processor 1. Further, the program memory 2 stores vehicle model data Dmd and frame graphic data Dfr described below.

  FIGS. 2A and 2B are diagrams for explaining the vehicle model data Dmd. FIG. 2A is a perspective view of an actual vehicle Vusr that rests on the road surface Frd. FIG. 2B is a view of the real vehicle Vusr shown in FIG. 2A as viewed from above (virtual viewpoint Pvt1). In the present embodiment, the vertical plane passing through the midpoint of the line connecting the rotation centers of the two front wheels and the midpoint between the two rear wheels in the actual vehicle Vusr in the straight traveling posture is referred to as a longitudinal center plane Fwc. A surface obtained by rotating the longitudinal center plane Fwc by 90 degrees about a vertical line passing through the center of the vehicle Vusr is referred to as a lateral center plane Flc. The vertical center plane Fwc and the horizontal center plane Flc are not illustrated in FIG. 2A for the convenience of illustration, but are illustrated in FIG. 2B using a one-dot chain line.

  2A and 2B show a three-dimensional coordinate system CSa having an X axis, a Y axis, and a Z axis. For convenience of illustration, the X-axis, Y-axis, and Z-axis of the three-dimensional coordinate system CSa are drawn at positions away from the actual vehicle Vusr. However, to simplify the explanation, the Z-axis is the vertical center plane Fwc. And the intersection line of the horizontal center plane Flc. The X axis is assumed to be equal to the intersection line of the lateral center plane Flc and the road surface Frd. The Y axis is assumed to be equal to the intersection line of the longitudinal center plane Fwc and the road surface Frd. The origin Oa of the three-dimensional coordinate system CSa is the intersection of the road surface Frd, the longitudinal center plane Fwc, and the lateral center plane Flc. Further, in FIGS. 2A and 2B, a virtual viewpoint Pvtl is depicted. In this embodiment, the virtual viewpoint Pvtl is preset above the actual vehicle Vusr, and more specifically, has a three-dimensional coordinate value (0, 0, zv).

  As shown in FIG. 3, the vehicle model data Dmd represents an image Smd of a vehicle obtained by reducing the actual vehicle Vusr projected on the road surface Frd as viewed from the virtual viewpoint Pvt1 with a predetermined reduction rate Fsc. Hereinafter, in this embodiment, what is represented by the vehicle model data Dmd is referred to as a vehicle model image Smd.

  In FIG. 1, a working area 3 is typically a RAM (Random Access Memory), and is used when the processor 1 executes a program PGa. The above drawing device Urnd is typically incorporated in the driving support device Uast. The driving support device Uast is installed in the actual vehicle Vusr, and besides the drawing device Urnd, the input device 4, the four imaging devices 5 to 8, the rudder angle sensor 9, the vehicle speed sensor 10, and the display device 11. And.

  The input device 4 is typically operated by a driver when the actual vehicle Vusr is parked. In response to this operation, the input device 4 generates a start instruction signal Istr and transmits it to the processor 1. In the present embodiment, the start instruction signal Istr is a signal for instructing the processor 1 to start execution of the program PGa.

  The imaging devices 5 to 8 are installed in the actual vehicle Vusr. Here, FIG. 4 is a diagram for explaining an installation example of the imaging devices 5 to 8. First, FIG. 4 depicts a view of an actual vehicle Vusr that rests on the road surface Frd from directly above.

  The first region R1 is the front side of the actual vehicle Vusr. Here, the front side means the forward direction side of the actual vehicle Vusr indicated by the arrow A with the lateral center plane Flc as a boundary. The second region R2 is on the right side of the actual vehicle Vusr. Here, the right side means the right side based on the forward direction of the actual vehicle Vusr indicated by the arrow A, with the longitudinal center plane Fwc as a boundary. Similarly, the third region R3 is on the rear side of the actual vehicle Vusr, and the fourth region R4 is on the left side of the actual vehicle Vusr.

  As shown in FIG. 4, the imaging device 5 is fixed to the top portion of the actual vehicle Vusr. The imaging device 5 captures an image representing a scene included in the first region R1 shown in FIG. 4 as a first surrounding image S1.

  Further, the imaging device 6 is fixed to the right side mirror with reference to the forward direction of the real vehicle Vusr indicated by the arrow A, and captures the second ambient image S2 representing the scene included in the second region R2 of FIG. . More specifically, the second surrounding image S2 includes a road surface Frd and represents a scene spreading to the right side of the actual vehicle Vusr.

  Further, the imaging device 7 is fixed to the rear end of the actual vehicle Vusr, and takes in the third surrounding image S3 representing the scene included in the third region R3. More specifically, the third surrounding image S3 includes a road surface Frd and represents a scene spreading behind the actual vehicle Vusr.

  Furthermore, the imaging device 8 is fixed to the left side mirror of the actual vehicle Vusr, and captures a fourth surrounding image S4 representing a scene included in the fourth region R4. More specifically, the fourth surrounding image S4 includes a road surface Frd and represents a scene spreading to the left side of the actual vehicle Vusr.

  The first surrounding image S1 to the fourth surrounding image S4 described above preferably represent not only the scenes of the first area R1 to the fourth area R4 but also a partial scene of each adjacent area. Including. For example, the first surrounding image S1 mainly represents a scene of the first region R1, but also represents a scene near the boundary with the first region R1 in the second region R2 and the fourth region R4.

  In FIG. 1, the steering angle sensor 9 detects the steering angle θ of the actual vehicle Vusr and transmits it to the processor 1. The steering angle θ means an angle at which the steering is rotated with reference to the initial position. The initial position generally means a steering position in a state where the steering is not turned off, that is, in a state where the actual vehicle Vusr is in a straight traveling posture. The vehicle speed sensor 10 is a sensor that detects the traveling speed of the actual vehicle Vusr, and is typically a wheel speed sensor that outputs a vehicle speed pulse. The display device 11 is typically a liquid crystal display.

  Next, functional blocks of the drawing apparatus Urnd will be described with reference to FIG. The drawing device Urnd receives a video signal acquired by the imaging devices 5 to 8 and converts it into a video viewed from the virtual viewpoint Pvtl, and a road surface Frd from the video converted by the viewpoint conversion processing unit 101. The parking frame recognition unit 102 for recognizing the parking frame depicted in FIG. 5 and the parking frame recognition score calculated from the matching between the parking frame recognized by the parking frame recognition unit 102 and the image converted by the viewpoint conversion processing unit 101 A recognition result evaluation unit 103, a frame graphic creation unit 104 that creates a frame graphic to be displayed on the display device 11 from the recognition score calculated by the recognition result evaluation unit 103, and a frame graphic created by the frame graphic creation unit 104 The image composition unit 105 is configured to generate an image synthesized at a predetermined position of the image converted by the conversion processing unit 101.

  Next, operation | movement of the above parking assistance apparatus Uast is demonstrated. The input device 4 is operated by the driver when the actual vehicle Vusr is parked. As a result, a start instruction signal Istr is transmitted from the input device 4 to the processor 1. The processor 1 starts executing the program PGa in response to the start instruction signal Istr. The start instruction signal Istr may be transmitted to the processor 1 upon detecting that the gear shift position of the actual vehicle Vusr is in the back range, in addition to the driver's operation on the input device 4.

  FIG. 6 is a flowchart showing a processing procedure of the processor 1 described in the program PGa. In FIG. 6, first, the viewpoint conversion processing unit 101 generates an imaging instruction signal Icpt and transmits it to all the imaging devices 5 to 8. The imaging instruction signal Icpt is a signal for instructing the imaging apparatuses 5 to 8 to perform imaging. In response to the reception instruction signal Icpt, each of the imaging devices 5 to 8 captures the first surrounding image S1 to the fourth surrounding image S4 described with reference to FIG. 4 and stores them in the working area 3 (step S601). ).

  Next, the viewpoint conversion processing unit 101 performs image processing on the first surrounding image S <b> 1 to the fourth surrounding image S <b> 4 stored in step S <b> 601, and performs viewpoint processing on a frame memory provided in advance in the working area 3. A converted image Scvt is created (step S602). The viewpoint conversion image Scvt is the same as the first surrounding image S1 to the fourth surrounding image S4 in that it represents a scene around the real vehicle Vusr. However, while the first surrounding image S1 to the fourth surrounding image S4 represent a scene viewed from the installation positions of the imaging devices 5 to 8, the viewpoint conversion image Sccv is viewed from the virtual viewpoint Pvtl shown in FIG. The scene around the actual vehicle Vusr is greatly different in that it is an image projected on the road surface Frd. Further, through the above-described image processing, the scene represented by the viewpoint conversion image Sccv is converted into an actual scene reduced at the reduction rate Fsc described with reference to FIG. In the processing in step S602 described above, the positions of the imaging devices 5 to 8 are known and the lens distortion of the first surrounding image S1 to the fourth surrounding image S4 is corrected. A method is used in which the images are viewed from Pvtl and joined together. Since this processing is a well-known technique, a detailed description thereof is omitted here.

  Next, in step S603, the parking frame recognition unit 102 recognizes the white line area drawn on the road surface Frd with respect to the viewpoint conversion image Sccv created in step S602 by using general image recognition, and the parking frame. Is detected. For example, it is assumed that the parking frame recognition unit 102 detects a white line 701 line segment AB, a white line 702 line segment CD, a white line 703 line segment EF, and a white line 704 line segment GH as shown in FIG. . The parking frame recognition unit 102 calculates the distance between each line segment and determines whether the distance is appropriate as the width of the parking frame corresponding to the vehicle width. In FIG. 7, the parking frame recognition unit 102 determines that the distance W1 between the line segments AC and the distance W3 between the line segments EG are appropriate as the width of the parking frame, and detects the parking frames 710 and 711. Since the distance between the line segments CE is shorter than the vehicle width, the parking frame recognition unit 102 determines that the CDFE region is not appropriate as a parking frame.

  In step S604, the recognition result evaluation unit 103 calculates a parking frame recognition result score for the parking frame recognized in step S603. Here, the score is a value representing the certainty of how much the extracted parking frame is correct as a parking frame. The score calculation method will be described later in detail.

  Next, in step S605, the recognition result evaluation unit 103 determines whether the score calculated in step S604 is equal to or greater than a predetermined threshold value. If the score is equal to or greater than the threshold value, the process proceeds to step S606. Proceed with the process. The frame graphic creating unit 104 creates a normal frame graphic in step S606, and creates a frame graphic corresponding to the score in step S607. Here, the frame graphic is a graphic for pointing the user to the parking frame on the display image of the display device 11. Then, the frame graphic creation unit 104 stores the created frame graphic in the program memory 2 as frame graphic data Dfr. The process of step S607 will be described in detail later.

  Next, in step S608, the image composition unit 105 creates a vehicle composite image Svhc. By the way, due to the installation positions of the imaging devices 5 to 8, the actual vehicle Vusr is hardly shown in the first surrounding image S1 to the fourth surrounding image S4. Therefore, as shown in FIG. 8A, the actual vehicle Vusr is not drawn in the region Rvhc where the actual vehicle Vusr must be present, in the viewpoint conversion image Sccv created in step 602. However, the viewpoint conversion image Scvt has the same virtual viewpoint Pvtl and reduction rate Fsc as the vehicle model image Smd (see FIG. 3). Accordingly, the shape of the region Rvhc matches the outer shape of the vehicle model image Smd. The image composition unit 105 reads the vehicle model data Dmd from the program memory 2 to the working area 3. Then, the vehicle model data Dmd on the working area 3 is processed, and the vehicle model image Smd is combined, that is, pasted, into the region Rvhc of the viewpoint conversion image Sccv. As a result, a vehicle composite image Svhc as shown in FIG. 8B is created (step S608).

  Next, in step S609, the image composition unit 105 reads the frame graphic data Dfr created in step S606 or S607 from the program memory 2 to the working area 3. Then, the frame graphic data Dfr on the working area 3 is processed, and the frame graphic is synthesized on the vehicle composite image Svhc to create the display image Sdsp. FIG. 9 is a diagram for explaining processing in which the image composition unit 105 creates the display image Sdsp. FIG. 9A shows a vehicle composite image Svhc obtained by synthesizing the vehicle model image Smd when the host vehicle 904 in the parking lot is viewed from the virtual viewpoint Pvtl. White line regions 901 and 902 indicating parking frames on the road surface Frd are shown. The vehicle 903 is parked in one of the parking frame spaces. FIG. 9B shows a display image Sdsp in which the frame graphic 905 is synthesized.

  Next, the image composition unit 105 transfers the created display image Sdsp to the display device 11. The display device 11 displays the transferred display image Sdsp.

  In step S609, the image composition unit 105 calculates a road surface prediction trajectory indicating how the actual vehicle Vusr will move on the road surface Frd and combines it with the display image Sdsp, and the combined image is displayed on the display image Sdsp. It is good. As a method for deriving a road surface prediction trajectory, first, the processor 1 generates a detection instruction signal Idtc for instructing the steering angle sensor 9 to detect the steering angle θ, and transmits the detection instruction signal Idtc to the steering angle sensor 9. 9 detects the steering angle θ in response to the detection instruction signal Idtc. The detected steering angle θ is stored in the working area 3. Next, based on the steering angle θ received this time, the processor 1 calculates a road surface prediction trajectory using, for example, an Ackerman model (two-wheel model). Since this derivation method is a well-known technique, a detailed description thereof is omitted here.

  Next, in step S610, if the processor 1 determines that parking is not completed (No determination), the process returns to step S601, and if it is determined that parking is completed (Yes determination), the series of processing ends. Note that the parking completion may be determined based on whether or not the shift position acquired from the shift sensor is at the parking position, or an input indicating that the parking is completed by providing a separate button or the like. You may determine by accepting.

  Next, the processing in step S604 in FIG. 6 will be described in detail with reference to the flowchart in FIG. In the process of step S603, since the white line region is recognized and the parking frame is detected, in step S604, the recognition result evaluation unit 103 calculates whether or not the detected parking frame fits well on the white line. And score.

  First, in step S1001, the recognition result evaluation unit 103 acquires a position where the parking frame is recognized. And the recognition result evaluation part 103 acquires RGB value with respect to each pixel of the viewpoint conversion image Scvt corresponding to the position of the acquired parking frame. For example, as shown in FIG. 11, if a parking frame 1103 indicated by a broken line is detected for the white line regions 1101 and 1102, each pixel on the straight line from the end point A to the end point B of the parking frame 1103 and the end point C An RGB value is acquired for each pixel on the straight line of the end point D. FIG. 12A shows an example of RGB values of each pixel on the straight line from the end point A to the end point B and each pixel on the straight line from the end point C to the end point D. The recognition result evaluation unit 103 calculates and holds the standard deviation of the acquired RGB values of each pixel.

  In step S1002, the recognition result evaluation unit 103 determines whether the standard deviation calculated in step S1001 is equal to or less than a predetermined threshold for the pixel to be evaluated. If the standard deviation is less than or equal to the threshold (Yes determination), the process proceeds to step S1003 assuming that the pixel may be a white line. If the standard deviation is greater than the threshold (No determination), the pixel is not a white line. Determination is made and the process proceeds to step S1005. Here, when all of the RGB values are equal, as shown in FIG. 12B, white ((R, G, B) = (255, 255, 255)) to gray ((R, G, B) = (0,0,0)). Therefore, a pixel having a small standard deviation of RGB values may be white.

  Next, in step S1003, the recognition result evaluation unit 103 determines whether or not the RGB value of the pixel to be evaluated is equal to or greater than a predetermined threshold value. Here, the average value of the RGB values may be used for comparison with the threshold value, and since it is already known in the determination in step S1002 that the pixel has a small standard deviation, it is the most among R, G, and B. You may compare with a threshold value using a big value. In the processing of step S1003, as shown in FIG. 12B, since the color is white (R, G, B) = (255, 255, 255), for example, the predetermined threshold is set to 200, and the evaluation target It is determined whether or not each pixel is white.

  In step S1003, the recognition result evaluation unit 103 advances the process to step S1004 when the RGB value of the pixel to be evaluated is greater than or equal to a predetermined threshold (Yes determination), and if smaller than the predetermined threshold (No determination), the step The process proceeds to S1005.

  In step S1004, the recognition result evaluation unit 103 turns on the flag of the pixel (flag = 1) because the pixel to be evaluated corresponds to the white line. In step S1005, the recognition result evaluation unit 103 sets the flag of the pixel to off (flag = 0) because the pixel to be evaluated does not correspond to the white line.

  Next, in step S1006, the recognition result evaluation unit 103 determines whether or not processing has been performed for all the pixels for which RGB values have been acquired in step S1001, and if processing has been completed for all pixels, the processing proceeds to step S1007 and ends. If not, the process returns to step S1002 with the next pixel as the pixel to be evaluated.

  Finally, in step S1007, the recognition result evaluation unit 103 calculates the number of pixels for which the flag is on (1) among all the pixels, and holds the ratio with respect to the total number of pixels as a score. Note that the processing in FIG. 10 is not limited to the above method, and any method may be used as long as the matching between the detected parking frame and the actual viewpoint-converted image Scvt can be calculated.

  Next, the processing in steps S606 and S607 in FIG. 6 will be described in detail with reference to FIG. Here, as a frame figure created in the processes of steps S606 and S607, only a frame line indicating a parking frame is created. FIG. 13A shows a case where there is no obstacle on the white line regions 1301 and 1302, and FIG. 13B shows a case where there is some obstacle 1310 on the white line region 1301.

  As shown in FIG. 13A, when there are no obstacles on the white line regions 1301 and 1302, the score shows a high value, so that the frame graphic creation unit 104 converts the frame graphic 1303 into a normal uniform pattern in step S606. Create with borders with low color and low transmittance. As shown in FIG. 13B, when there is an obstacle 1310 on the white line area 1301, the score shows a low value, so that the frame graphic creation unit 104 moves the frame graphic 1304 from the normal frame line in step S607. Change the color and create a frame with a high transmittance. Here, the high transmittance means that the obstacle under the combined portion can be easily recognized even on the display image Sdsp in which the frame graphic 1304 is combined with the vehicle combined image Svhc in step S609 in FIG. Moreover, the transmittance is such that the parking frame can be recognized by the frame line. By increasing the transmittance, it can be seen that the recognition rate of the image recognition is poor, and the user looks at the screen of the display device 11 more carefully without being caught by the displayed frame line. In addition, since the transmittance of the frame line is high, the user can recognize the obstacle 1301. Here, instead of increasing the transmittance of the entire frame line of the frame figure 1304, only the portion where the flag of each pixel of the viewpoint conversion image Scvt corresponding to the position of the parking frame is turned off (that is, the obstacle) It is also possible to increase the transmittance or not to create a frame line.

  Next, a case where the frame graphic creation unit 104 creates not only a frame line indicating a parking frame but also a frame graphic filled in the parking frame will be described with reference to FIG. FIG. 14A shows a case where there are no obstacles on the white line regions 1401 and 1402, and FIG. 14B shows a case where a flap plate 1410 exists as an obstacle on the white line region 1401. In the case of FIG. 14A, the score shows a high value as in FIG. 13A. Therefore, in step S606, the frame figure creation unit 104 reduces the transmittance in the parking frame with a normal uniform color. A frame figure 1403 is created.

  In the case of FIG. 14B, since the score shows a low value, the area 1404 without the flap plate 1410 is created with a normal uniform color and a low-permeability frame figure as in FIG. 14A. In a region 1405 where the flap plate 1410 is provided, a frame figure having a high transmittance or a transparent figure is used. Here, the high transmittance means that the obstacle under the combined portion can be easily recognized on the display image Sdsp in which the frame graphic 1405 is combined with the vehicle combined image Svhc in step S609 in FIG. Moreover, the transmittance is such that the parking frame can be recognized by the frame figure. Since the parking area is displayed with a frame figure that fills the inside of the parking frame, it is easier to recognize the parking area than the frame figure that displays only the frame line, and even if there is an obstacle, the user can recognize the obstacle quickly. . Further, here, only the area 1405 with the flap plate 1410 is made a frame figure with high transmittance or transparent, but the entire parking area may be made a solid figure with high transmittance. By increasing the transmittance, it can be seen that the recognition rate of the image recognition is poor, and the user looks at the screen of the display device 11 more carefully without being caught by the displayed frame figure. And since the transmittance | permeability of a frame figure is high, the user can recognize the flap board 1410. FIG.

  In the above-described embodiment, the recognition rate of white line image recognition due to the presence or absence of an obstacle has been described, but the same processing is performed even when the recognition rate of image recognition is low due to white line fading or the like. Even if the frame figure is drawn out of the actual parking frame because the recognition rate of image recognition is low, by increasing the transmittance of the frame figure, the user knows that the recognition rate of image recognition is bad, The screen of the display device 11 is viewed more carefully without being caught by the displayed frame figure. And since the transmittance | permeability of a frame figure is high, the user can recognize that the frame figure is drawn and shifted from an actual parking frame.

  In the above embodiment, the imaging devices 5 to 8 are mounted on the actual vehicle. However, the number of imaging devices may be one or more. Since the vehicle is often parked in the back, it can be similarly applied to a single imaging device by creating an image obtained by changing the viewpoint of the image of the single imaging device attached to the rear end of the actual vehicle.

  In the above embodiment, the program PGa is stored in the drawing device Urnd. However, the present invention is not limited to this, and the program PGa may be distributed in a state where it is recorded on a recording medium typified by a CD-ROM, or may be distributed through a communication network typified by the Internet.

  As described above, the drawing device according to the present invention can notify the accuracy of the recognition rate of the parking frame to the user, and even if the recognition rate of the parking frame is low and a correct frame figure cannot be displayed, the user can display it. The present invention has the effect of being able to recognize an obstacle without being caught by the frame figure being used, and is useful as a vehicle surrounding monitoring system or the like that allows a vehicle occupant such as a parking assistance device to recognize the situation around the vehicle.

The hardware block diagram of the drawing apparatus Urnd which concerns on embodiment of this invention It is a figure for demonstrating vehicle model data Dmd, (a) The perspective view of the real vehicle Vusr which rests on the road surface Frd, (b) The figure which looked at the real vehicle Vusr from the upper direction (virtual viewpoint Pvt1) The figure which shows vehicle model image Smd The figure which shows the area | region of the surrounding image taken in with each imaging device Functional block diagram of the drawing apparatus Urnd according to the embodiment of the present invention The flowchart which shows the process sequence of the processor 1 described in the program PGa The figure for demonstrating the process which the parking frame recognition part 102 detects a parking frame in FIG.6 S603. FIGS. 6A and 6B are diagrams for explaining the process of step S608 in FIG. 6; (a) a view showing the viewpoint conversion image Scvt created in step 602; and (b) a view in which the vehicle model image Smd is combined with the viewpoint conversion image Scvt. FIG. 5 is a diagram for explaining processing of creating a display image Sdsp by the image composition unit 105, (a) a diagram showing a vehicle composite image Svhc, and (b) a diagram showing a display image Sdsp with a frame graphic 905 synthesized. The flowchart which shows the detail of a process of step S604 of FIG. The figure for demonstrating the process of step S1001 of FIG. FIG. 11 is a diagram for explaining the processing of step S1001 in FIG. 10, (a) a diagram showing an example of RGB values, and (b) a diagram for explaining RGB values. FIGS. 6A and 6B are diagrams for explaining the processing of steps S606 and S607 in FIG. 6, where FIG. 6A shows a frame figure when there is no obstacle on the white line area, and FIG. Figure showing shape FIGS. 6A and 6B are diagrams for explaining the processing of steps S606 and S607 in FIG. 6, where FIG. 6A shows a frame figure when there is no obstacle on the white line area, and FIG. Figure showing shape

DESCRIPTION OF SYMBOLS 1 Processor 2 Program memory 3 Working area 4 Input device 5, 6, 7, 8 Imaging device 11 Display apparatus 101 View point conversion process part 102 Parking frame recognition part 103 Recognition result evaluation part 104 Frame figure preparation part 105 Image composition part 901,902 1101, 1102, 1301, 1302, 1401, 1402 White line area 905, 1303, 1304, 1403 Frame figure

Claims (4)

A drawing device that processes an ambient image of a vehicle captured by an imaging device and creates an image to be displayed on a display device,
A viewpoint conversion processing unit that converts an image acquired from the imaging device into an image viewed from a virtual viewpoint above the vehicle;
A parking frame recognition unit for detecting a parking frame by image recognition from the viewpoint-converted image;
A recognition result evaluation unit that calculates a score representing the likelihood of the detected parking frame;
Based on the calculated score, a frame graphic creating unit that creates a frame graphic indicating a parking frame;
A drawing apparatus comprising: an image composition unit for compositing the created frame figure with the viewpoint-converted image. The drawing apparatus according to claim 1, wherein the frame graphic creating unit creates a frame graphic composed of a frame line having a high transmittance when the score calculated by the recognition result evaluating unit is lower than a predetermined threshold. . 2. The frame graphic creating unit creates a frame graphic in which a region is filled by increasing the transmittance when the score calculated by the recognition result evaluating unit is lower than a predetermined threshold. Drawing device. The frame figure creation unit increases the transmittance for a region including a portion where the score calculated by the recognition result evaluation unit is lower than a predetermined threshold, and transmits the region for a region including a portion where the score is higher than a predetermined threshold. The drawing apparatus according to claim 1, wherein a frame figure in which a region is filled at a low rate is created.