JP2005196423A - Image display method, image display system and image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display method, an image display system and an image processor constituting the image display system for processing and displaying image data picked up by an image pickup device according to the image pickup region of the image pickup device when the image pickup device is additionally loaded on a vehicle or when the loading position of the image pickup device is changed. <P>SOLUTION: Standard patterns (P101, P102 and P103) predetermined according to the image pickup region of a video camera are stored, image data picked up with video cameras 1, 2, 3 and 4 are compared with the standard patterns (P101, P102 and P103), and the standard pattern approximating to the image data is decided so that the image pickup regions picked up by the video cameras 1, 2, 3 and 4 can be decided. Image processing for carrying out visual point transformation such as panorama display or bird's eye view display is performed to the image data picked up by the video cameras 1, 2, 3 and 4 according to the decided image pickup region, and the image data after the image processing are displayed on a display device 6. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image display method, an image display system, and an image processing apparatus that perform image processing on image data captured by an image capturing apparatus that is installed in a vehicle and captures the outside world, and display the image processed image data. .

  The view from the driver's seat of a vehicle such as an automobile is obstructed by the vehicle body excluding the window, even if the window is wide, and a blind spot is generated. For this reason, a video camera is arranged at an appropriate location on the vehicle body, the outside of the vehicle is imaged, image processing is performed on the captured image data, and an image that is easy for the driver to view is displayed on a display device in the vehicle to drive There is a system to support the elderly.

  For example, a video camera for right / left turn monitoring is mounted on the front of the vehicle body, and a plurality of video cameras such as a video camera for rear monitoring are mounted on the rear. A system has been proposed in which images captured by the above operation are switched and displayed on a driver's seat display device, and a plurality of images are combined and displayed.

  In addition, the image captured by the video camera is converted into an image so as to be easily seen by the driver and displayed on the display device. For example, there is a technique in which image data captured by a video camera for rear monitoring is subjected to coordinate conversion processing for video viewpoint conversion, and the converted image data is displayed on a display device as a bird's eye view.

Furthermore, when imaging a blind spot area to the driver and providing information on the area, providing a lot of information at the same time is troublesome for the driver and the driver's operation becomes complicated. Therefore, a method for selecting and providing information necessary for the driver in accordance with the situation of the vehicle has been proposed (see Patent Document 1).
JP 2000-238594 A

However, since the method of Patent Document 1 uses an image processing control program predetermined for each vehicle corresponding to the number of video cameras mounted on the vehicle or the mounting position of the vehicle, it is already mounted on the vehicle. When another video camera is additionally mounted on the video camera, it is necessary to use an image processing control program suitable for the additional mounted video camera when performing image processing on the image data captured by the additional mounted video camera. It was complicated.
In addition, when the video camera mounted on the vehicle is changed to another mounting position of the vehicle, it is necessary to perform image processing such as bird's-eye display processing according to the mounting position of the vehicle. It is necessary to use an image processing control program corresponding to the processing, which is complicated.

  The present invention has been made in view of such circumstances, and a pattern such as luminance or chromaticity is extracted from image data captured by the imaging apparatus, and is stored in advance according to the extracted pattern and the imaging area of the imaging apparatus. By comparing the measured standard pattern and determining the imaging region corresponding to the standard pattern that approximates the pattern, from the image data captured by the additional mounted imaging device or the imaging device whose mounting position has been changed. Since the imaging area of the imaging device can be determined and image processing corresponding to the imaging area can be performed, there is no need to use a predetermined image processing control program for each vehicle, and according to the imaging area of the imaging device. To provide an image display method, an image display system for processing and displaying image data captured by the imaging device, and an image processing device constituting the image display system. For the purpose.

  Another object of the present invention is to receive the standard pattern from the storage device storing the standard pattern and compare the pattern extracted from the image data captured by the imaging device with the received standard pattern. Since the imaging region of the imaging device can be determined from the image data captured by the additionally mounted imaging device or the imaging device whose mounting position has been changed, and image processing according to the imaging region can be performed. It is not necessary to use a predetermined image processing control program for each image, and it is not necessary to store the standard pattern in the image processing apparatus. Based on the received standard pattern, the image data is captured from the image data captured by the imaging apparatus. An image that can determine the imaging region of the device, and that processes and displays image data captured by the imaging device according to the imaging region of the imaging device How to Display, and to provide an image display system.

  Another object of the present invention is to transmit image data picked up by an image pickup device to a processing device having storage means for storing the standard pattern, and the processing device uses the received image data such as brightness or chromaticity. Added by extracting the pattern, comparing the extracted pattern with the stored standard pattern, determining the imaging area corresponding to the standard pattern that approximates the extracted pattern, and transmitting the determination result to the image processing device Since it is possible to determine the imaging region of the imaging device from the image data captured by the mounted imaging device or the imaging device whose mounting position has been changed and perform image processing according to the imaging region, for each vehicle Image display that processes and displays image data captured by the imaging device according to the imaging area of the imaging device without using a predetermined image processing control program Law, is to provide an image processing apparatus constituting the image display system, and the image display system.

  The image display method according to the present invention is an image display that displays image data processed by an image processing device that performs image processing according to an imaging region of the imaging device, image data captured by an imaging device that captures the outside of a vehicle. In the method, the image processing device stores a standard pattern predetermined according to an imaging region of the imaging device, compares the image data captured by the imaging device with the standard pattern, and determines the imaging region of the imaging device. It is characterized by determining.

  The image display method according to the present invention displays image data processed by an image processing apparatus that performs image processing according to an imaging region of the imaging apparatus, with respect to image data captured by an imaging apparatus that captures the outside of the vehicle. In the image display method, the image processing device receives the standard pattern from a storage device that stores a predetermined standard pattern according to an imaging region of the imaging device, stores the received standard pattern, and the imaging device captures an image. The imaging area of the imaging apparatus is determined by comparing the obtained image data with the stored standard pattern.

  Further, in the image display method according to the present invention, the image processing device calculates the number of pixels having the same luminance or chromaticity in the luminance or chromaticity pattern extracted from the image data captured by the imaging device and the standard pattern. If the count value exceeds a predetermined threshold according to the imaging area of the imaging device, the imaging area of the imaging device that captured the image data is determined to be an imaging area corresponding to the threshold value. It is characterized by.

  In the image display method according to the present invention, the image processing device calculates a distance between the luminance or chromaticity pattern extracted from the image data captured by the imaging device and the standard pattern, and the calculated value is determined in advance. If it is smaller than the threshold value, it is determined that the imaging area of the imaging device that has captured the image data is an imaging area corresponding to the standard pattern.

  The image display method according to the present invention displays image data processed by an image processing apparatus that performs image processing according to an imaging region of the imaging apparatus, with respect to image data captured by an imaging apparatus that captures the outside of the vehicle. In the image display method, the image processing apparatus transmits the image data captured by the imaging apparatus to a processing apparatus having a storage unit that stores a standard pattern predetermined according to an imaging area of the imaging apparatus, and the processing The apparatus receives the transmitted image data, compares the received image data with the standard pattern, determines an imaging region of the imaging apparatus that captured the image data, and transmits a determination result to the image processing apparatus. The image processing apparatus further receives the determination result.

  In the image display method according to the present invention, the processing device counts the number of pixels having the same luminance or chromaticity in the luminance or chromaticity pattern extracted from the image data captured by the imaging device and the standard pattern. When the count value exceeds a predetermined threshold according to the imaging area of the imaging device, it is determined that the imaging area of the imaging device that captured the image data is an imaging area corresponding to the threshold. Features.

  In the image display method according to the present invention, the processing device calculates a distance between a luminance or chromaticity pattern extracted from image data captured by the imaging device and the standard pattern, and a calculated value is predetermined. When smaller than a threshold value, it is determined that an imaging area of an imaging apparatus that has captured the image data is an imaging area corresponding to the standard pattern.

  In addition, an image display system according to the present invention includes an imaging device that captures an external environment of a vehicle, an image processing device that performs image processing on image data captured by the imaging device in accordance with an imaging region of the imaging device, and the image In an image display system including a display device that displays image data processed by a processing device, the image processing device includes a storage unit that stores a standard pattern that is predetermined according to an imaging region of the imaging device, and an imaging device. The image processing apparatus includes a determination unit that compares the captured image data with the standard pattern to determine an imaging region of the imaging apparatus.

  In addition, an image display system according to the present invention includes an imaging device that captures an external environment of a vehicle, an image processing device that performs image processing on image data captured by the imaging device in accordance with an imaging region of the imaging device, and the image In an image display system including a display device that displays image data processed by a processing device, the image processing device stores the standard pattern from a storage device that stores a standard pattern predetermined according to an imaging region of the imaging device. Receiving means for receiving, storage means for storing the received standard pattern, and determination means for comparing the image data captured by the imaging apparatus and the standard pattern to determine the imaging area of the imaging apparatus. Features.

  In addition, an image display system according to the present invention includes an imaging device that captures an external environment of a vehicle, an image processing device that performs image processing on image data captured by the imaging device in accordance with an imaging region of the imaging device, and the image In an image display system including a display device that displays image data processed by a processing device, the image processing device uses image data captured by the imaging device as a standard pattern determined in advance according to an imaging region of the imaging device. Transmission means for transmitting to a processing device having storage means for storing, wherein the processing device compares the received image data with the standard pattern, receiving means for receiving the transmitted image data, and the image data Determination means for determining the imaging region of the imaging apparatus that has captured the image, and transmission means for transmitting the determination result determined by the determination means to the image processing apparatus, Processing apparatus further characterized by comprising a receiving means for receiving the determination result.

  In the image display system according to the present invention, the determination unit counts the number of pixels in which the luminance or chromaticity pattern and the standard pattern have the same luminance or chromaticity extracted from the image data captured by the imaging device. When the count value exceeds a predetermined threshold according to the imaging area of the imaging device, it is determined that the imaging area of the imaging device that captured the image data is an imaging area corresponding to the threshold. Features.

  In the image display system according to the present invention, the determination unit calculates a distance between the luminance or chromaticity pattern extracted from the image data captured by the imaging device and the standard pattern, and the calculated value is predetermined. When smaller than a threshold value, it is determined that an imaging area of an imaging apparatus that has captured the image data is an imaging area corresponding to the standard pattern.

  The image processing apparatus according to the present invention includes a storage unit that stores a predetermined standard pattern and a comparison that compares the captured image data with the standard pattern in an image processing apparatus that performs image processing on the captured image data. The captured image data is subjected to image processing on the basis of the result of comparison between the means and the comparison means.

In the first invention, the eighth invention and the thirteenth invention, a standard pattern predetermined according to the imaging region of the imaging device is stored, and the image data captured by the imaging device and the standard pattern are stored. And a standard pattern approximate to the image data can be determined to determine the imaging region captured by the imaging device. For this reason, when another imaging device is additionally mounted on the imaging device mounted on the vehicle, the imaging region captured by the imaging device is determined based on the image data captured by the additionally mounted imaging device. Therefore, it is not necessary to use a predetermined image processing control program, and image data subjected to image processing can be displayed according to the imaging region of the imaging device mounted on the vehicle.
Further, when changing the imaging device mounted on the vehicle to another mounting position of the vehicle, the imaging area captured by the imaging device based on the image data captured by the imaging device changed to another mounting position Therefore, it is not necessary to use a predetermined image processing control program, and image data subjected to image processing can be displayed according to the imaging region of the imaging device mounted on the vehicle.

In the second and ninth aspects of the invention, the standard pattern is received from a storage device that stores a standard pattern predetermined according to the imaging area of the imaging device, and the image data captured by the imaging device By comparing the received standard pattern and determining a standard pattern that approximates the image data, it is possible to determine the imaging region captured by the imaging device. For this reason, when another imaging device is additionally mounted on the imaging device mounted on the vehicle, the imaging region captured by the imaging device is determined based on image data captured by the additionally mounted imaging device. It is not necessary to use a predetermined image processing control program, and it is not necessary to store a standard pattern in the image processing device, and image processing is performed according to the imaging area of the imaging device mounted on the vehicle. Displayed image data can be displayed.
In addition, when changing the imaging device mounted on the vehicle to another mounting position of the vehicle, the imaging area captured by the imaging device based on image data captured by the imaging device changed to another mounting position It is not necessary to use a predetermined image processing control program, and it is not necessary to store a standard pattern in the image processing device, depending on the imaging area of the imaging device mounted on the vehicle, Image processed image data can be displayed.

In the fifth and tenth inventions, the image processing device transmits image data captured by the imaging device to the processing device, and the processing device receives the received image data and a predetermined standard pattern. , The imaging region of the imaging device is determined, the determination result is transmitted to the image processing device, and the image processing device can receive the determination result. For this reason, when another imaging device is additionally mounted on the imaging device mounted on the vehicle, the processing device determines the imaging region based on image data captured by the additionally mounted imaging device. The determination result can be transmitted to the image processing apparatus, and there is no need to use a predetermined image processing control program, and there is no need to provide a determination means for determining an imaging area for each vehicle. The image data subjected to image processing can be displayed in accordance with the imaging region of the imaging device that has been selected.
Further, when the imaging device mounted on the vehicle is changed to another mounting position of the vehicle, the processing device determines the imaging area based on image data captured by the imaging device changed to another mounting position. Thus, the determination result can be transmitted to the image processing apparatus, and there is no need to use a predetermined image processing control program, and there is no need to provide a determination means for determining an imaging region for each vehicle. The image data subjected to image processing can be displayed in accordance with the imaging region of the imaging device mounted on the.

  In the third invention, the sixth invention, and the eleventh invention, the number of pixels having the same luminance or chromaticity in the extracted luminance or chromaticity pattern and the standard pattern is counted, The imaging area of the imaging device can be determined by comparing the threshold corresponding to the imaging area of the imaging device.

  In the fourth, seventh and twelfth inventions, the distance between the extracted luminance or chromaticity pattern and the standard pattern is calculated, and the calculated value is compared with a predetermined threshold value. Thereby, the imaging region of the imaging device can be determined.

  In the present invention, the imaging region of the mounted imaging device is determined based on the image data captured by the imaging device, and image processing is performed on the image data captured by the imaging device based on the determination result. In order to display image data that has been subjected to image processing, even when an additional imaging device is mounted on the vehicle, or when the imaging device mounted on the vehicle is changed to another mounting position, it is determined in advance. The image processing control program need not be used, and image processing of the image data captured by the image capturing apparatus can be performed for each image capturing apparatus, and the image processed image data can be displayed.

  In the present invention, a standard pattern for determining an imaging region of the imaging device is stored in a storage device, and the standard pattern is received from the storage device as necessary, so that the image processing device receives the standard pattern. There is no need to memorize a standard pattern. For this reason, it is not necessary to change the image processing device in order to add various standard image patterns corresponding to the imaging device, and for the new imaging device, the imaging is performed according to the imaging area of the imaging device. Image processing can be performed on image data captured by the apparatus, and the image processed image data can be displayed.

  In the present invention, the image processing device is provided with the determination means for transmitting the image data captured by the image capturing device to the processing device so that the processing device determines the image capturing area of the image capturing device. The imaging region of the imaging device can be easily determined even for a new imaging device, and image processing is performed on image data captured by the imaging device according to the imaging region of the imaging device. The image data after image processing can be displayed.

Embodiment 1
FIG. 1 is a schematic diagram showing an outline of an image display system according to the present invention. In the figure, 1, 2, 3, and 4 are video cameras that are imaging devices mounted on a vehicle, 1 is a video camera for omnidirectional monitoring, 2 is a video camera for backward monitoring, and 3 is monitoring for turning left and right. 4 is a video camera for forward monitoring.

  The omnidirectional video camera 1 is mounted below the left or right door mirror of the vehicle according to the position of the steering wheel (right or left side of the vehicle), and the optical axis of the lens is installed vertically upward. A video camera is provided on the upper side of the video camera and has a mirror surface on the outer periphery of a cone shape whose diameter is increased in the upward direction of the optical axis. The light from all around 360 degrees is reflected by the mirror surface and converges on the lens of the video camera, so that the video camera 1 for omnidirectional monitoring can take an image of the outside world in 360 degrees, It is used to secure the front view and the immediately preceding side driving view.

  The rear-view video camera 2 is a video camera provided with a wide-angle lens, and is mounted in the center of a bumper at the rear of the vehicle for imaging the rear of the vehicle. The optical axis of the wide-angle lens is It is arranged in a substantially horizontal direction. The rear-view video camera 2 is used to secure a rear view when the vehicle is moved backward.

  The left and right turn monitoring video camera 3 is mounted in the center of the front bumper of the vehicle to capture the front left and right direction of the vehicle, and the optical axis of the video camera lens is arranged in a substantially horizontal direction in front of the vehicle. ing. A pair of prisms in the horizontal direction across the optical axis is disposed so as to converge light from the horizontal direction of the vehicle onto the lens. The video camera 3 for monitoring the right / left turn is used to check a vehicle approaching from the left and right at an intersection with poor visibility while the vehicle is traveling.

  The video camera 4 for front monitoring is mounted on the front window portion to image the front of the vehicle, and is used for monitoring the traveling direction of the vehicle. As described above, the video cameras 1, 2, 3, and 4 mounted on the vehicle have different imaging areas (front, right side, left and right, rear, and the like) depending on the mounting location in the vehicle, and the vehicle depends on the imaging area. The video necessary for driving is acquired.

  The video cameras 1, 2, 3, and 4 are connected to an in-vehicle LAN communication line 30 that conforms to IEEE 1394, and are connected to the image processing device 5 via relays 7 and 7. In addition, a display device 6 having an operation unit is connected to the image processing device 5. Image data captured by the video cameras 1, 2, 3, and 4 is output to the image processing device 5, and each of the video cameras 1, 2, 3, The image processing corresponding to the four imaging regions is performed and displayed on the display device 6 to assist the driver in driving. Further, it is possible to switch the images captured by the respective video cameras 1, 2, 3, 4 and display them on the display device 6 by the operation of the driver.

  FIG. 2 is a block diagram showing the configuration of the video camera 1. In the figure, reference numeral 11 denotes an image reading unit, which is provided with a CCD (Charge Coupled Device) for converting an optical signal into an electric signal, and RGB (R: red, G: green, B: blue). The read RGB signal is output to the signal processing unit 12 via the internal bus 17.

  The signal processing unit 12 converts RGB analog signals input from the image reading unit 11 into RGB digital signals, processing for removing various distortions generated in the optical system, low-frequency noise removal processing, gamma characteristics Perform correction processing to correct Further, the RGB signal is converted into a YUV (Y: luminance, U, V: color difference) signal, and the converted YUV signal is temporarily stored in the image memory 16 as image data.

  The interface unit 13 communicates with the image processing device 5 via the LAN communication line 30 and outputs image data stored in the image memory 16 to the image processing device 5 in accordance with a command transmitted from the image processing device 5. Control, conversion of the transfer rate according to the resolution of the image captured by the video camera 1, creation of packet data from the image data, and the like. Further, the interface unit 13 outputs the image data stored in the image memory 16 to the image processing device 5 under the control of the control unit 14.

The control unit 14 controls processing of the image reading unit 11, the signal processing unit 12, and the interface unit 13 by executing a program stored in the storage unit 15. The video cameras 2, 3, and 4 have the same configuration.
The conventional image display system has the same configuration as that described above.

  FIG. 3 is a block diagram illustrating a configuration of the image processing apparatus 5. In the figure, reference numeral 51 denotes an interface unit that transfers instructions to the video cameras 1, 2, 3, and 4, transfers image data from the video cameras 1, 2, 3, 4, and the like. The image is output to the image memory 52.

  The image memory 52 temporarily stores the image data for each of the video cameras 1, 2, 3, 4 input from the interface unit 51 in units of frames.

  The image processing unit 53 reads one frame of image data stored in the image memory 52, extracts a luminance or chromaticity pattern from the read image data, and corresponds to the extracted pattern and an imaging area for each video camera. The image area of the video camera 1, 2, 3, 4 mounted on the vehicle is determined by determining the approximate standard pattern by comparing with a standard pattern of brightness or chromaticity determined in advance, and determining The result is output to the CPU 54.

  In addition, the image processing unit 53 reads out image data for one frame stored in the image memory 52, and the image processing unit 53 reads the image data according to the imaging areas of the video cameras 1, 2, 3, and 4 based on the determination result. Image processing such as panoramic display and bird's-eye display is performed on the data, and the processed image is temporarily stored in the image memory 52 in units of frames.

The storage unit 56 includes a standard pattern unit 566, an image conversion table unit 560, and a video camera configuration table unit 567.
The standard pattern unit 566 stores a standard pattern of luminance or chromaticity corresponding to the imaging area of the video camera. The standard pattern is a pattern represented by binarized luminance.
The communication unit 57 will be described later.

  FIG. 4 is an explanatory diagram showing a standard pattern of luminance. In the figure, P101 is a standard pattern corresponding to an imaging area captured by the omnidirectional video camera 1. Since the omnidirectional monitoring video camera 1 images a 360-degree omnidirectional imaging region on a substantially circular screen, the image data captured by the omnidirectional monitoring video camera 1 has a high luminance in the region within the substantially circular shape. On the other hand, a region outside the substantially circular shape is a low luminance region where light is not taken in, and the standard pattern P101 has a low luminance (black) region outside the high circular (white) region.

  Therefore, if the image data approximates the standard pattern P101 by comparing the imaged image data with the luminance data in the low-luminance area outside the substantially circular shape of the standard pattern P101, the video camera that has captured the image data Can be determined to be a video camera 1 for omnidirectional monitoring.

  The standard pattern P102 corresponds to an imaging area captured by the rear monitoring video camera 2. Since the rear-view video camera 2 captures the rear bumper of the vehicle when imaging the rear of the vehicle, the rear-monitor video camera 2 detects the edge of the rear bumper and converts it to high brightness. The captured image data has an arc-shaped high luminance area on the lower side of the screen, and the standard pattern P102 extends from the left side of the screen to the right side on the lower side in the low luminance (black) area within the rectangular frame. And an arcuate high luminance (white) region which is an edge portion of the rear bumper.

  Therefore, if the image data approximates the standard pattern P102 by comparing the captured image data with the luminance data in the arcuate high luminance area below the standard pattern P102, the video imaged of the image data is captured. The camera can be determined to be the video camera 2 for rear monitoring.

  The standard pattern P103 corresponds to an imaging region captured by the video camera 3 for monitoring the left / right turn. The left and right turn monitoring video camera 3 also picks up the front bumper of the vehicle when picking up the front left and right of the vehicle, and therefore detects the edge of the front bumper and converts it to high brightness to The image data captured by the video camera 3 has high brightness areas at the lower left and right corners on the screen, and the standard pattern P103 is on the lower left and right sides in the low brightness (black) area within the rectangular frame. And a linear high brightness (white) region which is an edge portion of the front bumper.

  Therefore, if the image data approximates the standard pattern P103 by comparing the captured image data with the luminance data in the linear high luminance area below the standard pattern P103, the video imaged of the image data is captured. It can be determined that the camera is a video camera 3 for monitoring a left / right turn.

The image conversion table unit 560 stores an image conversion table used to perform image processing such as panorama display processing and bird's-eye display processing on image data captured by the video cameras 1, 2, 3, and 4.
When the video camera configuration table unit 567 outputs image data captured by the video cameras 1, 2, 3, and 4 mounted on the vehicle to the image processing device 5, an identification number unique to each video camera to be output is displayed. The imaging area of the video camera is associated and stored.

The input / output unit 55 reads out the image data processed by the image processing unit 53 from the image memory 52 and outputs it to the display device 6, whereby the image data subjected to the image processing is displayed on the display device 6.
Moreover, it is possible to switch which image is displayed among the images captured by the video cameras 1, 2, 3, and 4 by operating the operation unit provided in the display device 6 by the driver of the vehicle. is there.

  When the input / output unit 55 receives a vehicle state signal such as a vehicle speed and a traveling direction from a speed sensor, a clutch switch, a turn signal, etc., the input / output unit 55 outputs the vehicle state signal to the CPU 54, and the CPU 54 outputs the vehicle state signal. In response, the image data to be displayed on the display device 6 is switched from the image data captured by the video cameras 1, 2, 3, and 4. For example, when the vehicle is decelerated to a low speed and the blinker is activated, the video is switched to display an image from the video camera 3 for monitoring the left / right turn. It switches so that the image from the camera 2 may be displayed.

  The CPU 54 controls processing of the interface unit 51, the image memory 52, the image processing unit 53, the input / output unit 55, the storage unit 56, and a communication unit 57 described later. 58 is an internal bus.

FIG. 5 is a block diagram illustrating a configuration of the image processing unit 53. In the figure, reference numeral 531 denotes a luminance extraction unit which extracts luminance data of each pixel of image data for one frame read from the image memory 52. For example, when the resolution of an image of one frame is 320 × 240, there are 76800 pixels, but the luminance of the pixel of interest is extracted for each pixel block composed of a plurality of pixels with the pixel of interest at the center. By doing so, the processing can be simplified.
The luminance extraction unit 531 outputs the extracted luminance data to the binarization processing unit 532 and the edge detection unit 533.

  The binarization processing unit 532 calculates an average luminance value for each pixel from the input luminance data for one frame, and binarizes each pixel using the calculated average value as a threshold value. As a result, the image data for one frame can be separated into a low luminance region and a high luminance region. The binarization processing unit 532 outputs the binarized luminance data to the comparison unit 534.

  The edge detection unit 533 creates a differential filter by performing an operation using the luminance gradation value difference in the vicinity of the target pixel on the luminance data for one frame input from the luminance extraction unit 531, and performs imaging. Edge detection is performed by the differential filter that emphasizes lines or edges in the image. The edge detection unit 533 separates the luminance data for one frame from which edge detection has been performed into high luminance and low luminance according to the difference in gradation value, and outputs the separated luminance data to the comparison unit 534.

  The comparison unit 534 includes luminance data indicating a luminance pattern for one frame input from the binarization processing unit 532 and the edge detection unit 533, and standard luminance data indicating a standard pattern of luminance corresponding to the imaging area of the video camera. Compare That is, whether or not the luminance (low luminance or high luminance) matches for each pixel of the binarized and edge-detected luminance data and the standard luminance data, and the number of pixels that match for each of the plurality of standard luminance data And the counting result is output to the determination unit 535.

  The determination unit 535 compares the count result input from the comparison unit 534 with a predetermined threshold (T101, T102, T103), and the number of matching pixels is equal to or greater than a predetermined threshold (T101, T102, T103). If there is, it is determined that the imaging area of the video camera is the imaging area of the video camera corresponding to the matched threshold value, and the determination result is output to the CPU 54. If none of the standard patterns match, a result that no matching standard pattern exists is output to the CPU 54.

  Each of the threshold values (T101, T102, T103) corresponds to predetermined standard patterns P101, P102, P103.

  In the case of the standard pattern P101, in order to compare the image data captured by the video camera and the number of pixels in the low luminance area outside the substantially circular shape of the standard pattern P101, the threshold T101 is the number of pixels in the low luminance area outside the substantially circular shape. Is set to

  In the case of the standard pattern P102, the threshold value T102 is set to the number of pixels in the arc-shaped high-intensity area in order to compare the image data captured by the video camera with the number of pixels in the arc-shaped high-intensity area of the standard pattern P102. Has been.

In the case of the standard pattern P103, the threshold value T103 is set to the number of pixels in the linear high-brightness area in order to compare the image data captured by the video camera and the number of pixels in the linear high-brightness area of the standard pattern P103. Has been.
Therefore, the threshold values T101, T102, and T103 have a relationship of T101>T102> T103.
Note that the threshold value can be changed according to the type of the video camera.

  An FPGA (Field Programmable Gate Array) 536 performs image processing on the image data for one frame input from the image memory 52, and temporarily outputs the image data after the image processing to the image memory 52. Image processing performed by an FPGA (Field Programmable Gate Array) 536 is performed for each frame by reading the image conversion table unit 560 for image conversion stored in the storage unit 56 under the control of the CPU 54.

  FIG. 6 is a block diagram illustrating a configuration of the storage unit 56. In the figure, reference numeral 566 denotes a standard pattern portion, which stores image data constituting a standard pattern having a predetermined luminance or chromaticity corresponding to the imaging area of the video camera.

  The image conversion table unit 560 is read out under the control of the CPU 54 based on the determination result of the determination unit 535, is output to the FPGA 536, and is captured by the video cameras 1, 2, 3, 4 Image processing is performed according to the imaging area of each video camera 1, 2, 3, 4 by switching the conversion table.

  The conversion table of the image conversion table unit 560 represents from which pixel of the image data captured by the video cameras 1, 2, 3, and 4 each pixel of the image data displayed on the display device 6 is mapped. Each pixel has 32-bit data arranged in a one-dimensional order in the scanning order of the screen. When mapping each pixel of the image data captured by the video cameras 1, 2, 3, 4 is performed by setting the pixel number and the coordinate value of the pixel.

  The panorama display processing unit 561 has a panorama display conversion table. When the determination unit 535 determines that the video camera is the omnidirectional monitoring video camera 1, the panorama display processing unit 561 captures the panorama display processing unit 561 from the omnidirectional monitoring video camera 1. The image data is output to the FPGA 536 for panoramic display.

  The bird's-eye display processing unit 562 has a conversion table for bird's-eye display, and when the determination unit 535 determines that the video camera is the rear-view video camera 2, an image captured from the rear-view video camera 2 is captured. The data is output to the FPGA 536 for bird's-eye display.

  The graphic mask processing unit 563 has a conversion table for mask processing, and outputs to the FPGA 536 in order to mask unnecessary portions of the image displayed on the screen when performing bird's-eye view display processing. The conversion table for mask processing designates the YUV value of the portion to be masked, and performs mask processing by displaying image data displayed with the YUV value.

The composite display processing unit 564 outputs the image data captured from two or more video cameras to the FPGA 536 for simultaneously displaying on the screen of the display device 6.
The enlarged display processing unit 565 performs an enlarged display process on the image data captured by the video camera and outputs the image captured by the video camera to the FPGA 536 in order to perform an enlarged display.
Further, when a driver of the vehicle operates an operation unit provided in the display device 6, an operation signal is output to the CPU 54, and conversion of the composite display processing unit 564, the enlarged display processing unit 565, and the like is controlled by the CPU 54. It is also possible to read the table.

  Next, an image display method according to the present invention will be described. In advance, the video camera is mounted at a required position of the vehicle. When the engine key is inserted and the engine is started, power is supplied from the secondary battery mounted on the vehicle to the image processing device 5 and the display device 6, and the display device 6 outputs an operation signal to the image processing device 5. The processing device 5 sends an initial command to the video cameras 1, 2, 3, 4 via the interface unit 51. The video cameras 1, 2, 3, 4 that have received the initial command start imaging the outside of the vehicle from the image reading unit 11. The signal processing unit 12 converts the captured image data from an RGB signal to a YUV signal, and the image data in which the identification number for each of the video cameras 1, 2, 3, 4 is inserted is image data for one frame. Each time, the image data is sent to the image processing device 5 via the interface unit 13.

The image processing apparatus 5 temporarily stores image data for one frame in the image memory 52 for each video camera identification number. The image processing unit 53 reads the image data stored in the image memory 52 and outputs it to the luminance extraction unit 531.
The luminance extraction unit 531 extracts the luminance signal of each pixel, calculates a gradation value, and outputs the calculated gradation value of each pixel to the binarization processing unit 532 and the edge detection unit 533.

  The binarization processing unit 532 calculates the average value of the gradation values of each pixel from the image data for one frame, binarizes the pixels for one frame using the calculated average value as a threshold, and reduces the luminance. The luminance data is separated into high luminance, and the separated luminance data is output to the comparison unit 534.

FIG. 7 is an explanatory diagram illustrating an example of an image and a luminance pattern captured by the omnidirectional monitoring video camera 1. As shown in the figure, an image captured by the omnidirectional monitoring video camera 1 is displayed in a circular shape at the center, and the image outside the circular shape is not captured. The brightness outside the circular shape is low.
An image obtained by processing the image captured from the omnidirectional monitoring video camera 1 by the binarization processing unit 532 has a luminance pattern that forms a circular high luminance region in the low luminance region, and has a circular shape. A feature is that there is a low luminance region outside the high luminance region.

The comparison unit 534 reads the standard patterns P101, P102, and P103 from the standard pattern unit 566 and stores them.
The comparison unit 534 compares the luminance data input from the binarization processing unit 532 with the standard luminance data indicating the low luminance area of the standard pattern P101, and compares whether the luminance matches for each pixel. The number of pixels to be counted is counted, and the count value is output to the determination unit 535.

The determination unit 535 compares the count value input from the comparison unit 534 with the threshold value T101 for the standard pattern P101. If the count value is equal to or greater than the threshold value T101, the imaging area of the video camera is set to the standard pattern P101. It is determined that it is for corresponding omnidirectional monitoring, and the determination result is output to the CPU 54.
On the other hand, when the count value is less than the threshold value T101, it is determined that the imaging area of the video camera is not for omnidirectional monitoring, and the determination result is output to the CPU 54 in order to continue the determination in the edge detection unit 533.

  The edge detection unit 533 performs an operation using the luminance gradation value difference in the vicinity of the target pixel on the luminance data for one frame input from the luminance extraction unit 531. That is, the gradation value in which the edge portion is emphasized by adding the filter obtained from the difference between the eight pixels adjacent to the pixel of interest to the gradation value of the eight pixels adjacent to the pixel of interest at the center. Find new image data with This process is performed for all the pixels for one frame, and the luminance data for one frame whose edge is detected is separated into high luminance and low luminance according to the difference in gradation value, and the separated luminance data is obtained. Output to the comparison unit 534.

FIG. 8 is an explanatory diagram illustrating an example of an image and a luminance pattern captured by the video camera 2 for rear monitoring. As shown in the figure, the image captured by the video camera 2 for rear monitoring is characterized in that the luminance is discontinuous on both sides of the boundary portion of the rear bumper.
In the image after the image captured by the rear monitoring video camera 2 is detected by the edge detection unit 533 in the lower quarter of the entire image, the edge portion is detected at the boundary of the rear bumper. At the same time, an edge portion is detected at a boundary such as a white line or a car stop on the road surface, and the edge portion is displayed with high luminance, resulting in a luminance pattern as shown in the figure.

FIG. 9 is an explanatory diagram illustrating an example of an image and a luminance pattern captured by the video camera 3 for monitoring a left / right turn. As shown in the figure, the image captured by the video camera 3 for monitoring the left / right turn is characterized in that the luminance is discontinuous on both sides of the boundary of the front bumper.
In the image after the image detected by the video camera 3 for left / right turn monitoring is detected by the edge detection unit 533 in the lower quarter of the entire image, the edge portion is detected at the boundary of the front bumper. In addition, an edge part corresponding to the scenery in the left-right direction of the front part of the vehicle is detected, the edge part is displayed with high luminance, and a luminance pattern as shown in the figure is obtained.

  The comparison unit 534 compares the luminance data input from the edge detection unit 533 with the standard luminance data indicating the high luminance areas of the read standard patterns P102 and P103, and determines whether or not the luminance matches for each pixel. The number of matching pixels is counted, and the count value is output to the determination unit 535.

  The determination unit 535 compares the count value input from the comparison unit 534 with the threshold value T102 for the standard pattern P102. If the count value is equal to or greater than the threshold value T102, the imaging area of the video camera is set to the standard pattern P102. It is determined that it is for corresponding rear monitoring, and the determination result is output to the CPU 54.

On the other hand, when the count value is less than the threshold value T102, the count value is compared with the threshold value T103 for the standard pattern P103, and when the count value is equal to or greater than the threshold value T103, the imaging area of the video camera is It determines with it being for the left-right turn monitoring corresponding to the pattern P103, and outputs a determination result to CPU54.
When the count value is less than the threshold value T103, it is determined that the imaging area of the video camera is for forward monitoring that does not correspond to any standard pattern, and the determination result is output to the CPU.

  Based on the identification number sent from each video camera, the CPU 54 continues to repeat the above-described processing for all video cameras mounted on the vehicle.

  Based on the determination result input from the determination unit 535, the CPU 54 creates a video camera configuration table unit 567 having a unique identification number of the video camera mounted on the vehicle and an imaging region associated with the identification number. And stored in the storage unit 56. Thereby, the imaging area of each video camera is associated and stored for each video camera mounted on the vehicle.

  When the driver of the vehicle operates the operation unit to use any of the video cameras mounted on the vehicle, an operation signal corresponding to the operation is output to the image processing device 5, and an image from the required video camera is displayed. Can be displayed.

  For example, when displaying an image captured by the omnidirectional monitoring video camera 1, the image processing device 5 sends an image data transfer command to the omnidirectional monitoring video camera 1 via the interface unit 51. Is sent out. Upon receiving the transfer instruction, the control unit 14 of the omnidirectional video camera 1 reads out image data for each frame from the image memory 16, generates a packet from the image data, and transfers the packet to the image processing device 5. At this time, the frame rate can be converted in accordance with the resolution of the video camera 1 for omnidirectional monitoring.

  The CPU 54 reads the image conversion table corresponding to the omnidirectional video camera 1 that is the image data transfer source from the image conversion table unit 560 with reference to the video camera configuration table unit 567. In this case, the panorama display processing unit 561 is read, and the FPGA 536 performs image processing using the panorama display processing unit 561.

  In the panorama display, an image close to that of a normal video camera is displayed by cutting out and developing the image data captured by the omnidirectional monitoring video camera 1. The expansion method is performed by taking the correspondence between the coordinates in the image displayed on the display device 6 and the circular coordinate system of the image captured by the video camera 1 for omnidirectional monitoring.

  FIG. 10 is an explanatory diagram illustrating an example of a video image converted by the panorama display process. The image output from the omnidirectional video camera is difficult to see as it is due to the characteristics of the optical system, but an easy-to-view image can be obtained by performing a panoramic display.

  The image processing unit 53 temporarily stores image data subjected to panorama display processing for each frame in the image memory 52, and outputs the image data from the image memory 52 to the display device 6 under the control of the CPU 54. In the panorama display process, image data input from the omnidirectional video camera 1 is processed in real time, and for example, 30 frames of image processing is performed per second.

  Next, when displaying an image captured by the rear monitoring video camera 2, the image processing apparatus 5 issues an image data transfer command to the rear monitoring video camera 2 via the interface unit 51. Send it out. Upon receiving the transfer instruction, the control unit 14 of the rear-view video camera 2 reads image data for each frame from the image memory 16, generates a packet from the image data, and transfers the packet to the image processing device 5. At this time, the frame rate can be converted in accordance with the resolution of the video camera 2 for rear monitoring.

  The CPU 54 refers to the video camera configuration table unit 567 and reads an image conversion table corresponding to the rear monitoring video camera 2 that is the transfer source of the image data from the image conversion table unit 560. In this case, the bird's-eye display processing unit 562 and the graphic mask processing unit 563 are read, and the FPGA 536 performs image processing using the bird's-eye display processing unit 562 and the graphic mask processing unit 563.

  The bird's-eye view display is a viewpoint conversion display, and the height of the position of the video camera 2 for rear monitoring and the mounting angle of the video camera 2 for rear monitoring with respect to the two-dimensional image captured by the video camera 2 for rear monitoring. , And using a bird's eye view conversion table in which the focal length is set in advance, the image is converted into a three-dimensional coordinate system, and the converted three-dimensional coordinate system is used as the center of conversion for the position of the video camera 2 for rear monitoring. Furthermore, by mapping to a plane, an image as seen from above is displayed.

  The graphic mask process masks a required figure and image by setting an appropriate YUV value for each pixel of image data captured by the video camera 2 for rear monitoring using a conversion table for graphic mask process. .

  FIG. 11 is an explanatory diagram illustrating an example of an image converted by the bird's-eye view display process and the graphic mask process. An image that can be seen from above can be obtained, and for example, the positional relationship between the parking space and the vehicle can be easily understood.

In Embodiment 1, the luminance is extracted from the image data captured by the video cameras 1, 2, 3, and 4. However, the luminance is not limited, and chromaticity may be extracted.
In addition, although three examples of the standard pattern are shown, the present invention is not limited to this, and can be determined according to the characteristics of various video cameras such as the angle of view of the video camera and the combination of lenses. For example, depending on the type of video camera, such as a VGA wide-angle video camera with an angle of view of 100 degrees, an SXGA ultra-wide angle video camera combined with a cylindrical lens with an angle of view of 150 to 180 degrees, and a VGA wide-angle video camera with two angles of view of 100 degrees Standard patterns can be set.

Embodiment 2
FIG. 12 is a block diagram illustrating a configuration of the image processing apparatus 200. The image processing apparatus 200 includes a CPU 201, a RAM 202, an input unit 203 that interfaces with the video cameras 1, 2, 3, and 4, an output unit 204 that interfaces with the display device 6, an internal storage device 205 that stores programs, and a CD An external storage device 206, which is a CD-ROM drive device for reading a program recorded in the ROM, and a communication unit 207 that performs communication via a communication network are provided.

The program 210, the standard pattern unit 566, and the image conversion table unit 560 recorded on the CD-ROM are installed from the external storage device 206 of the image processing apparatus 200 to the internal storage device 205.
The CPU 201 reads the program 210, the standard pattern unit 566, and the image conversion table unit 560 from the internal storage device 205 into the RAM 202, whereby the program 210 is executed, and the video camera 1, 2, 3, 4 is input from the input unit 203. Image processing is performed on the image data read into the RAM 202, the processed image data is output to the display device 6, and an image is displayed.

  13 and 14 are flowcharts showing the procedure of a computer program executed by the image processing apparatus 200. Image data captured by the video cameras 1, 2, 3, and 4 is temporarily stored in the RAM 202 via the input unit 203. The CPU 201 reads image data for one frame imaged by the video camera 1 from the RAM 202 (step S1), and extracts the luminance of the read image data (step S2).

  The CPU 201 calculates an average value of luminance in each pixel from the extracted luminance value, and binarizes the pixel by separating the pixel into low luminance and high luminance using the calculated average value as a threshold (step S3). The CPU 201 reads out the standard luminance data of the standard pattern unit 566 from the RAM 202 (step S4), and binarized luminance data after binarization processing and standard luminance data constituting the standard pattern P101 of the omnidirectional monitoring video camera. From the above, the number of pixels with the same brightness is calculated (step S5), and the calculated number of pixels is compared with the threshold T101. If the calculated number of pixels is equal to or greater than the threshold T101 (YES in step S6), the video camera It is determined that 1 is an omnidirectional video camera (step S7).

  When the calculated number of pixels is less than the threshold value T101 (NO in step S6), the CPU 201 uses the difference in luminance gradation value in the vicinity of the target pixel for the image data from which luminance is extracted in step S2. Then, the edge of the captured image is detected (step S8). The number of pixels having the same luminance is calculated from the luminance data detected at the edge and the standard luminance data constituting the standard pattern P102 of the rear monitoring video camera (step S9), and the calculated number of pixels is compared with the threshold T102. If the calculated number of pixels is equal to or greater than the threshold value T102 (YES in step S10), it is determined that the video camera 1 is a video camera for rear monitoring (step S11).

  If the calculated number of pixels is less than the threshold value T102 (NO in step S10), the pixels whose luminance matches from the luminance data detected at the edge and the standard luminance data constituting the standard pattern P103 of the video camera for left / right turn monitoring Is calculated (step S12), and the calculated number of pixels is compared with the threshold T103. If the calculated number of pixels is equal to or greater than the threshold T103 (YES in step S13), the video camera 1 uses the video for monitoring the right or left turn. It determines with it being a camera (step S14).

  When the calculated number of pixels is less than the threshold value T103 (NO in step S13), it is determined that the video camera 1 is a video camera for front monitoring (step S15). It is determined whether or not the confirmation of the imaging area has been completed for all the video cameras mounted on the vehicle. If the confirmation has not been completed (NO in step S16), the process returns to step S1, and the processes after step S1 are performed. Continue.

  On the other hand, when the confirmation has been completed for all the video cameras (YES in step S16), image processing corresponding to the imaging region is performed (step S17). The CPU 201 reads out the image data captured by the video camera from the image memory 52 for each frame (step S18). When displaying an image of a video camera for omnidirectional monitoring (step S19), the panorama display processing unit 561 of the image conversion table unit 560 is read from the RAM 202 and panorama display processing is performed (step S20).

  When displaying the image of the video camera for rear monitoring (step S21), the bird's-eye display processing unit 562 and the graphics mask processing unit 563 of the image conversion table unit 560 are read from the RAM 202, and the bird's-eye display processing (step S22) and the graphics mask are performed. Processing is performed (step S23).

  When displaying an image of a video camera for left / right turn monitoring (step S24), the composite display processing unit 564 of the image conversion table unit 560 is read from the RAM 202, and composite display processing is performed on images from both the left and right directions (step S25). .

  When displaying an image of a video camera for forward monitoring (step S26), the enlarged display processing unit 565 of the image conversion table unit 560 is read from the RAM 202, and an enlarged display process is performed (step S27).

It is determined whether or not an operation for ending display on the display device 6 has been performed. If the operation has not ended (NO in step S28), the process returns to step 17 to continue the processing from step 17 onward. On the other hand, if the operation is finished (YES in step S28), the image processing is finished.
In addition, it cannot be overemphasized that each image processing mentioned above can control whether a process is performed by operation from an operation part.

  Note that when image processing is performed according to the imaging region, it is not always necessary to perform image conversion processing. For example, a captured image may be displayed as it is without performing image processing such as composite display processing and enlarged display processing.

Embodiment 3
In the first and second embodiments, the luminance is the same for each pixel of the luminance data extracted from the image data input from the video camera and the standard luminance data indicating the standard pattern of luminance corresponding to the imaging area of the video camera. The number of matching pixels is counted, and the imaging area of the video camera is determined based on the counting result.

  Next, calculate the distance between the luminance pattern extracted from the image data input from the video camera and the standard pattern of luminance according to the imaging area of the video camera, and determine the imaging area of the video camera based on the calculation result The case where it does is demonstrated.

  FIG. 15 is a block diagram showing a configuration of the image processing unit 59. In the figure, the luminance extraction unit 591, the edge detection unit 592, the determination unit 594, and the FPGA 595 are the same as the luminance extraction unit 531, the edge detection unit 533, the determination unit 535, and the FPGA 536 in FIG. To do.

  The luminance extraction unit 591 outputs luminance data extracted from the image data for one frame read from the image memory 52 to the edge detection unit 592, and the edge detection unit 592 corresponds to one frame input from the luminance extraction unit 591. Based on the luminance data, the luminance data for one frame whose edge is detected is separated into high luminance and low luminance according to the difference in gradation value (for example, the edge portion is set to high luminance), and the separated luminance The data is output to the calculation unit 593.

  Based on the standard patterns P101, P102, and P103 stored in the storage unit 56, the calculation unit 593 converts a circle, a curve, and a straight line that form an edge of each standard pattern in advance using any point on the standard pattern as a reference. It is calculated and stored as a dimensional expression (hereinafter referred to as an edge expression). For example, formulas of circles, curves, and straight lines are calculated using the center point of the standard pattern image as a reference.

  The calculation unit 593 calculates the square of the distance between each pixel forming the edge portion of the luminance pattern input from the edge detection unit 592 and a point on the edge expression corresponding to the pixel, and the edge portion of the luminance pattern The sum of the squares of the distances from the corresponding points on the edge formula is calculated for all the pixels.

  The point on the edge formula corresponding to the pixel forming the edge portion of the input luminance pattern is, for example, the point on the edge formula where the straight line connecting the point representing the pixel and the center point of the standard pattern intersects the edge formula. Can be a point.

  The calculation unit 593 calculates the sum of the squares of the distances between the pixels forming the edge portion of the input luminance pattern and the points on the edge formula corresponding to the pixels to all the standard patterns P101, P102, and P103. The calculation result is output to the determination unit 594.

  The determination unit 594 compares the calculation result for each standard pattern input from the calculation unit 593 with a predetermined threshold T201. If the calculation result is smaller than the threshold T201, the imaging region of the video camera It is determined that the imaging area is specified by the standard pattern corresponding to the result, and the determination result is output to the CPU 54. When all the calculation results for each standard pattern are equal to or greater than the threshold value T201, a result indicating that there is no approximate standard pattern is output to the CPU.

  Based on the above determination result, the image processing by the FPGA 595 is the same as the image processing by the FPGA 536 in the first embodiment, and thus the description thereof is omitted.

In the third embodiment, the sum of the squares of the distances between the pixels forming the edge portion of the input luminance pattern and the points on the edge formula corresponding to the pixels is calculated. However, the present invention is not limited to this. Instead, the square root of the sum of the squares of the distances and the sum of the absolute values of the distances may be calculated.
Further, the maximum absolute value of the distance may be calculated.

Embodiment 4
16 and 17 are flowcharts showing the procedure of a computer program executed by the image processing apparatus 200. The image processing apparatus that executes the program according to the present invention has the same configuration as the image processing apparatus 200 in FIG.

  Image data captured by the video cameras 1, 2, 3, and 4 is temporarily stored in the RAM 202 via the input unit 203. The CPU 201 reads image data for one frame captured by the video camera 1 from the RAM 202 (step S101), and extracts the luminance of the read image data (step S102).

  CPU201 detects the edge of the imaged image using the difference of the gradation value of the brightness | luminance in the vicinity of an attention pixel with respect to the image data from which the brightness | luminance was extracted (step S103). Based on the standard patterns P101, P102, and P103 stored in the storage unit 56, the CPU 201 uses circles, curves, and straight lines that form the edges of the standard patterns P101, P102, and P103 in advance as reference points for arbitrary points on the standard pattern. As a two-dimensional expression (hereinafter referred to as an edge expression) (step S104).

  The CPU 201 calculates the square of the distance between each pixel forming the edge portion of the captured image and a point on the edge formula corresponding to the pixel, and for all the pixels in the edge portion of the luminance pattern, the corresponding edge The sum of the square of the distance to the point on the equation is calculated (step S105), and the calculated value is compared with the threshold value T201. If the calculated value is equal to or greater than the threshold value T201 (NO in step S106), it corresponds to the standard pattern. For all the edge expressions, it is determined whether or not the sum of the squares of the distances has been calculated. If not calculated (NO in step S107), the processing from step S105 is continued.

  If all edge expressions have been calculated (YES in step S107), the video camera is determined to be for front monitoring, and the processing from step S109 described later is continued. If the calculated value is smaller than the threshold value T201 (YES in step S106), the video camera is determined to be a video camera in the imaging region represented by the standard pattern corresponding to the edge formula (step S108). That is, for all the pixels in the edge portion of the luminance pattern, the standard patterns P101, P102, P103 are compared by comparing the calculated value obtained by calculating the sum of the squares of the distances to the points on the corresponding edge formula with the threshold T201. Correspondingly, video cameras for omnidirectional monitoring, backward monitoring, right / left turn monitoring, and forward monitoring can be determined.

  It is determined whether or not the confirmation of the imaging region has been completed for all the video cameras mounted on the vehicle. If the confirmation has not been completed (NO in step S109), the process returns to step S101, and the processes after step S101 are performed. Continue.

  On the other hand, when the confirmation has been completed for all the video cameras (YES in step S109), image processing corresponding to the imaging region is performed (step S110). Since the process after step S110 is the same as the process after step S17 of FIG. 14 in Embodiment 2, description is abbreviate | omitted.

Embodiment 5
FIG. 18 is a schematic diagram showing an outline of an image display system according to the present invention. The image processing apparatus 5 mounted on the vehicle includes a communication unit 57 and is connected to a communication network 9 such as the Internet via wireless communication. A server 10 is connected to the communication network 9. In the server 10, reference numeral 101 denotes a storage unit that stores a standard pattern unit 566, an image conversion table unit 560, a video camera configuration table unit 567, and the like.

  The communication unit 57 of the image processing apparatus 5 transmits image data captured by the video cameras 1, 2, 3, and 4 to the server 10. Further, the standard pattern portion 566 stored in the server 10 is received from the server 10. Further, as described later, the determination result of the imaging configuration of the video camera performed by the server 10 is received.

The receiving unit 102 of the server 10 receives image data captured by the video cameras 1, 2, 3, and 4 via the image processing device 5.
The transmission unit 103 reads the standard pattern unit 566 from the storage unit 101 and transmits it to the image processing apparatus 5. The determination result of the imaging area of the video camera is transmitted to the image processing device 5.

The image processing unit 104 includes an image memory, performs processing such as luminance extraction, binarization processing, and edge detection from the received image data, compares the processed luminance data with the standard luminance data, and The imaging area is determined.
The control unit 105 controls processing of the storage unit 101, the reception unit 102, the transmission unit 103, and the image processing unit 104.

In the fifth embodiment, the case where the image processing apparatus 5 receives the standard pattern unit 566 from the storage unit 101 of the server 10 and performs processing will be described.
In this case, the standard pattern unit 566 is stored in the storage unit 101 of the server 10 instead of being stored in the storage unit 56 of the image processing apparatus 5. A standard pattern composed of standard luminance data to be compared with luminance data subjected to binarization processing or edge detection processing from image data captured by the video cameras 1, 2, 3, 4 is received from the server 10 and video The imaging area of the cameras 1, 2, 3, 4 is determined. Since other processes are the same as those in the first embodiment, description thereof is omitted.

  Accordingly, it is not necessary to store the standard pattern portion 566 for each vehicle, and the standard pattern portion 566 having the standard patterns corresponding to various video cameras is stored in the server 10 together, thereby creating and updating the standard pattern. The standard pattern for the video camera can be received from the server 10 even when a new video camera is commercialized, and the degree of freedom in selecting the video camera is improved. .

Next, in the fifth embodiment, the case where the server 10 determines the imaging area of the video cameras 1, 2, 3, 4 based on the image data captured by the video cameras 1, 2, 3, 4 will be described. .
In this case, the image processing unit 53 includes an FPGA 536, and the luminance extraction unit 531, binarization unit 532, edge detection unit 533, comparison unit 534, and determination unit 535 are replaced with the image processing unit 104 of the server 10.

Image data captured by the video cameras 1, 2, 3, and 4 is transmitted to the server 10 via the communication unit 57. The receiving unit 102 receives the image data and stores the image data in the image memory of the image processing unit 104.
The image processing unit 104 reads the image data and performs the same processing as in the first embodiment. Note that the description of the process is the same as that of the first embodiment, and thus the description thereof is omitted.

The determination results of the imaging areas of the video cameras 1, 2, 3, and 4 determined by the image processing unit 104 are transmitted to the image processing apparatus 5 via the transmission unit 103.
The image processing apparatus 5 that has received the determination result creates a video camera configuration table unit 567 based on the determination result and stores it in the storage unit 56.
When the driver operates the operation unit to operate a required video camera, an image from the video camera can be displayed. Since the image processing in the FPGA 536 is the same as that in Embodiment 1, the description thereof is omitted.

  Thereby, it is not necessary to determine the imaging area of the video camera by the image processing device 5 installed in each vehicle, and the determination result of the imaging area of the video camera is also stored in the server for the newly installed video camera. 10 can be obtained.

1 is a schematic diagram showing an outline of an image display system according to the present invention. It is a block diagram which shows the structure of a video camera. It is a block diagram which shows the structure of an image processing apparatus. It is explanatory drawing which shows the standard pattern of a brightness | luminance. It is a block diagram which shows the structure of an image process part. It is a block diagram which shows the structure of a memory | storage part. It is explanatory drawing which shows the example of the image and luminance pattern which the video camera for omnidirectional monitoring imaged. It is explanatory drawing which shows the example of the image and luminance pattern which the video camera for back monitoring imaged. It is explanatory drawing which shows the example of the image and luminance pattern which the video camera for the left-right turn monitoring imaged. It is explanatory drawing which shows the example of the image | video converted by the panorama display process. It is explanatory drawing which shows the example of the image | video converted by the bird's-eye view display process and the graphic mask process. It is a block diagram which shows the structure of an image processing apparatus. It is a flowchart which shows the procedure of the computer program performed by an image processing apparatus. It is a flowchart which shows the procedure of the computer program performed by an image processing apparatus. It is a block diagram which shows the structure of an image process part. It is a flowchart which shows the procedure of the computer program performed by an image processing apparatus. It is a flowchart which shows the procedure of the computer program performed by an image processing apparatus. 1 is a schematic diagram showing an outline of an image display system according to the present invention.

Explanation of symbols

1, 2, 3, 4 Video camera (imaging device)
DESCRIPTION OF SYMBOLS 5 Image processing apparatus 6 Display apparatus 7 Relay machine 9 Communication network 10 Server 11 Image reading part 12 Signal processing part 13 Interface part 14 Control part 15 Memory | storage part 16 Image memory 51 Interface part 52 Image memory 53 Image processing part 54 CPU
55 Input / output unit 56 Storage unit 57 Communication unit 59 Image processing unit 101 Storage unit 102 Reception unit 103 Transmission unit 104 Image processing unit 105 Control unit 200 Image processing device 201 CPU
202 RAM
203 Input Unit 204 Output Unit 205 Internal Storage Device 206 External Storage Device 531 Luminance Extraction Unit 532 Binarization Processing Unit 533 Edge Detection Unit 534 Comparison Unit 535 Determination Unit 536 FPGA
560 Image conversion table unit 561 Panorama display processing unit 562 Bird's eye display processing unit 563 Graphic mask processing unit 564 Composite display processing unit 565 Enlarged display processing unit 566 Standard pattern unit 567 Video camera configuration table unit 591 Luminance extraction unit 592 Edge detection unit 593 Calculation Part 594 judgment part 595 FPGA

Claims (13)

In an image display method for displaying image data processed by an image processing device that performs image processing according to an imaging area of the imaging device, image data captured by an imaging device that captures the outside of the vehicle,
The image processing apparatus includes:
A standard pattern that is predetermined according to the imaging area of the imaging device is stored,
An image display method, comprising: comparing image data captured by an imaging device and the standard pattern to determine an imaging region of the imaging device. In an image display method for displaying image data processed by an image processing device that performs image processing according to an imaging area of the imaging device, image data captured by an imaging device that captures the outside of the vehicle,
The image processing apparatus includes:
Receiving the standard pattern from a storage device that stores a predetermined standard pattern according to the imaging region of the imaging device;
Memorize the received standard pattern,
An image display method, comprising: comparing image data captured by an imaging device with a stored standard pattern to determine an imaging region of the imaging device.   The image processing device counts the number of pixels in which the luminance or chromaticity pattern extracted from the image data captured by the imaging device and the standard pattern have the same luminance or chromaticity, and the counted value is an imaging region of the imaging device The imaging region of the imaging device that captured the image data is determined to be an imaging region corresponding to the threshold value when a predetermined threshold value is exceeded according to the threshold value. The image display method described in 1.   The image processing device calculates the distance between the luminance or chromaticity pattern extracted from the image data captured by the imaging device and the standard pattern, and if the calculated value is smaller than a predetermined threshold, the image data 3. The image display method according to claim 1, wherein an image pickup area of the image pickup apparatus picked up is determined to be an image pickup area corresponding to the standard pattern. In an image display method for displaying image data processed by an image processing device that performs image processing according to an imaging area of the imaging device, image data captured by an imaging device that captures the outside of the vehicle,
The image processing apparatus includes:
The image data captured by the imaging device is transmitted to a processing device having storage means for storing a standard pattern determined in advance according to the imaging area of the imaging device,
The processor is
Receive the transmitted image data,
Compare the received image data with the standard pattern, determine the imaging area of the imaging device that captured the image data,
Transmitting the determination result to the image processing apparatus;
The image processing apparatus includes:
Furthermore, the said display result is received, The image display method characterized by the above-mentioned.   The processing device counts the number of pixels in which the luminance or chromaticity pattern extracted from the image data captured by the imaging device matches the luminance or chromaticity of the standard pattern, and the count value is stored in the imaging region of the imaging device. 6. The image according to claim 5, wherein when a predetermined threshold value is exceeded, the imaging region of the imaging device that captured the image data is determined to be an imaging region corresponding to the threshold value. Display method.   The processing device calculates a distance between a luminance or chromaticity pattern extracted from image data captured by the imaging device and the standard pattern, and captures the image data when the calculated value is smaller than a predetermined threshold value. The image display method according to claim 5, wherein the image pickup area of the image pickup apparatus is determined to be an image pickup area corresponding to the standard pattern. An imaging device that images the outside of the vehicle, an image processing device that performs image processing on the image data captured by the imaging device in accordance with the imaging area of the imaging device, and a display that displays the image data processed by the image processing device In an image display system comprising the apparatus,
The image processing apparatus includes:
Storage means for storing a standard pattern predetermined according to the imaging area of the imaging device;
An image display system comprising: a determination unit that compares image data captured by an imaging device and the standard pattern to determine an imaging region of the imaging device. An imaging device that images the outside of the vehicle, an image processing device that performs image processing on the image data captured by the imaging device in accordance with the imaging area of the imaging device, and a display that displays the image data processed by the image processing device In an image display system comprising the apparatus,
The image processing apparatus includes:
Receiving means for receiving the standard pattern from a storage device that stores a predetermined standard pattern according to the imaging area of the imaging device;
Storage means for storing the received standard pattern;
An image display system comprising: a determination unit that compares image data captured by an imaging device and the standard pattern to determine an imaging region of the imaging device. An imaging device that images the outside of the vehicle, an image processing device that performs image processing on the image data captured by the imaging device in accordance with the imaging area of the imaging device, and a display that displays the image data processed by the image processing device In an image display system comprising the apparatus,
The image processing apparatus includes:
A transmission unit that transmits image data captured by the imaging device to a processing device having a storage unit that stores a standard pattern that is predetermined according to the imaging region of the imaging device;
The processor is
Receiving means for receiving the transmitted image data;
A determination unit that compares the received image data with the standard pattern and determines an imaging region of the imaging device that has captured the image data;
Transmission means for transmitting the determination result determined by the determination means to the image processing apparatus,
The image processing apparatus includes:
The image display system further comprises receiving means for receiving the determination result.   The determination unit counts the number of pixels in which the luminance or chromaticity pattern extracted from the image data captured by the imaging device matches the luminance or chromaticity of the standard pattern, and the count value is stored in the imaging area of the imaging device. 11. The method according to claim 8, wherein when a predetermined threshold value is exceeded, the imaging region of the imaging device that images the image data is determined to be an imaging region corresponding to the threshold value. The image display system described in any one.   The determination unit calculates a distance between a luminance or chromaticity pattern extracted from image data captured by the imaging apparatus and the standard pattern, and captures the image data when the calculated value is smaller than a predetermined threshold value. The image display system according to any one of claims 8 to 10, wherein an imaging region of the imaging device is determined to be an imaging region corresponding to the standard pattern. In an image processing apparatus that performs image processing on captured image data,
Storage means for storing a predetermined standard pattern;
A comparison means for comparing the captured image data with the standard pattern;
An image processing apparatus that performs image processing on captured image data based on a result of comparison by the comparison unit.

Images