JP2005303450A - Apparatus for monitoring vehicle periphery - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for monitoring a vehicle periphery which continuously displays moving images enabling recognition of a peripheral status of a vehicle without delay and facilitates an installation work. <P>SOLUTION: Radio communication is utilized between a transmitter side processing device 1 provided with a camera 6 and a receiver side processing device 11 provided with a display device 16, thereby facilitating the work for installation and wiring. Also, image processing units 2, 12 perform compression encoding of image data based on wavelet transform to separate the image data into a low-frequency component and a high-frequency component, and the data of low frequency component are preferentially transmitted/received, thereby realizing the apparatus for monitoring vehicle periphery requiring to display moving images continuously without delay in preference to image quality. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a vehicle periphery monitoring device that displays an image of the periphery of a vehicle on a display device installed in the vehicle without delay.

  In recent years, mounting of vehicle periphery monitoring devices that use moving images for the purpose of acquiring peripheral information on vehicles has increased. According to the vehicle periphery monitoring device, for example, as shown in FIG. 7, it is installed at the front of the vehicle when the left and right situations of the intersection to be entered cannot be visually recognized due to the obstacle 41 on the left and right of the driver's seat. The moving image data picked up by the camera 40 can be displayed on the display device to check the situation of the intersection.

  MPEG (Motion Picture Experts Group) format is often used for compression coding of moving images. An MPEG format compression encoding method is often used in the field related to computers, and importance is placed on reducing the data capacity while maintaining the quality of moving images. And in order to implement | achieve this, the prediction encoding technique which encodes the frame data obtained by performing a discrete cosine transform (DCT: Discret Cosine Transform) using the frame data before and behind that is used.

  However, for moving image data for recognizing the surrounding information of the vehicle, information on other vehicles, passers-by, etc. that changes every moment is continuously transmitted to the driver without delay and without interruption. It is required to do.

  Therefore, a prediction encoding technique that has a possibility that subsequent frame data encoded using this frame data cannot be decoded because some frame data is lost during transmission is used for a vehicle periphery monitoring device. There was a problem that it was not suitable. In addition, as the vehicle surrounding information, it is more important to obtain an image that allows the driver to recognize the surrounding situation of the vehicle than to depict details in detail, so image data of a low frequency component is important, In the encoding technique using DCT transform, there is a problem that when the compression rate is increased, data loss of low frequency components increases, and block noise is generated and image quality is significantly deteriorated.

  Furthermore, it relates to the installation work and the connection between the camera and the display device to display the moving images captured by the plurality of cameras installed in the front and rear of the vehicle on the display device installed near the driver's seat. There was also a problem that the work was complicated.

  Accordingly, an object of the present invention is to provide a vehicle periphery monitoring device with easy installation work that can continuously display a moving image capable of recognizing the surrounding situation of the vehicle without delay.

  In order to solve the above-mentioned problem, the invention of claim 1 is a vehicle periphery monitoring device that displays a video image of the periphery of the vehicle on a display device installed in the vehicle, and a camera that images the periphery of the vehicle; The image processing apparatus includes: the display device installed in the vehicle; and image processing means for encoding or decoding image data captured by the camera using wavelet transform.

  Further, the invention of claim 2 is the vehicle periphery monitoring device according to the invention of claim 1, further comprising a communication means for transmitting and receiving the image data processed by the image processing means by wireless communication. Features.

  The invention of claim 3 is the vehicle periphery monitoring device according to claim 1 or claim 2, wherein the communication means is a predetermined one of the data encoded by the image processing means by the wavelet transform. The low-frequency component data below the frequency is preferentially transmitted / received.

  According to the first aspect of the present invention, it is possible to realize a vehicle periphery monitoring device that continuously reproduces and displays a moving image capable of grasping the situation around the vehicle without delay.

  According to invention of Claim 2, a vehicle periphery monitoring apparatus can be installed in a vehicle, without performing complicated wiring operation.

  According to the third aspect of the present invention, important data is transmitted and received preferentially at high speed, and a moving image capable of recognizing the situation around the vehicle is displayed without delay even if there is some degradation in image quality. A monitoring device can be realized.

  FIG. 1 is a diagram showing a vehicle periphery monitoring device according to an embodiment of the present invention. As shown in FIG. 1, the vehicle periphery monitoring apparatus includes a transmission side processing unit 1 and a reception side processing unit 11, and transmits a moving image around the vehicle captured by the camera 6 from the transmission side processing unit 1 by wireless communication. The receiving side processing unit 11 has a function of outputting and displaying the decoded moving image on the display device 16. For example, the camera 6 is installed in the front part of the vehicle, the rear part of the vehicle, the side mirror, or the like in order to image the periphery of the vehicle, and the transmission side processing unit 1 is disposed in the vicinity thereof. The display device 16 is installed, for example, on an instrument panel around the driver's seat, and the reception side processing unit 11 is installed in the vicinity thereof.

  The transmission side processing unit 1 includes an image processing unit 2 that processes moving image data input from the camera 6 via the connection terminal 5, a wireless communication unit 3 that wirelessly transmits the processed data, and an image processing unit 2. And a memory 4 used for processing moving image data and the like. The reception-side processing unit 11 reproduces a moving image output from the image processing unit 12 via the wireless communication unit 13 that receives data by wireless communication, an image processing unit 12 that processes the received data, and the connection terminal 15. A display device 16 for displaying and a memory 14 used by the image processing unit 12 for processing moving image data and the like are included.

  The camera 6 has a function of converting an object imaged using a sensor such as a CCD or CMOS into an NTSC (National Television System Committee) or digital video signal and outputting it. Although FIG. 1 shows a mode in which one camera is used, a mode in which two or more cameras are used may be used.

  The image processing unit 2 has a function of compressing and encoding moving image data input from the camera 6 using wavelet transform. Similarly, the image processing unit 12 has a function of using the wavelet transform to decompress and decode the compressed and decoded data into the original moving image data. The image processing units 2 and 12 (image processing means) are realized by dedicated hardware and software using, for example, an ASIC or FPGA.

  The wireless communication unit 3 has a function of wirelessly transmitting data processed by the image processing unit 2. Similarly, the wireless communication unit 13 has a function of receiving data through wireless communication. The wireless communication units 3 and 13 (communication means) are realized by dedicated hardware and software, and realize wireless communication between the transmission side processing unit 1 and the reception side processing unit 1. The wireless communication method may be based on, for example, the IEEE 802.11 specification recommended as a standard specification of a wireless LAN, or may be based on Bluetooth or the like used as a short-range wireless technology. Alternatively, a narrow-area communication system (DSRC) used in an automatic toll collection system (ETC) may be applied.

  The memories 4 and 14 are working memories for temporary storage when the image processing apparatuses 2 and 12 perform processing such as encoding and decoding of moving image data. For example, SDRAM is used.

  The display device 16 is a device having a function of reproducing and displaying the moving image decompressed and decoded by the receiving side processing unit 11, and uses, for example, a liquid crystal display device.

  The connection terminals 5 and 15 have a function of electrically connecting the transmission side processing unit 1 and the camera 6, and the reception side processing unit 11 and the display device 16. Since it is a detachable aspect realized by a connector or the like, the camera 6 and the display device 16 used for the vehicle periphery monitoring device can be easily changed.

  The vehicle periphery monitoring apparatus having the above configuration transmits and receives moving image data compressed and encoded using wavelet transform by wireless communication. Specifically, the image processing units 2 and 12 process moving image data based on a JPEG2000 (Joint Photographic Experts Group 2000) method using wavelet transform.

  The MPEG method uses DCT conversion, whereas the JPEG2000 method uses DWT (Discrete Wavelet Transform). For entropy coding, EBCOT (Embedded Block Coding with Optimized Truncation) that performs bit-plane coding is employed. Hereinafter, details of the compression encoding process of moving image data performed by the image processing unit 2 will be described.

  FIG. 2 is a functional block diagram illustrating a schematic configuration of the image processing unit 2. The image processing unit 2 realizes functional blocks as shown in FIG. 2 by dedicated hardware or software, and performs compression coding. Hereinafter, the procedure will be described with reference to FIG.

  The moving image data input from the camera 6 to the image processing unit 2 via the connection terminal 5 is subjected to DC level conversion as necessary by the DC level shift unit 21 and then output to the color space conversion unit 22. . Next, the color space conversion unit 22 converts the color space of the data input from the DC level shift unit 21. Specifically, for example, the RGB signal is converted into a YCbCr signal (a signal composed of a luminance signal Y and color difference signals Cb and Cr).

  Next, the tiling unit 23 divides the image data input from the color space conversion unit 22 into a plurality of rectangular area components called “tiles”, and outputs them to the DWT unit 24. The DWT unit 24 performs integer type or real number type DWT on the image data input from the tiling unit 23 in units of tiles, and outputs the conversion coefficient obtained as a result.

  In DWT, a one-dimensional filter that divides a high-frequency component (high-frequency component) and a low-frequency component (low-frequency component) is applied to a two-dimensional image signal in the order of vertical and horizontal directions. Then, only the band component divided to the low frequency side in both the vertical direction and the horizontal direction is further recursively divided into bands (octave division). The number of times the band is recursively divided at this time is called a decomposition level.

  FIG. 3 is a schematic diagram showing a two-dimensional image 31 subjected to decomposition level 3 DWT according to the octave division method. At decomposition level 1, the two-dimensional image 31 is divided into four band components HH1, HL1, LH1, and LL1 (not shown) by sequentially applying the above-described one-dimensional filter in the vertical direction and the horizontal direction. The Here, “H” indicates a high frequency component, and “L” indicates a low frequency component. For example, HL1 is a band component composed of a high frequency component H in the horizontal direction and a low frequency component L in the vertical direction at the decomposition level 1. That is, “XYn” (X and Y are either H or L; n is an integer equal to or greater than 1) indicates a band component composed of a horizontal band component X and a vertical band component Y at the decomposition level n. .

  At decomposition level 2, the low frequency component LL1 is band-divided into HH2, HL2, LH2, and LL2 (not shown). Further, at the decomposition level 3, the low frequency component LL2 is band-divided into HH3, HL3, LH3 and LL3. FIG. 3 shows the arrangement of the band components HH1 to LL3 generated in this way. In FIG. 3, for the sake of simplicity, an example of a tertiary decomposition level is shown, but the decomposition level of the present embodiment may be higher than this.

  Next, the quantization unit 25 quantizes the transform coefficient output from the DWT unit 24. The quantization unit 25 also has a function of performing a bit shift process that prioritizes the image quality of a designated region (ROI; Region Of Interest) by the ROI unit 26.

  Next, the transform coefficient output from the quantization unit 25 is sequentially subjected to block-based entropy coding by the coefficient bit modeling unit 27 and the arithmetic coding unit 28 in accordance with the above-described EBCOT, and the code amount control unit At 29, the rate is controlled. Specifically, the coefficient bit modeling unit 27 divides the band component of the input transform coefficient into regions called “code blocks” of about 16 × 16, 32 × 32, and 64 × 64, and further each code block. , It is decomposed into a plurality of bit planes composed of a two-dimensional array of bits.

FIG. 4 is a schematic diagram showing a two-dimensional image 31 that is divided into a plurality of code blocks 32. FIG. 5 is a schematic diagram showing _n bit planes 33 _{0 to} 33 _n−1 (n: natural number) constituting the code block 32. When the binary value 34 of one conversion coefficient in the code block 32 is “011... 0” as shown in FIG. 5, the bits constituting the binary value 34 are respectively bit planes 33 _{n−. 1, 33 n-2, 33} n-3, ..., is decomposed to belong to 33 _0. Bit plane 33 _n-1 in the figure, represents the most significant bit plane consisting of only the most significant bits of the transform coefficients (MSB), bit plane 33 ₀ is the least significant bit plane consisting of only the least significant bit (LSB) Represents.

Further, the coefficient bit modeling unit 27 performs context determination of each bit in each bit plane 33 _k (k = 0 to n−1), and as shown in FIG. According to the result, the bit plane 33 _k is decomposed into three types of coding passes, that is, a CL pass (CLeanup pass), an MR pass (Magnitude Refinement pass), and a SIG pass (SIGnificance propagation pass). The context determination algorithm for each coding pass is defined by EBCOT. According to it, “significant” means a state in which the coefficient of interest is known to be non-zero in the encoding process so far, and “not significant” means that the coefficient value is zero. , Or a state that may be zero.

  The coefficient bit modeling unit 27 performs the SIG pass (significant coefficient coding pass with a significant coefficient around it), the MR path (significant coefficient coding pass), and the CL path (the SIG path, the MR path not corresponding). Bit plane encoding is performed in three types of encoding passes). Bit plane encoding is performed by scanning the bits of each bit plane in units of 4 bits from the most significant bit plane to the least significant bit plane and determining whether or not a significant coefficient exists. The number of bit planes composed only of insignificant coefficients (0 bits) is recorded in the packet header, and actual encoding is started from the bit plane in which significant coefficients first appear. The encoding start bit plane is encoded only by the CL pass, and the bit planes lower than the bit plane are sequentially encoded by the above three types of encoding passes.

  Next, the arithmetic encoding unit 28 uses the MQ coder to perform arithmetic encoding on the coefficient sequence from the coefficient bit modeling unit 27 in units of encoding passes based on the context determination result. There is also a mode in which the arithmetic encoding unit 28 performs a bypass process in which a part of the coefficient sequence input from the coefficient bit modeling unit 27 is not arithmetically encoded.

  Next, the code amount control unit 29 controls the final code amount by performing post-quantization that truncates the lower bit planes of the code string output by the arithmetic encoding unit 28. Then, the bit stream generation unit 30 generates a bit stream in which the code string output from the code amount control unit 29 and additional information (header information, layer configuration, scalability information, quantization table, etc.) are multiplexed, and a compressed image Output as data.

  The image processing unit 12 decodes the original image data by following the above-described compression encoding procedure in reverse.

  In the present embodiment, the ROI unit 26 and the processing units 27 to 29 for realizing entropy coding by EBCOT are shown. However, the present embodiment is not limited to this, and the ROI unit 26 is not used and the ROI is used. It may be an aspect that does not, or an aspect that uses entropy coding other than EBCOT.

  Since the conventional MPEG system uses predictive coding, each frame data depends on the frame data before and after in the time axis direction. For this reason, when one frame data is lost due to, for example, noise or the like at the time of data transmission / reception, there is a possibility that other frame data is affected and a plurality of frame data cannot be decoded. However, in the vehicle periphery monitoring apparatus described above, each frame data constituting the moving image data is compression-coded as one independent still image data. Therefore, even if a problem occurs in one frame data, the other frame data is not affected, and the moving image data captured by the camera 6 can be continuously displayed on the display device 16 without interruption. In addition, the fact that processing such as predictive encoding and motion compensation is not required unlike the MPEG system also leads to reduction of the load on encoding and decoding processing.

  As the vehicle surrounding information, it is important that the driver can continuously recognize the surrounding situation of the vehicle without delay, even if the image quality is somewhat poor, rather than depicting details in detail with high-quality images. The image data can reproduce the approximate contents by the low frequency component, and the details are reproduced by the high frequency component. As described above, when encoding is performed by DWT using wavelet transform, image data is separated into a low frequency component and a high frequency component in the processing process. Thus, it is possible to set processing priority by frequency components, such as processing with priority on low frequency component data.

  Specifically, for example, when the data output from the image processing unit 2 is transmitted and received as packet data by the wireless communication units 3 and 13, the frequency range of the image data included in the packet data can be recognized. Thus, packet data including a predetermined frequency component can be preferentially transmitted / received so as to be reliably transmitted / received. Alternatively, if the data size of the packet data including the data of the low frequency component is made smaller than the data size of the packet data including the data of the high frequency component, it is possible to improve the establishment of successful transmission / reception for the data of the low frequency component. The effect when packet data is lost can be suppressed. At this time, it is also possible to perform processing such as ignoring even if packet data is lost for high frequency components of a predetermined frequency or higher. Alternatively, it is assumed that a high frequency component of a predetermined frequency or higher is not necessary for the vehicle periphery monitoring device, and the data of the corresponding high frequency component is cut off at the data generation stage or is cut off at the data transmission stage according to the communication situation. Processing is also possible.

  In this way, if the low-frequency component of image data is preferentially handled, even if there is a communication failure such as a loss of part of the data to be transmitted / received, the overall situation around the vehicle can be achieved even if there is a slight deterioration in image quality. Can be obtained. In addition, it is possible to suppress the entire capacity of data to be transmitted and received to realize high-speed communication, and it is possible to continuously display moving images captured by the camera 6 on the display device 16 without delay.

  Further, if the function of the ROI unit 26 is used, it is possible to compress and encode only an important region without degrading the image quality. For example, when imaging the situation around a vehicle, if processing is performed so as to suppress degradation of image quality by giving priority to other vehicles, passers-by, and obstacles that the driver is paying attention to, the data It is possible to more accurately capture only important information around the vehicle while reducing the capacity.

  In addition, according to the above-described compression encoding using the wavelet transform, block noise does not occur in the decoded image as in the case of using the DCT transform, and even when compared with the conventional JPEG or the like with the same data capacity. There are various advantages that a high-quality decoded image can be obtained and editing and processing are easy.

  Thus, if wavelet transform is used for processing of moving images to be captured, various requirements required for the vehicle periphery monitoring device can be satisfied. That is, it is possible to realize a vehicle periphery monitoring device having a specific effect as compared with a conventional device that processes a moving image by a method such as MPEG.

  In addition, since the communication between the transmission side processing unit 1 and the reception side processing unit 11 is made wireless, it is difficult to handle cables for connection and to draw cables from outside the vehicle compartment to the vehicle interior as in the past. Thus, the vehicle periphery monitoring device can be easily installed without performing any troublesome work.

It is a figure which shows the vehicle periphery monitoring apparatus which concerns on one embodiment of this invention. It is a functional block diagram which shows the image processing part which concerns on one embodiment of this invention. It is a schematic diagram which shows the two-dimensional image which gave DWT which concerns on one embodiment of this invention. It is a schematic diagram which shows the two-dimensional image decomposed | disassembled into the some code block which concerns on one embodiment of this invention. It is a schematic diagram which shows the several bit plane which comprises the code block which concerns on one embodiment of this invention. It is a schematic diagram which shows three types of encoding passes based on one embodiment of this invention. It is a top view which shows the vehicle in which the vehicle periphery monitoring apparatus was installed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Transmission side processing part 2, 12 Image processing part 3, 13 Wireless communication part 4, 14 Memory 5, 15 Connection terminal 11 Reception side processing part

Claims (3)

A vehicle periphery monitoring device that displays an image of a vehicle periphery on a display device installed in the vehicle,
A camera for imaging the periphery of the vehicle;
The display device installed in the vehicle;
Image processing means for encoding or decoding image data captured by the camera using wavelet transform;
A vehicle periphery monitoring device comprising: The vehicle periphery monitoring device according to claim 1, further comprising:
Communication means for transmitting and receiving image data processed by the image processing means by wireless communication;
A vehicle periphery monitoring device comprising: The vehicle periphery monitoring device according to claim 1 or 2,
The vehicle periphery monitoring device, wherein the communication unit preferentially transmits / receives data of a low frequency component equal to or lower than a predetermined frequency among the data encoded by the image processing unit by the wavelet transform.

Images