JP2004212658A - System and method for image display and image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system and a method for image display and an image processor that detect and report a moving body present in an imaging area of an imaging device. <P>SOLUTION: The image processor 1 is equipped with a microcomputer 10 which detects a change area in image frames acquired from a camera 20 successively in time series and an image processing part 12 having an image data generation part 15 which puts together frame image data arranged surrounding the change area detected by the microcomputer 10, an enlarged image acquired by extracting and enlarging the change area, and area image data indicating an imaging area of the camera 20 imaging the change area in the image frame acquired from the camera 20. The image processor 1 sends the generated composite data to a display device 3. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image display system, an image display method, and an image processing device that display image data captured by an imaging device mounted on a vehicle on a display device mounted on the vehicle.
[0002]
[Prior art]
As a device that assists driving of a vehicle such as an automobile, an in-vehicle camera that captures an image including a blind spot from a driver and an in-vehicle display device that displays an image captured by the in-vehicle camera are used. For example, an image of the rear of the vehicle is taken by a vehicle-mounted camera, and the garage is entered while checking the vehicle rear image displayed on the display of the vehicle-mounted display device. In addition, it is possible to image the front left and right or the rear left and right of the vehicle with an on-vehicle camera, and to confirm a child or the like in the vicinity of the vehicle such as front left or right or rear left or right with an onboard display device.
[0003]
Further, in such a driving assistance device, a plurality of cameras are mounted on a vehicle, and images from the respective cameras are switched and displayed, or images from the plurality of cameras are combined and displayed, so that the in-vehicle display device is provided. The area that can be displayed is enlarged by (for example, see Patent Document 1).
[0004]
[Patent Document 1]
JP 2001-219784 A
[0005]
[Problems to be solved by the invention]
However, the in-vehicle display device is limited in the size of the display screen depending on the mounting location in the vehicle and the like. Also, as described above, since the imaging area of a plurality of cameras is wide, the subject displayed on the in-vehicle display device is The size in the screen becomes smaller.
For this reason, there is a problem that it is difficult for the driver to find a moving object near the vehicle, particularly a short child or the like, although it is displayed on the in-vehicle display device.
[0006]
The present invention has been made in view of such circumstances, and by combining data in a change region in image data input in time series from an imaging device with the image data and transmitting the combined data to a display device. An image display that extracts a moving object present in the imaging region of the imaging device indicated by the change region from the image data and causes the display device to display the extracted moving object so that the driver can easily find the presence of the moving object. It is an object to provide a system, an image display method, and an image processing device.
[0007]
Another object of the present invention is to generate a table that associates data in the change area detected by the detection means with a combination position in the image data by the generation means, and based on the generated table, generates data in the change area. Another object of the present invention is to provide an image display system capable of reducing a calculation load in a combining process for generating combined data by combining the image data with image data from an imaging device.
[0008]
Still another object of the present invention is that a table used in a synthesizing process for generating synthesized data expands and associates data in a change area detected by the detection means, so that a simple table reference process can be performed. Another object of the present invention is to provide an image display system capable of combining data in the enlarged change area with image data from an imaging device.
[0009]
Still another object of the present invention is to generate frame image data surrounding a changing region in image data input in time series from an imaging device, and generate the frame image data by the detection unit in the image data. By combining the detected change area with a position surrounding the detected change area and sending it to the display device, the moving object present in the imaging region of the imaging device indicated by the change area is displayed by being surrounded by frame image data, and the presence of the moving object is displayed. It is an object of the present invention to provide an image display system and an image processing device that allow a driver to easily find a vehicle.
[0010]
Still another object of the present invention is to detect an imaging device that has captured image data including a changing region in image data input in time series from each of a plurality of imaging devices, and to indicate an imaging region of the detected imaging device. By combining the area data with the image data from the imaging device and sending it to the display device, the region data indicating the imaging region of the imaging device that captured the change region is displayed, and the position of the moving object indicated by the change region is displayed. An image display system and an image processing device capable of notifying a driver are provided.
[0011]
[Means for Solving the Problems]
An image display system according to a first aspect of the present invention is an image display system for displaying, on a display device mounted on a vehicle, image data captured by an imaging device mounted on a vehicle, the image input in time series from the imaging device. Detecting means for detecting a changing area in the data, extracting means for extracting data in the changing area detected by the detecting means, and combining data in the changing area extracted by the extracting means with the image data. It is characterized by comprising a synthesizing means for generating synthetic data, and a transmitting means for transmitting the synthetic data generated by the synthesizing means to the display device.
[0012]
An image display system according to a second aspect of the present invention includes a generation unit that generates a table that associates data in the change area detected by the detection unit with a synthesis position in the image data. It is characterized by comprising means for generating the combined data based on the generated table.
[0013]
An image display system according to a third invention is characterized in that the table enlarges and associates data in the change area detected by the detection means.
[0014]
An image display system according to a fourth aspect of the present invention is an image display system for displaying, on a display device mounted on a vehicle, image data captured by an imaging device mounted on a vehicle, the image input in time series from the imaging device. Detecting means for detecting a changing area in the data, generating means for generating frame image data surrounding the changing area detected by the detecting means, and detecting the frame image data generated by the generating means in the image data; The present invention is characterized in that it comprises a synthesizing means for synthesizing the data at a position surrounding the change area detected by the means to generate synthesized data, and a transmitting means for transmitting the synthesized data generated by the synthesizing means to the display device.
[0015]
An image display system according to a fifth aspect of the present invention is an image display system that displays image data captured by a plurality of imaging devices attached to a vehicle on a display device mounted on the vehicle. Detecting means for detecting each of the change areas in the input image data; detecting means for detecting an imaging device which has captured the image data including the change area detected by the detection means; and The image processing apparatus further includes a combining unit that combines region data indicating an imaging region with the image data to generate combined data, and a sending unit that sends the combined data generated by the combining unit to the display device. .
[0016]
An image display method according to a sixth aspect of the present invention is an image display method in which image data captured by an imaging device mounted on a vehicle is displayed on a display device mounted on the vehicle by a calculation device. Detecting a change area in the image data input in chronological order, extracting data in the detected change area, and combining the extracted data in the change area with the image data to generate synthesized data. And transmitting the generated combined data to the display device.
[0017]
An image processing apparatus according to a seventh aspect of the present invention is an image processing apparatus for a vehicle that transmits received image data to an external device. Detecting means for detecting a change area of the image, extracting means for extracting data in the change area detected by the detecting means, and combining data in the change area extracted by the extracting means with the image data. , And transmission means for transmitting the synthesized data generated by the synthesis means to the outside.
[0018]
An image processing apparatus according to an eighth aspect of the present invention is an image processing apparatus for a vehicle that transmits received image data to an external device. Detecting means for detecting the change area of the image, generating means for generating frame image data surrounding the change area detected by the detecting means, and the frame image data generated by the generating means, by the detecting means in the image data It is characterized by comprising a synthesizing unit that generates synthesized data by synthesizing the data at a position surrounding the detected change region, and a sending unit that sends out the synthesized data generated by the synthesizing unit to the outside.
[0019]
An image processing apparatus according to a ninth aspect is an image processing apparatus for a vehicle that transmits received image data to an external device. Detecting means for detecting a change area of the image, detecting means for detecting a position indicated by image data including the change area detected by the detecting means, and position data representing the position detected by the detecting means, and synthesizing the image data. And synthesizing means for generating the synthesized data by the synthesizing means, and sending means for sending out the synthesized data generated by the synthesizing means to the outside.
[0020]
In the case of the first, sixth and seventh inventions, a change area in image data input in time series from the imaging device is detected by the detection means, and data in the detected change area is extracted by the extraction means; The data in the extracted change area is synthesized with the image data by the synthesizing means, and is sent to the display device by the sending means, so that the change in the time-series continuous image data detected by the detecting means is displayed on the display device. A moving object present in the imaging region of the imaging device indicated by the region can be extracted from the image data and displayed, and a driver can be notified of the presence of the moving object.
[0021]
In the case according to the second invention, the data in the change area detected by the detection means is generated by the generation means to generate a table corresponding to the synthesis position in the image data, and the data in the change area is generated based on the generated table. By combining the image data with the image data from the imaging device, it is possible to reduce the calculation load in the combining process for generating the combined data.
[0022]
In the case of the third invention, since the table used in the combining process for generating the combined data is for expanding and associating the data in the change area detected by the detecting means, the reference process of the simple table is performed. The data in the enlarged change area can be combined with the image data from the imaging device.
Therefore, it is possible to cause the display device to enlarge and display the moving object present in the imaging region of the imaging device, and the driver or the like of the vehicle performs the enlarged display while viewing the image data captured by the imaging device on the display device. The moving body near the vehicle that has been set can be easily found.
[0023]
In the case of the fourth and eighth inventions, a change area in image data input in time series from the imaging device is detected by the detection means, and frame image data surrounding the detected change area is generated by the generation means. The synthesized frame image data is synthesized by the synthesizing unit at a position surrounding the change area detected by the detecting unit in the image data, and is sent to the display device by the sending unit. A moving object present in the imaging region of the imaging device, which is indicated by a change region in the image data that is continuous in time series, can be displayed by being surrounded by frame image data, and the presence of the moving object can be notified to a driver. it can.
[0024]
In the case of the fifth and ninth aspects, the imaging device detects the changing regions in the image data input in time series from each of the plurality of imaging devices, and captures the image data including the detected changing regions. Is detected by the detection means, and the area data indicating the detected imaging area of the imaging device is synthesized with the image data from the imaging apparatus by the synthesis means, and transmitted to the display device by the transmission means. It is possible to display the region data indicating the imaging region of the imaging device that has captured the change region in the image data that is continuous in time series, and to notify the driver of the position of the moving object indicated by the change region. it can.
[0025]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, the present invention will be specifically described with reference to the drawings showing the embodiments. In the present embodiment, an automobile (vehicle) equipped with a camera (imaging device) and a display device that displays an image captured by the camera will be described as an example. An image is composed of, for example, 30 image frames (image data) per second.
[0026]
FIG. 1 is a block diagram illustrating a configuration example of an image display system according to the present invention. An image display system 100 according to the present invention includes four cameras (imaging devices) 20, 21, 22, and 22 that capture images and capture image frames. 23, a display device 3 such as a liquid crystal display or a CRT (Cathode-Ray Tube) display, and image frames captured by the cameras 20, 21, 22, and 23. And an image processing apparatus 1 for sending the image data to the computer.
[0027]
The image processing device 1, the cameras 20, 21, 22, 23, and the display device 3 can be connected by dedicated wiring or can be connected via a vehicle-mounted LAN (Local Area Network). , The image processing apparatus 1 and the cameras 20, 21, 22, and 23 are connected to the image processing apparatus 1 and the display device 3 via an in-vehicle LAN compliant with the IEEE (Institute of Electrical and Electronic Engineers) 1394. Connected via wires.
[0028]
Each of the cameras 20, 21, 22, and 23 captures an image with a camera body (not shown) including a CCD (Charge Couple Device) or the like, and performs predetermined processing on the obtained image frame. , To the image processing apparatus 1 via an in-vehicle LAN conforming to IEEE 1394.
[0029]
FIG. 2 is a diagram showing the mounting positions of the above-described cameras 20, 21, 22, and 23 on a vehicle. In FIG. 2, a vehicle S equipped with the image display system 100 according to the present invention is viewed from above. ing.
As shown in FIG. 2, the camera 20 is attached to the left end of a front bumper (not shown) provided on the vehicle S so as to capture an image of the front left of the vehicle S. , Is attached to the right end of the front bumper so as to image the right front of the vehicle S.
[0030]
Further, the camera 22 is attached to the left end of a rear bumper (not shown) provided to the vehicle S so as to capture an image of the rear left of the vehicle S. The camera 23 is attached to the right end of the rear bumper. It is attached so as to image the rear right of the vehicle S.
The cameras 20, 21, 22, and 23 attached to the respective positions in this way sequentially transmit the image frames respectively captured to the image processing apparatus 1.
[0031]
In FIG. 1, a display device 3 receives, via a dedicated line, an image frame captured by a camera 20, 21, 22, or 23 and subjected to predetermined processing by an image processing device 1, and displays the received image frame. The information is sequentially displayed on a display (not shown).
[0032]
Further, the image processing apparatus 1 includes a microcomputer (hereinafter, referred to as a microcomputer) 10, a communication interface 11 for performing data transmission conforming to IEEE1394, and reception from the cameras 20, 21, 22, and 23 via the communication interface 11. An image processing unit 12 for performing predetermined processing on the processed image frame, a monitor interface 19 as transmission means for transmitting the image frame subjected to various processings by the image processing unit 12 to the display device 3, a frame memory 16, A ROM 17 and a table RAM 18 are provided.
[0033]
The microcomputer 10 includes a clock unit (not shown), controls the above-described hardware units while measuring timing, and sequentially executes a computer program stored in the ROM 17 as an operation of the image processing apparatus 1. Execute.
The ROM 17 stores various computer programs necessary for various operations of the image processing apparatus 1, information on mounting positions of the cameras 20, 21, 22, and 23 on the vehicle S, an image capturing direction, and the like. A predetermined threshold used when the unit 12 detects a moving object present in an image frame captured by the camera 20, 21, 22, 23, frame image data used when notifying the detected moving object, a conversion table And area image data (area data).
[0034]
The image processing unit 12 is connected to a frame memory 16 such as a RAM, a ROM 17, and a table RAM 18. The image processing unit 12 performs predetermined processing on each of the image frames and temporarily generated data. Are stored in the frame memory 16 or the table RAM 18.
When the image processing unit 12 uses the conversion table in the ROM 17, the conversion table is temporarily stored in the table RAM 18, and the conversion table is referenced at high speed by accessing the table RAM 18.
[0035]
In addition, the image processing unit 12 sends the image frames acquired from the cameras 20, 21, 22, and 23 to the display device 3, and captures moving objects in the image frames sequentially acquired from the cameras 20, 21, 22, and 23. An image data creation unit 15 for creating image data such as NTSC (National Television System Standard) data from the image frames respectively acquired from the cameras 20, 21, 22, and 23; A difference calculator 13 for calculating a difference value of each pixel value included in an image frame continuously acquired in time series from 20, 21, 22, 23, and each difference value calculated by the difference calculator 13 Are compared with threshold values stored in the ROM 17 in advance, so that And a binarization processing section 14 for conversion.
[0036]
Here, when a moving object is present in the imaging region captured by the cameras 20, 21, 22, and 23, the image processing apparatus 1 causes the image processing apparatus 1 to change a moving region in an image frame continuously acquired from the cameras 20, 21, 22, and 23. Occurs. That is, since a difference is generated between consecutive image frames before and after, the image processing apparatus 1 can detect a moving object in the imaging region of the camera 20, 21, 22, or 23 based on the difference between the image frames. .
[0037]
In the image processing apparatus 1 having the above-described configuration, the image processing unit 12 includes the image frames before and after sequentially input in time series from the cameras 20, 21, 22, and 23 via the communication interface 11. The difference value of each pixel value to be calculated is calculated by the difference calculation unit 13 to generate an image frame based on each difference value.
FIG. 3A is a diagram illustrating an example of an image frame based on the difference value calculated by the difference calculation unit 13 of the image processing unit 12. In the present embodiment, since the image frames obtained from the cameras 20, 21, 22, and 23 indicate each pixel with 8-bit data, the image frames shown in FIG. It is assumed that each difference value corresponding to each of the pixels has a value from 0 to 255.
[0038]
Further, the image processing unit 12 uses the binarization processing unit 14 to store each difference value included in the image frame based on the difference value generated by the difference calculation unit 13 in a threshold value (30 in FIG. ), And performs a binarization process in which a difference value smaller than the threshold value is set to 0 and a difference value larger than the threshold value is set to 1 to generate an image frame of binary data.
FIG. 3B is a diagram illustrating an example of an image frame of binary data obtained by performing the binarization processing by the binarization processing unit 14 on the image frame based on the difference value in FIG.
[0039]
In the present embodiment, by converting the difference value between successive image data calculated by the difference calculation unit 13 into binary data, a change region indicated by the difference value in the image frame can be remarkably expressed, This facilitates detection of a change area in an image frame by a microcomputer 10 described later.
[0040]
In the image processing apparatus 1, the microcomputer 10 operates as a detection unit that detects a change area in the image frame based on the image frame of the binary data generated by the binarization processing unit 14.
Specifically, the microcomputer 10 has one data in the image frame of the binary data in FIG. 3B and classifies the pixels that are close to each other into groups, and the change areas 40, 41, It is detected whether or not the moving object indicated by 42, 43, 44, and 45 is a moving object that requires the driver's attention when the vehicle S equipped with the image processing device 1 is started or the like.
[0041]
Here, the microcomputer 10 reads information such as the mounting positions and the imaging directions of the cameras 20, 21, 22, and 23 stored in the ROM 17 in advance, thereby changing each of the change areas 40, 41, It is possible to detect whether or not the moving object indicated by 42, 43, 44, 45 is a moving object that requires the driver's attention. For example, when each change region is above the image frame, the microcomputer 10 determines that the change region is a change region caused by a bird jumping up or a leaf of a tree swinging on a tree branch. It is determined that the moving object does not require the driver's attention.
[0042]
For the change areas 40, 41, 42, 43, 44, and 45 shown in FIG. 3B, the microcomputer 10 displays the change areas 41, 42, and 44 at the top of the screen. It is determined that the area is a change area caused by the leaves of the tree swaying in the branches, and the areas 43 and 45 are determined to be errors or the like included in the image frame because one piece of data is scattered.
In addition, the microcomputer 10 determines that the data 1 is hardened to some extent and exists near the ground for the change area 40 and indicates a moving object requiring attention, and determines that the moving object requires attention. The changed area 40 is notified to the driver.
[0043]
FIG. 4 is a diagram showing a configuration example of a notification screen. The display device 3 displays a notification screen as shown in FIG. 4 so that the display device 3 displays images near the vehicle captured by the cameras 20, 21, 22, and 23. An existing moving object can be notified to the driver.
Note that the notification screen in FIG. 4 illustrates an example of notifying a moving object in an image frame captured by one camera 20. For example, in the image frame captured by the four cameras 20, 21, 22, and 23, In the case where no moving object is detected, all the image frames from the cameras 20, 21, 22, and 23 are displayed by dividing the display of the display device 3 into four parts. When a moving object is detected, the display image may be switched to an image frame from the camera 20 and a notification screen as shown in FIG. 4 may be displayed.
[0044]
As shown in FIG. 4, the notification screen is divided into three areas 1, 2, 3, and the area 1 displays an image frame captured by the camera 20, and the area 2 displays the image frame. An enlarged image obtained by extracting the changed region 40 detected as described above in the image frame by a predetermined process and enlarging the image, and an area 3 is an image pickup area of the camera 20 that has picked up the image frame including the changed area 40. The displayed area image data is displayed.
Here, the image data creation unit 15 of the image processing unit 12 in FIG. 1 is a synthesizing unit that creates notification image data to be transmitted to the display device 3 so that the notification screen as shown in FIG. It is working.
[0045]
As described above, when the microcomputer 10 determines that the change area 40 to be notified to the driver exists, the microcomputer 10 determines the change area 40 to be extracted from the image frames from the camera 20 sequentially acquired through the communication interface 11. Then, it notifies the image data creation unit 15 of the image processing unit 12.
When notified of the change area 40 by the microcomputer 10, the image data creation unit 15 reads the frame image data from the ROM 17 and surrounds the read frame image data around the change area 40 of the image frame obtained from the camera 20. The combined data is generated at the position.
[0046]
Thereby, notification image data for displaying a notification screen in which a moving object is surrounded by a frame as shown in FIG. 4 can be generated in the area 1 of the display device 3.
It should be noted that the size of the frame indicated by the frame image data is set in advance, and if the change region surrounded by the frame is too small, it is determined that the moving body is far from the vehicle, and the driver is not notified. It can also be configured.
[0047]
Further, the microcomputer 10 enlarges the change area 40 displayed around the area 1 of the notification screen with the frame image data and displays the notification image data to be displayed in the area 2 of the notification screen by the image data creation unit 15. To generate the data, the conversion table stored in the ROM 17 is read out to the table RAM 18.
Here, the size of the enlarged image displayed in the area 2 of the notification screen is set in advance, and the conversion table stored in the ROM 17 is a basic table for specifying each pixel data in the enlarged image. The image data creating unit 15 converts the enlarged image from the camera 20 based on the conversion table generated by the microcomputer 10 updating the basic table along with the movement of the change area 40 in the image frame from the camera 20. The data is synthesized with the image frame to be obtained to generate synthesized data.
[0048]
Specifically, as described above, the microcomputer 10 operates as an extraction unit that extracts pixel data in a change area 40 in an image frame from the camera 20 displayed in the area 1 of the notification screen, and performs the change to be extracted. Two points (x ₁ , Y ₁ ) And (x ₂ , Y ₂ ) Is determined, and the two determined points (x ₁ , Y ₁ ) And (x ₂ , Y ₂ ) Are two points (X ₁ , Y ₁ ) And (X ₂ , Y ₂ ) Is calculated, and a conversion table for associating each pixel data in the change area 40 with each pixel in the area of the enlarged image based on the calculated enlargement rate is generated. .
[0049]
Thereby, as shown in FIG. 4, notification image data for enlarging and displaying the moving object captured by the camera 20 can be generated in the area 2 of the display device 3, and the image data creation unit 15 By simply reading out the pixel data in the conversion table generated as described above in a predetermined order, the enlarged image data of the change area 40 indicating the detected moving object can be sent to the display device 3.
[0050]
Further, as described above, when the microcomputer 10 determines that there is a change area 40 to be notified to the driver, the microcomputer 10 detects which camera has captured the image frame including the detected change area 40. It also operates as a detection unit, and detects the presence of the change area 40 and also detects the camera 20 that has captured an image frame including the change area 40.
Here, the area image data stored in the ROM 17 includes data indicating the vehicle S, data indicating the imaging area 20a of the camera 20, data indicating the imaging area 21a of the camera 21, as shown in area 3 of FIG. It includes data indicating the imaging area 22a of the camera 22 and data indicating the imaging area 23a of the camera 23.
[0051]
When notified by the microcomputer 10 of the change area 40 and the camera 20 that has captured the change area 40, the image data creation unit 15 reads the area image data from the ROM 17 and reads the image area of the camera 20 included in the read area image data. By applying a color or the like to the image 20a, the image data is synthesized with an image frame obtained from the camera 20 to generate synthesized data, distinguished from the other imaging regions 21a, 22a, and 23a.
[0052]
Thereby, as shown in FIG. 4, notification image data for displaying the imaging area of the camera 20 that has captured the moving object captured by the camera 20 can be generated in the area 3 of the display device 3 as shown in FIG. By displaying the generated notification image data on the display device 3, the driver can easily grasp the positional relationship between the captured moving object and the vehicle.
[0053]
Here, when a plurality of change regions indicating a moving object to be notified to the driver are detected, all the change regions are displayed with frame screen data for region 1, and each change region is displayed for region 2. Are displayed side by side, and for the area 3, by displaying all imaging areas corresponding to the detected change area by coloring or the like, the driver can be notified of all moving objects. The change areas 1, 2, and 3 need to be clearly associated with each other so as not to confuse the driver.
[0054]
Hereinafter, the moving object detection processing according to the image display method by the image processing device 1 in the image display system 100 having the above-described configuration will be described. When the driver of the vehicle S using the image display system 100 starts the stopped vehicle S, based on the notification screen displayed on the display device 3, the driver of each of the cameras 20, 21, 22, and 23 is activated. It can be confirmed that there is no moving object in the imaging area, especially around the vehicle S.
[0055]
FIG. 5 is a flowchart showing a moving object detection processing procedure in the image display system 100 according to the present invention.
In the image processing apparatus 1, the microcomputer 10 determines whether or not a predetermined time has elapsed by a clock means provided therein (S1), and when the predetermined time has not elapsed (S1: NO), the predetermined time has elapsed. If it is determined that the predetermined time has elapsed (S1: YES), the acquired image frame is temporarily stored in the frame memory 16 (S2).
[0056]
Next, the microcomputer 10 uses the difference calculation unit 13 of the image processing unit 12 to calculate the pixel values in each of the two image frames stored in the frame memory 16 and continuously acquired at predetermined time intervals. By subtraction, a difference value corresponding to each pixel is calculated, and an image frame based on these difference values is generated (S3).
The image frame based on the difference value generated by the difference calculation unit 13 is subjected to binarization processing based on a threshold value stored in the ROM 17 in advance by the binarization processing unit 14 (S4). Converted to image frames.
[0057]
The microcomputer 10 groups adjacent pixels having one data in the binary data image frame generated as described above, and detects a change area indicated by each group (S5).
Further, the microcomputer 10 determines whether or not each detected change area is a change area indicating a moving object to be notified to the driver (S6), and notifies the driver of each detected change area. If the change area is not the change area indicating the moving object (S6: NO), the change area is excluded from the processing target of the moving object detection processing (S7).
[0058]
On the other hand, when it is determined that the detected change area is a change area indicating a moving object to be notified to the driver (S6: YES), the microcomputer 10 indicates that the detected change area is to be notified to the driver. Notify the processing unit 12.
In the image processing unit 12 notified of the detection of the change area to be notified to the driver, the image data creating unit 15 reads the frame image data from the ROM 17 and sequentially acquires the read frame image data from the camera 20. The image frame is synthesized at a position surrounding the change area (S8).
[0059]
Next, the microcomputer 10 generates a conversion table for generating an enlarged image displayed in the area 2 of the notification screen from each pixel of the changed area based on the detected size of the changed area (S9).
Further, the microcomputer 10 detects the camera 20 that has captured the image frame including the detected change area (S10), and notifies the image processing unit 12 of the detected camera 20.
[0060]
In the image processing unit 12 that has detected the camera 20 that has captured the image frame including the change area, the image data creating unit 15 reads the area image data from the ROM 17 and captures the image area of the camera 20 included in the read area image data. The data indicating the image 20a is colored and combined with image frames sequentially acquired from the camera 20 (S11).
[0061]
As described above, the microcomputer 10 determines whether or not the determination process for determining whether or not each of the change regions detected in step S5 is a change region indicating a moving object to be notified to the driver is completed. If it has been determined (S12) and the processing has not been completed (S12: NO), the process returns to step S6, and it is determined whether or not each of the change areas is a change area indicating a moving object existing near the vehicle. to decide.
If it is determined that the determination process has been completed for all the change areas (S12: YES), the microcomputer 10 determines in step S8 the image frame obtained by combining the frame image data and the conversion table generated in step S9. Is transmitted to the display device 3 via the monitor interface 18 via the monitor interface 18 (step S13).
[0062]
The display device 3 displays a notification screen based on the notification image data received from the image processing device 1 so that the driver of the vehicle S, which is referencing the displayed notification screen, receives a moving object in the imaging region of the camera 20. The presence can be notified.
FIG. 6 is a diagram showing an example of the notification screen. The notification screen displays an image frame captured by the camera 20 and enclosing the moving object in a frame on the upper side, and a frame in an area 1 on the lower left side. An enlarged image obtained by enlarging the moving object surrounded by is displayed, and area image data indicating the imaging area 20a of the camera 20 that has imaged the detected moving object is displayed on the lower right side.
[0063]
With such a notification screen, the driver or the like of the vehicle can grasp the moving object existing in the imaging region of the camera 20, confirm the moving object displayed on the display device 3, and check the moving object on the vehicle. By determining whether or not the vehicle collides with the vehicle, a collision between the moving object and the vehicle can be avoided.
[0064]
As described above, for the sequentially acquired image frames, a moving object existing in the imaging region of the camera 20 can be detected based on a change between successive image frames at a predetermined time interval. By enclosing the frame in the image frame displayed on the display device 3 and displaying it as an enlarged image separately from the image frame acquired from the camera 20, and further displaying the positional relationship with the vehicle together with the vehicle information. In addition, it is possible to reliably notify the driver or the like of the vehicle that the moving object exists in the imaging region of the camera 20.
In the present embodiment, in each of the regions 1, 2, and 3 of the notification screen, the moving body is notified to the driver, but any one or two of the structures may be provided. .
[0065]
In the above-described embodiment, the image processing apparatus 1 calculates a difference value between image frames continuously acquired from the camera 2, and based on the difference value, a moving object exists in the imaging area of the camera 2. Although it is determined whether or not the moving object is detected, a moving object may be detected by performing an optical flow process on image frames that are continuously acquired.
In the optical flow process, a target block including a plurality of pixels is set in one image frame, the target block existing in a subsequent image frame is detected, and the target block in the two image frames is detected. By detecting the movement of each block of interest, it is possible to detect the presence of a moving object in the imaging area of the camera 2.
[0066]
Further, in the present embodiment, a configuration for detecting a moving object in an image frame imaged by each of four cameras 20, 21, 22, and 23 has been described. However, the configuration is limited to a configuration including four cameras. Not something.
Furthermore, the moving object detection processing in the image display system 100 of the present invention can also be applied to image data obtained by performing panorama expansion on an image frame captured by an omnidirectional camera.
[0067]
Further, for example, in the image frame obtained when an image of a child 1 m or 2 m away from the vehicle is taken, the number of pixels indicating the child is set in advance, and the driver detected by the microcomputer 10 should be notified. Based on the number of pixels in the change area, it is possible to determine whether the moving object near the vehicle is a child or not, and it is also possible to execute only the detection of the child near the vehicle.
[0068]
【The invention's effect】
According to the first, sixth, and seventh aspects of the invention, a change area in image data input in time series from an imaging device is detected, data in the detected change area is extracted, and the data is combined with the image data. Then, by sending the image data to the display device, the display device detects a moving object present in the imaging region of the imaging device, which is detected by the detection unit and indicates a change region in the image data that is continuous in time series, from the image data. It can be extracted and displayed, and the presence of the moving object can be notified to the driver.
[0069]
In the case according to the second aspect, a table is created that associates data in the change area in the image data that is continuous in time series with a synthesis position in the image data, and based on the generated table, data in the change area is generated. Is combined with the image data from the imaging device, thereby reducing the calculation load in the combining process for generating the combined data.
[0070]
In the case of the third aspect, since the table used in the combining process for generating the combined data is to enlarge and associate the data in the change area in the image data that is continuous in time series, the simple table is referred to. In the processing, the data in the enlarged change area can be combined with the image data from the imaging device.
Therefore, it is possible to cause the display device to enlarge and display the moving object present in the imaging region of the imaging device, and the driver or the like of the vehicle performs the enlarged display while viewing the image data captured by the imaging device on the display device. The moving body near the vehicle that has been set can be easily found.
[0071]
According to the fourth and eighth aspects, frame image data surrounding a change region in the image data input in time series from the imaging device is generated, and the generated frame image data is converted to the change region in the image data. By synthesizing it at the surrounding position and sending it to the display device, the display device detects the moving body present in the imaging region of the imaging device, which is detected by the detection means and indicates a changing region in the image data that is continuous in time series. It can be displayed by being surrounded by the frame image data, and the driver can be notified of the presence of the moving object.
[0072]
According to the fifth and ninth aspects, a change region in image data input in time series from each of the plurality of imaging devices is detected, and an imaging device that has captured image data including the detected change region is detected. By combining the detected area data indicating the imaging region of the imaging device with the image data from the imaging device and sending it to the display device, the display device detects the image data in the time-series continuous image data detected by the detection unit. Area data indicating the imaging area of the imaging device that has captured the change area of the moving object can be displayed, and the driver can be notified of the positional relationship between the moving object indicated by the change area and the vehicle.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of an image display system according to the present invention.
FIG. 2 is a diagram showing a mounting position of a camera on a vehicle.
3A is a diagram illustrating an image frame based on a difference value, and FIG. 3B is a diagram illustrating an image frame based on binary data.
FIG. 4 is a diagram illustrating a configuration example of a notification screen.
FIG. 5 is a flowchart showing a moving object detection processing procedure in the image display system according to the present invention.
FIG. 6 is a diagram illustrating an example of a notification screen.
[Explanation of symbols]
100 image display system
1 Image processing device
10. Microcomputer (detection means, generation means, extraction means, detection means)
12 Image processing unit
15 Image data creation unit (composition means)
17 ROM
19 monitor interface (transmission means)
20, 21, 22, 23 Camera (imaging device)
3 Display device

Claims (9)

In an image display system for displaying image data captured by an imaging device attached to a vehicle on a display device mounted on the vehicle,
Detecting means for detecting a change region in image data input in time series from the imaging device;
Extracting means for extracting data in the change area detected by the detecting means;
Combining means for combining data in the change area extracted by the extracting means with the image data to generate combined data;
An image display system comprising: a transmitting unit configured to transmit the combined data generated by the combining unit to the display device. Generating means for generating a table that associates data in the change area detected by the detecting means with a combination position in the image data;
The image display system according to claim 1, wherein the combining unit includes a unit configured to generate the combined data based on a table generated by the generating unit. 3. The image display system according to claim 2, wherein the table enlarges and associates data in the change area detected by the detection unit. In an image display system for displaying image data captured by an imaging device attached to a vehicle on a display device mounted on the vehicle,
Detecting means for detecting a change region in image data input in time series from the imaging device;
Generating means for generating frame image data surrounding the change area detected by the detecting means;
Synthesizing means for synthesizing the frame image data generated by the generating means at a position surrounding the change region detected by the detecting means in the image data to generate synthesized data;
An image display system comprising: a transmitting unit configured to transmit the combined data generated by the combining unit to the display device. In an image display system for displaying image data captured by a plurality of imaging devices attached to a vehicle on a display device mounted on the vehicle,
Detecting means for detecting each of the change areas in the image data input in time series from a plurality of imaging devices,
Detecting means for detecting an imaging device that has captured image data including the change area detected by the detecting means;
Synthesizing means for synthesizing area data indicating an imaging area of the imaging device detected by the detecting means with the image data to generate synthesized data;
An image display system comprising: a transmitting unit configured to transmit the combined data generated by the combining unit to the display device. In an image display method of displaying image data captured by an imaging device attached to a vehicle on a display device mounted on the vehicle by a computing device,
The computing device comprises:
Detecting a change area in image data input in time series from the imaging device,
Extract the data in the detected change area,
The data in the extracted change area is combined with the image data to generate combined data,
An image display method, wherein the generated combined data is transmitted to the display device. In an image processing device for a vehicle that transmits received image data to the outside,
Receiving means for receiving image data in chronological order from outside;
Detecting means for detecting a change area in the image data received by the receiving means;
Extracting means for extracting data in the change area detected by the detecting means;
Combining means for combining data in the change area extracted by the extracting means with the image data to generate combined data;
An image processing apparatus comprising: a sending unit that sends out the combined data generated by the combining unit to the outside. In an image processing device for a vehicle that transmits received image data to the outside,
Receiving means for receiving image data in chronological order from outside;
Detecting means for detecting a changing area in the image data received by the receiving means; generating means for generating frame image data surrounding the changing area detected by the detecting means;
Synthesizing means for synthesizing the frame image data generated by the generating means at a position surrounding the change region detected by the detecting means in the image data to generate synthesized data;
An image processing apparatus comprising: a sending unit that sends out the combined data generated by the combining unit to the outside. In an image processing device for a vehicle that transmits received image data to the outside,
Receiving means for receiving image data in chronological order from outside;
Detecting means for detecting a change area in the image data received by the receiving means; detecting means for detecting a position indicated by the image data including the change area detected by the detection means;
Combining means for combining position data representing the position detected by the detecting means with the image data to generate combined data;
An image processing apparatus comprising: a sending unit that sends out the combined data generated by the combining unit to the outside.

Images