JP2007186916A - Window image area detection device and power window device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an window image area detection device capable of automatically extracting an window image area portion using photographed images provided from a photographing device such as a camera and a power window device. <P>SOLUTION: A vehicle 1 comprises an image processing device 15 having pinch detection function. An infrared camera 14 capable of photographing the areas around the window frame is installed on the door 2 of the vehicle 1. The image processing device 15 extracts the moving areas in a window 3 from image data Dpic acquired from the infrared camera 14 and counts them in order to extract the window image area. The image processing device 15 which extracts the window image area extracts the detection area in the photographed image Pic from that area. The image processing device 15 recognizes that pinch occurs when it determines that the foreign matter in the photographed image invades into the detection area, and notifies it to the power window ECU6 to perform safe operations by the power window ECU6. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention uses a photographed image acquired by a photographing device such as a camera, uses a window image region detection device that extracts a window part region in the photographed image, and a window image region obtained by the window image region detection device, The present invention relates to a power window device for detecting whether a foreign object such as a human body is caught by a window.

  In recent years, in many vehicles, a power window device that automatically raises and lowers a window using a DC motor or the like as a drive source is mounted for the purpose of improving the convenience of opening and closing the window. This type of power window device includes a model having a pinching prevention function for preventing pinching by the window during the opening / closing operation of the window, and this technique is disclosed in Patent Documents 1 and 2, for example. When the pinching is detected, this pinching prevention function stops the rising window or turns the window glass from rising to lowering.

The pinching prevention function described in these documents detects the presence or absence of pinching by a window by looking at the pulse signal of a pulse sensor that detects the rotation speed of the motor. This pulse sensor is a sensor for viewing the position of the window by counting the number of pulses of the pulse signal, but when the window is caught, the rise of the window is restricted from that point and the number of rotations of the motor, that is, The number of pulses of the pulse signal decreases. Therefore, a power window device equipped with a pulse sensor adopts a method (pulse signal detection method) that determines the presence or absence of pinching by the window by looking at the change in the number of pulses per unit time of the pulse signal output by the pulse sensor. Is done.
JP 2004-092314 A Japanese Patent Laid-Open No. 2004-244958

  However, when the pinching detection is a pulse signal detection type, when a foreign object is actually pinched in the window, this is detected as being pinched. In this way, when the pinching detection is a pulse signal detection type, it is assumed that a foreign object has been pinched when the pinch detection is performed. However, a part of the human body such as a hand is sandwiched between windows. Therefore, even if the power window device is provided with a pinching prevention function to prevent pinching by the window, there is a problem that the user's discomfort is not solved.

  Here, as an example of preventing the pinching by the window, for example, a method of determining the presence or absence of a foreign object from an image captured by a camera is conceivable. As this method, it is necessary to specify a window portion image (window image area) in a captured image acquired from a camera and set a detection area as a reference for foreign object intrusion determination from the window image area. This detection area corresponds to an area where it can be determined that a foreign object has entered if foreign objects such as hands and fingers enter the area in the captured image.

  However, in the pinch detection using the captured image, the captured image captured around the window is captured in advance, and image data in which the detection area is set by adding the captured image to the captured image with, for example, paint software is used as a template. It is necessary to register in advance as data (vehicle window position data) in a memory. Therefore, in order to make this system compatible with a plurality of vehicle types, it is necessary to prepare template data for each vehicle type. Since this template data is image data having a large data size, a plurality of template data can be obtained in one system. It is difficult to have

  An object of the present invention is to provide a window image region detection device and a power window device that can automatically extract a window image region portion using a photographed image obtained from a photographing device such as a camera.

  In order to solve the above problems, according to the present invention, an image capturing unit that captures the periphery of a window frame and image data captured by the image capturing unit are used, and a window portion of a captured image is detected from a change in luminance of the image data. The gist of the invention is that the image processing apparatus includes extraction means for extracting a window image area as the area.

  According to this configuration, the area around the window frame is imaged by the imaging unit, and the extraction unit extracts the window image area from the area reflected in the captured image. Here, for example, when the vehicle type is different, the window shape and position reflected in the photographed image change, so that the window image region changes for each vehicle type. Therefore, when a window image area is prepared for each vehicle type, the user acquires a captured image in advance by the imaging unit, and image data in which the window image region is defined for the captured image is used as template data for the vehicle type. It is necessary to prepare for each. However, since this type of template data is image data, the amount of data per data is large, and it is practically difficult to have a plurality of template data in one system.

  In contrast, in the present invention, the extraction means automatically extracts the window image area from the photographed image of the photographing means. Therefore, even if the window image area changes for each vehicle type as described above, the template data for each vehicle type is used. It is not necessary to prepare a large amount of data relating to the window image area. In addition, for example, when the photographing means is assembled at the installation location, even if an assembling error occurs at the assembling position of the photographing means, the window will extract the increase area after assembling. Manual correction work is also unnecessary.

  According to the present invention, when detecting the movement of the object shown in the photographed image of the photographing means, the extracting means extracts the motion region using a variable threshold value that changes according to the illuminance at that time, and extracts the motion area. The gist is to extract the window image region by sequentially integrating the motion regions.

  According to this configuration, the window image region is calculated by extracting the motion region and integrating the motion region. When extracting the motion region, the motion region is extracted using a threshold value that changes according to the illuminance at that time. . Therefore, it is possible to extract the motion region with higher accuracy, and errors are less likely to occur in the extracted window image region.

  According to the present invention, there is provided a noise removing means for removing, from the window image area extracted by the extracting means, an area inappropriate as the window image as noise.

  According to this configuration, noise is removed from the window image area extracted by the extracting unit by the noise removing unit. Therefore, since an unsuitable part as a window is removed from the window image area, the detection accuracy of the window image area is increased, and an unnecessary part is hardly reflected in the extracted window image area.

  According to the present invention, the noise removing unit compares the window image area obtained by the extracting unit with a reference size of the window image area, recognizes an area protruding from the reference size as noise, and the area. Is removed from the window image area.

  According to this configuration, it is possible to remove noise from the extracted window image region by a simple method of comparing the window image region obtained by the extraction unit with its reference size.

  According to the present invention, the noise removing unit recognizes a moving region in a direction different from the integration direction as noise when obtaining the motion region and integrating the motion region, and removes the region from the window image region. The gist is to do.

  According to this configuration, it is possible to remove noise from the extracted window image area by a method with high noise detection accuracy, in which a motion area having a direction different from the integration direction of the motion area is recognized as noise and deleted from the window image area. Is possible.

The gist of the present invention is that a heat shielding material is used as the material of the window.
According to this configuration, if the material of the window is a heat shielding material, the window portion appears black in the captured image, that is, the image is reflected in a state where the luminance change is large relative to other portions. Highly effective in improving detection accuracy.

  According to the present invention, in a power window device having a pinching prevention function for preventing foreign objects from being pinched by a window that is moving up and down, a photographing unit that photographs the periphery of a window frame, and image data photographed by the photographing unit The extraction means for extracting the window image area as the area of the window portion in the photographed image from the change in luminance of the image data, and the detection area used for the foreign substance determination is set from the window image area extracted by the extraction means An image processing unit that detects the presence or absence of the pinching by monitoring the intrusion of foreign matter into the detection area using a feature amount obtained from the brightness reflected in the image data, and the image processing unit. When the pinching is detected, a safe operation that can prevent the pinching is controlled to raise and lower the window. And summarized in that and a running means for running the elevator control unit.

  According to the present invention, it is possible to automatically extract a window image region portion using a photographed image obtained from a photographing device such as a camera.

Hereinafter, an embodiment of a window image region detection device and a power window device embodying the present invention will be described with reference to FIGS.
As shown in FIG. 1, each door 2 of the vehicle 1 is provided with a power window switch (hereinafter referred to as a PW switch) 4 that is operated when the window 3 is moved up and down. The PW switch 4 is, for example, a two-stage click-type oscillating switch, and when one end side (downward side) is clicked one step, the window 3 is lowered while the click operation is being performed, and the one end side is moved two steps. When clicked, the operation mode is automatically lowered, and the window 3 is continuously lowered until the switch is operated again. The same movement is performed when the PW switch 4 is raised.

  In the vehicle 1, motor units 5 that automatically raise and lower the window 3 based on an operator's switch operation are disposed on the doors of the driver's seat, front passenger seat, rear right seat, and rear left seat, respectively. The motor unit 5 is equipped with a power window ECU (Electric Control Unit) 6 that controls the raising and lowering of the window 3. Connected to the power window ECU 6 are a drive motor 7 that is a drive source when the window 3 is raised and lowered, and a pulse sensor 8 that detects the rotational speed of the drive motor 7. The power window ECU 6 corresponds to the lift control means.

  The power window ECU 6 includes a CPU (Central Processing Unit) 9 that controls the ECU 6 in an integrated manner. A PW switch 4 is connected to the input side of the CPU 9 via a switch circuit 10 in the power window ECU 6, and a pulse sensor 8 is connected to each other via a pulse input circuit 11 in the power window ECU 6. A drive motor 7 is connected to the output side of the CPU 9 via a drive circuit 12 in the power window ECU 6.

  When the CPU 9 detects that the PW switch 4 is operated, the CPU 9 transmits the driving force of the driving motor 7 to the window 3 via the regulator 13 that converts the driving force of the driving motor 7 into the vertical movement force of the window 3. The window 3 is moved up and down. At this time, the CPU 9 determines the position of the window 3 by counting the pulses of the pulse signal Spa output from the pulse sensor 8.

  On the inner surface of each door 2, an infrared camera 14 capable of photographing the ascending / descending operation of the window 3 from the inside of the vehicle is disposed. The infrared camera 14 is a camera that irradiates a near-infrared LED 14a with a near-infrared ray to a camera photographing region and photographs the near-infrared reflected by a subject as a photographed image. Infrared rays such as near infrared rays are invisible light, so if you shoot using this type of infrared camera 14, you can shoot around the window frame of window 3 without being noticed by the driver at night. Is possible. Note that the infrared camera 14 corresponds to a photographing unit.

  For example, an image processing device 15 embedded in the inner wall of the door 2 is connected to each infrared camera 14 via a cable 16. The image processing device 15 is connected to the power window ECU 6 through a signal line 17. The image processing device 15 controls the operation of the infrared camera 14, acquires a captured image (image data Dpic) from the infrared camera 14 as a monochrome image, and uses this image data Dpic to display information such as a finger or a hand in the window 3. The pinch prevention control is performed to prevent the human body from being pinched. The image processing device 15 constitutes an extraction unit, a noise removal unit, an image processing unit, a setting unit, and an execution unit.

  When the infrared camera 14 receives the pulsed image request signal Sreq from the image processing device 15, the infrared camera 14 captures an image at the input timing and outputs the captured image Pic to the image processing device 15. Therefore, the image processing device 15 acquires the captured image Pic from the infrared camera 14 each time the image request signal Sre is output to the infrared camera 14, and thus the captured image Pic obtained every time the image request signal is transmitted is converted into image data. Get as Dpic.

  Further, if the infrared camera 14 inputs the image request signal Sreq with multiple pulses (short cycle) per unit time, the infrared camera 14 sequentially sends the captured images Pic to the image processing device 15 with a short cycle. For this reason, if the image processing device 15 is set to output the image request signal Sreq to the infrared camera 14 in a short cycle, the number of captured images that the image processing device 15 acquires from the infrared camera 14 increases, and the image processing device. No. 15 can obtain continuous images (moving images).

  The image processing apparatus 15 includes an image processing control unit (CPU) 18 that controls the apparatus 15 in an integrated manner. The image processing control unit 18 is connected to a cable 16 extending from the infrared camera 14 via an interface 19 in the image processing apparatus 15. A noise removal circuit 20 for removing noise in the image data is connected between the image processing control unit 18 and the interface 19 on the input line of the image data.

  Connected to the image processing control unit 18 are a ROM 21 that stores various programs, a RAM 22 that is used as a work area of the image processing control unit 18 when various types of processing are executed, and an EEPROM 23 that holds data in a rewritable state. . The ROM 21 stores an anti-pinch control program 24 that the image processing control unit 18 executes as anti-pinch control. The image processing control unit 18 operates along the pinching prevention control program 24 to detect the presence or absence of pinching by the window 3 from the image data acquired from the infrared camera 14.

  The image processing control unit 18 is connected to one end of a signal line 17 extending from the power window ECU 6 via an interface 25 in the image processing device 15. Further, the CPU 9 of the power window ECU 6 is connected to the other end of the signal line 17 via the interface 26 in the power window ECU 6. The image processing control unit 18 that has detected the inclusion of a foreign object can send a command to that effect via the signal line 17 to cause the power window ECU 6 to execute various safety operations.

  In the pinching prevention control program 24, a plurality of trapping detection processing algorithms (pinching detection methods) are registered. As an algorithm of this example, there are the following four patterns. That is, a method for detecting an optical flow from a captured image and detecting the movement of a foreign object (optical flow detection method), a method for determining a difference between captured images and detecting a movement of a foreign object (image difference detection method), There are a method (a feature amount detection method) for viewing a change in a feature amount (contour, area area) and a method (a marker monitoring detection method) for acquiring a change in a marker attached to a window frame from a captured image.

Next, details of these detection methods will be described in order.
By the way, as shown in FIG. 2, the captured image Pic captured by the infrared camera 14 also includes unnecessary portions (the surrounding scenery, etc.) other than the window 3, so that an area to be used when detecting pinching is cut out and detected. It is necessary to set as area E. Therefore, in this example, a captured image obtained by capturing the window frame 27 is acquired in advance, and a window frame peripheral region is cut out from the image using paint software or the like, thereby detecting this area when detecting pinching. As specified. Then, the image data specifying the detection area E is registered in advance in the EEPROM 23 as the template data Dtemp.

  First, an optical flow detection method will be described. Since the image processing control unit 18 inputs the window position information Dwi indicating the current position of the window 3 from the power window ECU 6 connected by the signal line 17, the window 3 is moving up and down, The current position can be recognized. Therefore, when the image processing control unit 18 detects that the window 3 is open, it starts outputting the image request signal Sreq to the infrared camera 14 and starts photographing around the window frame. Detection is performed using the image data Dpic after the removal.

  When the image processing control unit 18 acquires the image data Dpic from the infrared camera 14, the image processing control unit 18 obtains an optical flow 28 as shown in FIG. 3 from the image data Dpic, and reflects the image in the image using the optical flow 28. Is detected as a feature amount T (movement trajectory 29). Then, the image processing control unit 18 compares the detected position, direction, and area of the movement of the object with each foreign object determination threshold value, and determines whether or not the foreign object has entered the area sandwiched by the window 3. The foreign substance determination threshold referred to here is a set value that can determine that an object is a foreign object when the calculated characteristic value on the image exceeds this value. As a calculation method of the optical flow 28, for example, a gradient method, a block matching method, a Horn and BG Schunck method, a Lucas Kanade method, or the like is used.

  When determining the presence or absence of pinching using the optical flow 28, as the first processing, as shown in FIG. 5A, the luminance is obtained for each pixel (pixel) in the photographed image Pic, and the vertical axis direction of the screen is obtained. A luminance waveform 30 is obtained for each pixel column and each pixel column in the horizontal axis direction of the screen. For example, as in the example shown in FIG. 5A, when the luminance in the vicinity of the middle in a given column decreases, a luminance waveform 30 having a negative waveform slope in the middle portion is obtained.

  When calculating the optical flow, it is necessary to compare the preceding and following captured images Pic, so that the preceding and following captured images Pic are continuous, that is, the preceding and following captured images Pic are continuously connected. The luminance waveform 30 shown in FIG. 5A is converted into a correction waveform 31 shown in FIG. The correction waveform 31 is a waveform in which the inflection point of the luminance waveform 30 is gently curved. The process of obtaining the correction waveform 31 is performed in all columns in the screen vertical axis direction and the screen horizontal axis direction every time the image processing control unit 18 acquires the captured image Pic.

  Then, in the captured images Pic that are successively moved back and forth, a waveform difference La (shaded area in FIG. 5) as shown in FIG. 5C is calculated between the correction waveforms 31 of the respective columns. For example, if the correction waveform 31a indicated by the solid line in FIG. 5C is the correction waveform of the captured image Pic immediately after the correction waveform 31b shown in FIG. 5B, the correction waveform 31a is corrected between the correction waveform 31b and the correction waveform 31b. The waveform difference La is calculated by taking the difference at. Then, the optical flow 28 is calculated from the waveform difference La. In addition, the optical flow 28 is a value that represents how much the object moves in the next moment and how much the object flows, and can be said to be a value that can be obtained as the velocity of the object in other words. Then, the image processing control unit 18 calculates an optical flow 28 as shown in FIG. 3 for each pixel of the captured image Pic using the waveform difference La obtained from each pixel column in the screen vertical axis direction and the screen horizontal axis direction.

  Here, the details of the gradient method will be described. If the luminance of the pixel (x, y) on the image is f (x, y, z) and the point reaches time (t + dt), and the luminance moves to f (x + dx, y + dy, t + dt), Formula (1) is materialized.

f (x, y, z) = f (x + dx, y + dy, t + dt) (1)
Then, when the equation (1) is Taylor-expanded and the second and subsequent dx, dy, and dt are rounded down, the following equation (2) is established.

f _x (x, y, z) · u + f _y (x, y, z) · v + f _t (x, y, z) = 0 (2)
Thus, f (x + dx, y + dy, t + dt) can be obtained by solving equation (2). However, since coefficient u and coefficient v are unknown values, it is necessary to first obtain coefficient u and coefficient v. . Therefore, a constraint equation for the gradient method is created using two filters as shown in the following equations (3) and (4).

(g * f) _x · u + (g * f) _y · v + (g * f) _t = 0… (3)
(h * f) _x · u + (h * f) _y · v + (h * f) _t = 0… (4)
The filter g and the filter h have two conditions: they are different because they are differentiable and are continuous, and these two constraint equations are independent. Among these, the condition that it is continuous because it can be differentiated corresponds to a portion in which the luminance waveform 30 shown in FIG. 5A is smoothed to the correction waveform 31 shown in FIG. 5B. Then, the coefficient u and the coefficient v are calculated from the expressions (3) and (4), and the coefficient u and the coefficient v are substituted into the expression (2) and solved to obtain f (x + dx, y + dy, t + dt) And the optical flow 28 is obtained.

  That is, the outline of the gradient method is shown in FIG. 6. First, the original image is strongly smoothed in the x direction to obtain a smooth image Px, and the original image is strongly smoothed in the y direction to obtain the smoothed image Py. Ask for. The coefficients u and v are obtained using the smooth images Px and Py, and the speed and direction of the image are obtained using the coefficients u and v.

  The image processing control unit 18 that has obtained the optical flow 28 detects the movement of the object using the optical flow 28, and foreign objects are detected in the region sandwiched by the window 3 from the position, direction, and area of the detected movement of the object. It is determined whether it has entered. That is, as shown in FIG. 4, the optical flow 28 in the same direction is connected to determine the movement of the object (movement trajectory 29), and the position, direction, and area of this movement are compared with the respective threshold values. If these parameters exceed the foreign object determination threshold, it is determined that the foreign object has entered the area sandwiched by the window 3.

  As a specific processing example, the image processing control unit 18 sequentially monitors the movement (movement trajectory 29) of an object, and the object has entered the detection area E with the movement of a human hand or finger. When detected, the object has an area that can be determined as a foreign object, and it is determined whether the detection area intrusion direction of the object obtained from the movement locus 29 is the direction of movement of the hand or finger (for example, the horizontal axis direction of the screen). To do. When these conditions are met, the image processing control unit 18 determines that a foreign object has entered the area sandwiched by the window 3 and outputs a safe operation request signal Ssaf to the power window ECU 6 via the signal line 17.

  Receiving the safe operation request signal Ssaf, the power window ECU 6 executes a safe operation to avoid pinching by the window 3. As this safe operation, for example, when the window 3 is stopped, an operation of prohibiting the window from rising during a predetermined time, or when the window 3 is rising, the window 3 is stopped at that position, or There is an operation of lowering the window 3 to a safe position. Further, as other safe operation, an operation of giving a warning to the operator by voice or light by an alarm device installed in the vehicle may be used.

  Next, an image difference detection method will be described. When the image processing control unit 18 acquires the image data Dpic from the infrared camera 14, the images of the recognized objects 32 a and 32 b reflected between the two consecutive captured images Pic 1 and Pic 2 as shown in FIG. The difference M is taken, and using this image difference M, the movement of the object (movement trajectory 33) shown in FIG. Then, the image processing control unit 18 compares the detected position, direction, and area of the movement of the object with each foreign object determination threshold value, and determines whether or not the foreign object has entered the area sandwiched by the window 3.

  When determining the presence / absence of pinching using the image difference M, first, the image processing control unit 18 determines whether the video object reflected in the captured image Pic is an object based on the size and color of each individual reflected in the captured image Pic. Recognize whether or not. The individual recognition on the captured image Pic is performed by comparing the brightness on the image with the object determination threshold, and the part exceeding the object determination threshold is recognized as the recognition object 32a (32b). The object determination threshold mentioned here is a set value that can be recognized as an object when the calculated characteristic value on the image exceeds this value. The image processing control unit 18 that has performed the object recognition takes the image difference M of the recognized object as shown in FIG. 7 in the captured images Pic that are successively back and forth, and obtains the movement of the recognized object from the image difference M. This is performed every time the image data Dpic is obtained from the infrared camera 14, and the movement of the object (movement locus 32) shown in FIG. 8 is obtained.

  As a specific processing example, the image processing control unit 18 that has captured an image Pic from the infrared camera 14 and performed object recognition calculates a center of gravity G on the image of the recognized object. The center of gravity G is a value calculated in units of pixels of the captured image Pic, and is calculated by the following method. That is, the sum of all of the x and y coordinate values of the recognized object is calculated, and the average value of the x and y coordinate values is calculated by dividing the sum by the number of pixels of the recognized object. The obtained x-coordinate average value and y-coordinate average value are used as the barycentric coordinates.

  The image processing control unit 18 calculates the center of gravity of the recognized object 32a (32b) every time the captured image Pic is acquired from the infrared camera 14, and calculates the difference between the centers of gravity G (see FIG. 8) between these images as the image difference M. This is derived as the motion of the object. Then, the image processing control unit 18 derives a path of the recognized object by connecting the centers of gravity G of the recognized objects 32a, 32b,...

  The image processing control unit 18 monitors the position, direction, and area of the movement (movement trajectory 33) of the object in the same manner as in the optical flow detection method, and determines whether or not a foreign object has entered the area sandwiched by the window 3. judge. In this determination, when the image processing control unit 18 determines that a foreign object has entered the region sandwiched by the window 3, the image processing control unit 18 outputs a safe operation request signal Ssaf to the power window ECU 6 via the signal line 17. When the power window ECU 6 receives the safe operation request signal Ssaf, the power window ECU 6 executes the safe operation as described in the optical flow detection method.

  Next, a feature amount detection method will be described. When image data Dpic is input from the infrared camera 14, the image processing control unit 18 extracts an image of the detection area E (see FIG. 2) for each input captured image Pic, and the image feature amount in the detection area E T is calculated. Examples of the feature amount T include the outline 34 of the window frame 27 shown in FIG. 9C, the area 35 of the window frame 27 shown in FIG. 12A, and the luminance itself (luminance change). The value of may be used. The image processing control unit 18 compares the feature amount T with a foreign matter determination threshold value, and determines whether or not a foreign matter has entered a region sandwiched by the window 3.

  Here, for example, when the contour 34 of the window frame 27 shown in FIG. 9C is used as the feature amount T, the processing performed by the image processing control unit 18 is a window when no foreign matter such as a finger or a hand enters. In order to recognize the outline of the frame 27 (hereinafter referred to as the initial outline 34a (see FIG. 9C)), a process for obtaining the luminance waveform 36 as shown in FIG. Do. The luminance waveform 36 is a waveform obtained by connecting the luminances of the respective pixels (pixels) of the captured image Pic along the screen horizontal axis direction, and is obtained for each column in the screen horizontal axis direction.

  The image processing control unit 18 shows the luminance waveform 36 obtained for each column extending in the horizontal direction of the screen in the captured image Pic, as shown in FIG. 9B in which the waveform inclination (in this example, the absolute value) is changed to the edge height. Each is converted into a luminance edge waveform 37. Then, as shown in FIG. 9C, the image processing control unit 18 performs processing for connecting portions where the edge 38 exceeds the object determination threshold in the luminance edge waveforms 37 adjacent to each other in the vertical axis direction of the screen. Recognized as the initial outline 34 a of the frame 27. The image processing control unit 18 derives the outline 34 of the window frame 27 every time the captured image Pic is acquired from the infrared camera 14.

  Here, for example, as shown in FIG. 10, when the window frame 27 is blocked by a hand or a finger, near infrared light is reflected by this foreign substance in front of the normal, so that the luminance waveform 36 has a luminance at the foreign substance portion. The waveform shown in FIG. 11 (a), which is increased (this is referred to as luminance waveform 36a), is taken. The image processing control unit 18 converts the luminance waveform 36a into a luminance edge waveform. When the luminance waveform 36a is converted into a waveform, the edge position changes from the initial state in FIG. 11B. The luminance edge waveform 37a shown is derived. Therefore, the image processing control unit 18 is in a state where an outline of the window frame 27 different from the normal one (hereinafter referred to as a modified outline 34b (see FIG. 11C)) is obtained.

  The image processing control unit 18 constantly monitors the contour 34 of the window frame 27, and whether or not the contour difference (for example, the difference in the length of the contour line) Lb between the deformed contour 34b and the initial contour 34a exceeds the foreign substance determination threshold value. Sequentially monitor. When the image processing control unit 18 determines that the contour difference Lb has exceeded the foreign object determination threshold, the image processing control unit 18 recognizes that the foreign object has entered the area sandwiched by the window 3 and sends the safe operation request signal Ssaf to the power window via the signal line 17. It outputs to ECU6. When the power window ECU 6 receives the safe operation request signal Ssaf, the power window ECU 6 executes the safe operation as described in the optical flow detection method and the image difference detection method. Note that, when foreign matter detection is performed using a luminance change, foreign matter determination can be performed based on the same principle as that described for the edge.

  When the area area 35 of the window frame 27 shown in FIG. 12A is used as the feature amount T, the image processing control unit 18 shown in FIG. 12A when a foreign object such as a hand or a finger does not enter. An initial contour 34a of the object is obtained, and a region area surrounded by the contour 34a (hereinafter referred to as an initial region area 35a) is obtained. The image processing control unit 18 derives a region area 35 surrounded by the contour 34 every time the captured image Pic is acquired from the infrared camera 14.

  The image processing control unit 18 constantly monitors the area area of the window frame 27, and the area difference between the area area when the area value of the area area 35 is changed (hereinafter referred to as a deformed area area 35b) and the initial area area 35a. It is sequentially monitored whether or not Lc exceeds the foreign substance determination threshold value. When the image processing control unit 18 determines that a foreign matter such as a hand or a finger enters the detection area E and the area difference Lc exceeds the foreign matter determination threshold as shown in FIG. And the safe operation request signal Ssaf is output to the power window ECU 6 via the signal line 17 to cause the safe operation request signal Ssaf to be executed.

  Next, a marker monitoring detection method will be described. As shown in FIG. 13, a marker 39 is applied around the window frame of the door 2 at a position where it is reflected on the infrared camera 14. The marker 39 is made of a paint or pigment having a high light reflectance, and in this example, a material having a high near-infrared light reflectance is used. Therefore, when near infrared light from the near infrared LED 14a is irradiated around the window frame, a lot of light is reflected (strongly) at the marker 39 portion, and the image captured by the infrared camera 14 has a luminance at the marker 39 portion. Reflects in a high state.

  When the image processing control unit 18 inputs the image data Dpic from the infrared camera 14, the image processing control unit 18 acquires the image area of the marker 39 as shown in FIG. An image feature amount T is obtained in the image region 40. As the feature amount T, for example, the outline 41 of the marker image area 40 shown in FIG. 14C, the area 42 of the marker image area 40 shown in FIG. The image processing control unit 18 compares the feature amount T with a foreign matter determination threshold value, and determines whether or not a foreign matter has entered a region sandwiched by the window 3. The image processing control unit 18 determines whether or not the marker 39 is broken or contour is deformed by using the obtained feature amount T.

  An example of this processing will be described. For example, when the contour 41 of the marker image area 40 shown in FIG. 14C is used as the feature amount T, the processing performed by the image processing control unit 18 is based on the first captured image Pic. The outline of the marker image area 40 when a foreign object such as a hand or a finger does not enter the marker image area 40 (hereinafter referred to as an initial outline 41a) is recognized. In this processing, since the light with high luminance is reflected from the marker 39, the image processing control unit 18 takes in the light with high luminance as the marker image region 40, as shown in FIG. A bright luminance waveform 43 is obtained. The luminance waveform 43 is a waveform obtained by connecting the luminance of each pixel (pixel) of the captured image along the horizontal axis direction of the screen, and is obtained for each column in the horizontal axis direction of the screen.

  The image processing control unit 18 converts the luminance waveform 43 obtained for each column extending in the horizontal direction of the screen in the captured image into the luminance shown in FIG. 14B in which the waveform inclination (absolute value in this example) is converted into the edge height. Each is converted into an edge waveform 44. Then, as shown in FIG. 14C, the image processing control unit 18 performs processing for connecting portions where the edge 45 exceeds the object determination threshold in the luminance edge waveforms 44 adjacent to each other in the vertical axis direction of the screen, It is recognized as the initial contour 41a of the image area 40. The image processing control unit 18 derives the contour 41 of the marker image area 40 every time the captured image Pic is acquired from the infrared camera 14.

  Here, for example, as shown in FIG. 15, when the marker 39 is blocked by a hand or the like, the light reflected by the marker 39 is blocked thereby, and the luminance reflected on the infrared camera 14 becomes lower than usual at the blocked portion. The luminance waveform 43 is a waveform in which the luminance is lowered at the foreign material portion (this is referred to as a luminance waveform 43a). The image processing control unit 18 converts the luminance waveform 43a into a luminance edge waveform. When the luminance waveform 43a is converted into a waveform, the edge position is changed as shown in FIG. The luminance edge waveform 44a shown will be derived. Therefore, the image processing control unit 18 obtains a contour of the marker image region 40 different from the normal one (hereinafter referred to as a modified contour 41b (see FIG. 16C)).

  The image processing control unit 18 constantly monitors the contour 41 of the marker image region 40 and sequentially monitors whether or not the contour difference Ld between the deformed contour 41b and the initial contour 41a exceeds the foreign substance determination threshold value. When the image processing control unit 18 determines that the contour difference Ld has exceeded the foreign object determination threshold, the image processing control unit 18 recognizes that the foreign object has entered the region sandwiched by the window 3 and sends the safe operation request signal Ssaf to the power window via the signal line 17. It outputs to ECU6 and makes power window ECU6 perform safe operation | movement.

  Further, when the area of the marker 39 is used as the feature amount T, the image processing control unit 18 initializes the marker image region 40 when a foreign object such as a hand or a finger does not enter, as in the case of the feature amount detection method. A region area 42a (see FIG. 17A) is obtained. Then, the image processing control unit 18 calculates the area difference Le between the deformation area 42b (see FIG. 17B) and the initial area 42a when a foreign object such as a hand or a finger enters the marker image area 40. Monitor sequentially. When the image processing control unit 18 determines that the area difference Le has exceeded the foreign object determination threshold value, the image processing control unit 18 recognizes that the foreign object has entered the region sandwiched by the window 3 and sends the safe operation request signal Ssaf to the power window via the signal line 17. It outputs to ECU6 and performs safe operation request signal Ssaf.

  Next, processing when the image processing control unit 18 performs the pinching prevention control will be described with reference to a flowchart shown in FIG. This pinching prevention control is started on the condition that, for example, the engine switch of the vehicle is operated to the ACC position, and is repeatedly executed in units of a predetermined cycle (for example, in units of several hundred μs).

  In step 100, it is determined whether or not the window 3 is open. That is, since the image processing control unit 18 takes in the window position information Dwi from the power window ECU 6, the image processing control unit 18 determines whether or not the window 3 is in an open state using the window position information Dwi. At this time, if the window 3 is closed, the process proceeds to step 101, and if the window 3 is open, the process proceeds to step 102.

In step 101, without starting the image request signal Sreq to the infrared camera 14, the activation of the infrared camera 14 is left in the OFF state.
In step 102, output of the image request signal Sreq to the infrared camera 14 is started, and the infrared camera 14 is turned on. The infrared camera 14 that has been turned on starts near-infrared light irradiation from the near-infrared LED 14a and also starts image capturing around the window frame. At this time, the image processing control unit 18 continuously outputs the image request signal Sreq to the infrared camera 14 and acquires the image data Dpic from the infrared camera 14 as a continuous captured image.

  In step 103, the illuminance K around the window 3 is calculated using the image data Dpic obtained from the infrared camera 14. For example, the image processing control unit 18 calculates the illuminance K by using the first captured image among the continuously acquired captured images.

  In step 104, an algorithm used for pinching detection is selected using the calculated illuminance K. In this example, since there are four patterns of an optical flow detection method, an image difference detection method, a feature amount detection method, and a marker monitoring detection method, a method that matches the illuminance K at that time is selected from these four patterns. For example, since the contour of the captured image is strongly raised when the ambient illuminance of the window 3 is low, it is preferable to determine the foreign matter using the feature amount T. As an entrapment detection algorithm when the illuminance K is low, a feature amount detection method is used. And marker monitoring detection techniques are used. On the other hand, when the illuminance around the window 3 is high, it is less likely that a foreign object is erroneously detected when the image motion is observed than when the feature amount T is used. An optical flow detection method or an image difference detection method is used.

  In step 105, using the calculated illuminance K, threshold values (object determination threshold value, foreign object determination threshold value) for use in the algorithm are selected. That is, the threshold used when the pinch detection is performed is set to a value corresponding to the illuminance K.

In step 106, a pinching detection process using the image data Dpic is executed by executing the selected algorithm.
In step 107, the presence / absence of a foreign object is determined by checking whether or not a foreign object has entered the position of being sandwiched in the window 3 as a result of the pinching detection process. If it is determined that there is a foreign object, the process proceeds to step 108. If it is determined that there is no foreign object, this flowchart is terminated.

In step 108, a safe operation request signal Ssaf is output to the power window ECU 6 to cause the power window ECU 6 to execute a safe operation.
Therefore, in this example, since the presence / absence of pinching is detected using the image data Dpic acquired from the infrared camera 14, the presence / absence of pinching by the window 3 can be detected even if hands or fingers are not actually pinched by the window 3. It can be detected in advance. Therefore, when detecting pinching, there is no situation in which a hand, a finger, or the like is pinched by the window 3 at that time, so that the user does not have to feel uncomfortable when pinching is detected.

  Next, the template data Dtemp specifying the detection area E by paint software or the like is not registered in the memory (EEPROM 23) in advance, but the detection area E is automatically extracted using the image data Dpic acquired from the infrared camera 14. The detection area extraction function to be performed will be described below. By the way, since the arrangement position and window frame shape of the infrared camera 14 change for each vehicle type, the detection area E (window position data) also changes for each vehicle type. Therefore, it is conceivable to register a plurality of template data Dtemp in the EEPROM 23 to cope with a plurality of vehicle types. However, since the template data Dtemp is image data, the data size is large and a plurality of template data Dtemp are registered in one system. It is practically difficult to keep it.

  Further, when the infrared camera 14 is actually assembled to the door 2, an assembly error may occur in the infrared camera 14, and there is a difference between the actually acquired image data Dpic and the template data Dtemp registered in advance. May occur. In this case, a process of reassembling the infrared camera 14 or correcting the template data Dtemp in the EEPROM 23 is required. Even if the assembling error is eliminated by this method, this process takes time. Therefore, it is not practical in practice.

  Therefore, the image processing control unit 18 of this example extracts an image area of the window 3 (hereinafter referred to as a window image area 46 (see FIG. 21)) from the image data Dpic in which the window 3 is reflected, and the window 3 The detection area E is specified by extracting the detection area E from the image area. The image processing control unit 18 starts this detection area extraction processing when the window 3 is fully opened.

  As this detection area extraction processing, the image processing control unit 18 moves the window 3, detects the movement by a method similar to the optical flow detection method and the image difference detection method, and extracts the detection area E based on the detection method. When the motion detection is performed, the image processing control unit 18 obtains a captured image Pic from the infrared camera 14, and the optical flow 28 related to the captured image Pic or the luminance change between two consecutive images determines which object is between the consecutive images. The movement of the object as shown in FIG. 19 indicating whether it has moved to the position is detected. At this time, the object recognition performed by the image processing control unit 18 is performed by checking whether or not the luminance in the captured image Pic has exceeded an object determination threshold value that is a reference for object determination.

  Then, the image processing control unit 18 extracts the motion region 47 that has moved on the captured image Pic in this way with the value of the number of pixels. The image processing control unit 18 performs processing for integrating the extracted motion region 47 until the window 3 is fully closed, and obtains this integrated region as the window image region 46. In FIG. 19 and FIG. 21 and FIG. 22 described later, for the sake of convenience of explanation, an image of the door 2 viewed from the side is taken as a photographed image Pic. The image shown in FIG. 23 is taken from the side.

  When the window 3 is irradiated with other light such as disturbance light, for example, the luminance reflected in the captured image Pic as the window 3 changes as the window 3 moves up and down as shown in FIG. For example, when the window 3 starts to close from the fully closed state, an image of the window 3 can be obtained with a normal luminance. However, when the window 3 is fully closed, if the upper end of the window is affected by ambient light, that portion The image of the window 3 is obtained in a state where the brightness of is low. Therefore, when recognizing the portion of the window 3 from the captured image Pic, for example, if image recognition is to be performed using one threshold value until the fully opened window 3 is closed, the threshold level needs to be as low as possible. For this reason, there is a high possibility that a portion that is not actually the window 3 is taken in as the window image area 46, which leads to erroneous determination of the window image area 46.

  Therefore, in this example, the image processing control unit 18 calculates the illuminance K at that time from the luminance value of the captured image Pic each time the captured image Pic is acquired and the motion region 47 is extracted, and the illuminance K is calculated. Threshold values (object determination threshold value, foreign object determination threshold value) are set to the corresponding values. As described above, if the object determination threshold value is set each time when the motion region 47 is extracted, the optimum object determination threshold value can be used for the motion region extraction. Is less likely to occur.

  The image processing control unit 18 that has extracted the window image area 46 removes noise from the window image area 46. By the way, the image processing control unit 18 grasps in advance the size of the window 3, that is, the window image area that can be taken when the window 3 is closed (hereinafter referred to as the reference size 46a) from the pixel pattern data. Therefore, as noise removal processing, first, when calculating the window image region 46, the image processing control unit 18 compares the window image region 46 with the reference size 46a, and a portion that protrudes from the reference size 46a as shown in FIG. Is regarded as an object unexpectedly reflected in the captured image Pic, and is erased from the window image area 46.

  In addition, as another noise removal process, the image processing control unit 18 can recognize a part that has moved in a direction different from the image integration direction as noise and delete it from the window image area 46. That is, in this example, since the window image area 46 is obtained by detecting the movement of the object reflected in the captured image Pic, the movement direction of the object can be recognized. Therefore, when the image processing control unit 18 recognizes the direction in which the amount of the movement direction is large as the image integration direction, and recognizes the direction of the arrow A illustrated in FIG. 22 as the image integration direction, for example, the image acquired from the movement in the other direction. (The image of the dot region in FIG. 22) is deleted from the window image region 46.

  The image processing control unit 18 erasing noise from the window image area 46 extracts the detection area E using the window image area 46 after noise elimination. That is, if the window image area 46 is known on the captured image Pic, the upper edge of the window 3 when the window 3 is closed can be recognized. Therefore, the image processing control unit 18 sets, for example, a region of several tens of pixels on the upper side (dot region in FIG. 23) as the detection area E along the vertical axis direction of the screen with reference to the upper edge 48 of the window 3. Is written in the EEPROM 23.

  Accordingly, in this example, the detection area E can be set as appropriate from the image data Dpic of the infrared camera 14. Therefore, even if the mounting destination of the image processing apparatus 15 is spread over a plurality of vehicle types, it is not necessary to prepare the template data Dtemp for the detection area E for each vehicle type, which increases costs such as preparing a large-capacity memory. There is no connection problem. Further, even if an assembly error occurs when the infrared camera 14 is assembled to the door 2, the detection area E may be set by performing the detection area extraction process after the assembly, so that the infrared camera 14 may be reassembled. Or, it is not necessary to perform troublesome work such as modifying the template data Dtemp.

According to the configuration of the present embodiment, the following effects can be obtained.
(1) A window area 46 is extracted from the image data Dpic of the infrared camera 14, and a detection area extraction function for detecting the area E from the window image area 46 is provided. Accordingly, even if the image processing apparatus 15 is mounted on a plurality of vehicle types, it is not necessary to prepare the template data Dtemp for the detection area E for each vehicle type, and the cost increases due to the provision of a large-capacity memory. Does not occur. In addition, even if an assembly error occurs when the infrared camera 14 is assembled to the door 2, the detection area E may be set by performing the detection area extraction process after the assembly. This eliminates the need for data correction work for the template data Dtemp.

  (2) Each time the captured image Pic is acquired from the infrared camera 14, the motion region 47 is extracted, and the window image region 46 is extracted by integrating the motion region 47. When extracting the motion region 47, the illuminance K at that time is extracted. The motion region 47 is extracted using threshold values (object determination threshold value, foreign object determination threshold value) that change according to the above. Therefore, the motion region 47 is extracted with higher accuracy, and the window image region 46 and thus the detection area E can be obtained with higher accuracy.

  (3) When extracting the window image area 46, the size of the window 3 (reference size 46a) shown in the captured image Pic, the integration direction of the motion area 47 when obtaining the motion area 47 (arrow A direction in FIG. 22), etc. Is used to eliminate noise from the window image area 46. Accordingly, the detection accuracy when obtaining the window image region 46 from the captured image Pic is increased, and as a result, the detection accuracy of the detection area E is improved.

  (4) When the method of viewing the size of the window image area 46 is used as the noise removal technique, the noise portion is erased from the window image area 46 by a simple technique of comparing the window image area 46 and the reference size 46a. it can.

  (5) When the method of viewing the integration direction of the motion region 47 is used as the noise removal method, the noise portion can be erased from the window image region 46 by a method with high noise detection accuracy of viewing the motion on the captured image. .

  (6) When a heat shielding material is used as the material of the window 3, when the window 3 is photographed by the infrared camera 14, a difference in luminance between the window portion and the other portions in the photographed image becomes large. In Pic, the window 3 appears black on the photographed image. Therefore, when the window image area 46 is obtained from the captured image Pic, the window image area 46 can be extracted with high accuracy.

  (7) Since the pinching detection is performed using the image data Dpic acquired from the infrared camera 14, the pinching can be detected before the hand or the finger is actually pinched in the window 3. Therefore, it is possible to detect the pinching by the window 3 without giving the user unpleasant feeling.

  (8) When an optical flow detection method or an image difference detection method is used as an entrapment detection algorithm, entrapment is detected by looking at the movement of an object reflected in the captured image Pic. By the way, since the movement of the human body such as a hand or a finger takes its own movement, if the pinching detection is performed by looking at the movement of the object shown in the captured image Pic, it is determined whether or not the human body has entered the detection area E. It can be determined with high accuracy. Therefore, if an optical flow detection method or an image difference detection method is used as an entrapment detection algorithm, it is possible to prevent erroneous detection during entrapment detection.

  (9) When the marker monitoring detection method is used as the trapping detection algorithm, the marker image area 40 is extracted from the image data Dpic of the infrared camera 14 and used as a foreign substance determination reference area in the foreign substance intrusion determination. Therefore, for example, it is not necessary to register the template data Dtemp specifying the detection area E in the memory such as the EEPROM 23 by marking the image with paint software or the like, and it is not necessary to perform troublesome data editing operations.

  (10) A plurality of algorithms are prepared as a method for detecting pinching using image processing, and an optimum algorithm corresponding to the illuminance at the time of pinching detection is selected from the plurality of algorithms to perform pinching detection. . Therefore, it is possible to accurately determine whether or not a foreign object has entered the detection area E (marker is the image area 40), and erroneous determination is less likely to occur when detecting the presence or absence of pinching.

  (11) An infrared camera 14 is used that irradiates near infrared LED 14a with a near infrared ray that is invisible to human eyes, and uses the near infrared ray bounced off by the subject to capture an image around the window frame. Therefore, in a dark place such as at night, an image (motion) around the window frame can be taken without affecting the human field of view or illuminating unnecessary light around the window frame. Further, although it is conceivable to use a model that irradiates infrared rays as the infrared camera 14, since there is a wavelength region in which infrared rays are not reflected in the infrared camera 14, there is a possibility that a portion where an image is not reflected may occur depending on circumstances. However, if near infrared rays are used, this type of problem is less likely to occur.

(12) Since the infrared camera 14 is activated when the window 3 is in the open state, it is effective for power saving of the in-vehicle battery.
In addition, this embodiment is not limited to the said structure, For example, you may change to the following aspects.

  The pinching detection by the window 3 is not limited to determining that pinching is present on the condition that an object that seems to be a foreign object has entered the detection area E (marker image area 40). For example, it may be determined that the object is caught on the condition that an object that seems to be a foreign object enters the detection area E (marker image area 40) and the window 3 is moving up and down at that time.

  The pinching detection function of this example using image processing may be used in combination with the conventional pinching detection function for detecting pinching by the output of the pulse sensor 8 as described in the background art. In this case, for example, even if the pinching detection function based on image processing fails, the presence or absence of pinching can be monitored by the pinching detection function based on pulse output, so that fail-safe can be satisfied.

-It is not limited to preparing a plurality of trapping detection algorithms, and only one of the four patterns described in the embodiment may be used.
The photographing means for photographing the periphery of the window frame is not necessarily limited to the infrared camera 14, and for example, a color camera may be used. When a color camera is used as the photographing means and the pinching detection is performed by the feature amount detection method or the marker monitoring detection method, it is possible to perform foreign object determination using the hue or saturation as the feature amount T.

  When extracting the detection area E from the image data Dpic of the infrared camera 14 using the detection area extraction function, the area corresponding to several tens of pixels from the upper edge of the extracted window image area 46 is set as the detection area E. Can be appropriately set as desired.

  -You may employ | adopt heat insulation glass for the material of the window 3. As shown in FIG. In this case, when shooting is performed with the infrared camera 14, the window portion is reflected darkly, and the determination accuracy when performing object determination from the image data Dpic can be improved.

  -The adoption object of the pinching detection function by the image processing of this example is not necessarily limited to the door 2 of the vehicle 1, but may be a power window device of another device such as a door of a house, for example.

Next, technical ideas that can be grasped from the above-described embodiment and other examples will be described below together with their effects.
(1) In Claim 7, the image processing means detects a motion of an object reflected in the image data based on the feature amount, and detects the presence or absence of the pinching from the motion.

  (2) In claim 7 and the technical idea (1), the window frame is provided with a light reflecting material having a high light reflectance, and the image processing means reflects the light from an image obtained by photographing the light reflecting material. An image region of the material is acquired, the feature amount is obtained in this region, and the presence or absence of the sandwiching is detected from the change in the feature amount.

  (3) In claim 7 and the technical ideas (1) and (2), the illuminance calculating means for calculating the illuminance around the window frame from the image data of the photographing means is different from the detection method for the presence or absence of pinching. A storage unit storing a plurality of algorithms, wherein the image processing unit selects an algorithm corresponding to the illuminance from the plurality of algorithms in the storage unit, and detects the presence or absence of the pinching using the algorithm To do.

  (4) In Claim 7 and the technical ideas (1) to (3), the photographing means emits near infrared light around the window frame and the presence or absence of the pinching using the near infrared light reflected by the subject. Is detected.

  (5) In claim 7 and the technical ideas (1) to (4), the image processing unit includes an illuminance calculating unit that calculates an illuminance around the window frame from the image data of the imaging unit, In an algorithm executed when detecting the presence / absence of pinching, a threshold used in the algorithm is set to a value corresponding to the illuminance, and the presence / absence of pinching is detected using the threshold. In this case, when detecting pinching, the illuminance around the window frame is calculated, and the threshold used when detecting pinching by executing the algorithm is set to a value corresponding to the illuminance at that time. Therefore, when executing the algorithm, it is possible to detect pinching with an optimum threshold value according to the illuminance at that time, and it is very effective in preventing erroneous determination when determining whether pinching has occurred.

The block diagram which shows the structure of the power window apparatus in one Embodiment. FIG. 6 is a photographed image view around a window frame photographed by an infrared camera. The conceptual diagram showing the optical flow calculated | required from a picked-up image. The conceptual diagram at the time of explaining how to obtain the movement of an object by the optical flow detection method. It is explanatory drawing of an optical flow detection method, (a) is the luminance waveform figure of the pixel column of the predetermined screen horizontal axis direction calculated | required from the brightness | luminance of the picked-up image, (b) is the luminance edge waveform figure, (c) is continuous. The wave form diagram showing the waveform difference of the brightness | luminance edge waveform between picked-up images. Schematic for demonstrating the gradient method. The conceptual diagram explaining the outline of an image difference detection method. The conceptual diagram at the time of explaining how to obtain the motion of an object by the image difference detection method. It is explanatory drawing of the feature-value detection method when the outline in a detection area is used as a feature-value, (a) is a luminance waveform figure of the pixel row of the predetermined screen horizontal axis direction calculated | required from the brightness | luminance of the picked-up image, (b). Is a luminance edge waveform diagram, (c) is a waveform diagram showing the waveform difference of the luminance edge waveform between consecutive captured images. The photographed image figure of the window frame periphery when a foreign material penetrate | invades into a detection area. It is explanatory drawing at the time of detecting the foreign substance invasion by the feature amount detection method, (a) is a luminance waveform diagram of a pixel column in a predetermined screen horizontal axis direction obtained from the luminance of the photographed image at that time, (b) is the luminance edge The waveform diagram, (c) is a waveform diagram showing the waveform difference of the luminance edge waveform between consecutive captured images. It is explanatory drawing of the feature-value detection method when the area | region area in a detection area is used as a feature-value, (a) is a picked-up image figure when the foreign material has not penetrate | invaded the detection area, (b) is a detection area. The photographed image figure when a foreign material invades. The side view of the door used when explaining the outline of a marker monitoring detection method. It is explanatory drawing of a marker monitoring detection method, (a) is the luminance waveform figure of the pixel row | line of the predetermined screen horizontal axis direction calculated | required from the brightness | luminance of the picked-up image, (b) is the luminance edge waveform figure, (c) is a marker image. FIG. 6 is a photographed image diagram when no foreign matter has entered the area. FIG. 6 is a photographed image view around a window frame when a foreign object enters the marker image area. It is explanatory drawing when a foreign substance intrusion is detected by the marker monitoring detection method, (a) is a luminance waveform diagram of a pixel column in a predetermined screen horizontal axis direction obtained from the luminance of a captured image at that time, and (b) is a luminance edge thereof Waveform diagram, (c) is a photographed image at that time. It is explanatory drawing of the marker monitoring detection method when the area | region area of a marker image area is used as a feature-value, (a) is a picked-up image figure when the foreign material has not penetrate | invaded a marker image area, (b) is a marker image. FIG. 6 is a photographed image diagram when a foreign object enters an area. The flowchart of the pinching prevention control performed at the time of pinching detection. The side view of the door used when explaining a detection area extraction function. The wave form diagram which shows the luminance change according to the window position of the predetermined location of a window. Explanatory drawing at the time of erasing a size-like noise from a window image area. Explanatory drawing which erases the part which is not an integration direction from a window image area as noise. The picked-up image figure at the time of specifying a detection area from a window image area | region.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 3 ... Window, 6 ... Power window ECU as raising / lowering control means, 14 ... Infrared camera as imaging | photography means, 15 ... Extraction means, Noise removal means, Image processing means, Setting means, The image processing apparatus which comprises an execution means, 27 ... window frame, 46 ... window image area, 46a ... reference size, 47 ... motion area, Dpic ... image data, Pic ... photographed image, T ... feature, K ... illuminance, E ... detection area, A ... accumulation direction Arrow direction.

Claims (7)

Photographing means for photographing the periphery of the window frame;
A window image area detection apparatus comprising: extraction means for extracting a window image area as an area of a window portion in a photographed image using image data photographed by the photographing means from a change in luminance of the image data.   The extraction means, when detecting the motion of an object shown in the photographed image of the photographing means, extracts the motion area using a variable threshold that changes according to the illuminance at that time, and sequentially integrates the extracted motion areas. The window image area detection apparatus according to claim 1, wherein the window image area is extracted by the above-described process.   The window image area detection apparatus according to claim 1, further comprising: a noise removal unit that removes an inappropriate area as an image of the window as noise from the window image area extracted by the extraction unit. .   The noise removing unit compares the window image area obtained by the extracting unit with a reference size of the window image area, recognizes an area protruding from the reference size as noise, and extracts the area from the window image area. The window image area detecting apparatus according to claim 3, wherein the window image area detecting apparatus is removed.   The noise removing means recognizes a moving area in a direction different from the accumulation direction as noise when obtaining the moving area and integrating the moving area, and removes the area from the window image area. The window image area detection device according to claim 3 or 4.   The window image area detection device according to claim 1, wherein a heat shielding material is used as a material of the window. In the power window device having a pinching prevention function for preventing foreign objects from being pinched by the window during the lifting operation,
Photographing means for photographing the periphery of the window frame;
An extraction means for extracting a window image area as an area of a window part in a photographed image from a change in luminance of the image data using image data photographed by the photographing means;
Setting means for setting a detection area used in foreign object determination from the window image region extracted by the extraction means;
Image processing means for detecting the presence or absence of pinching by monitoring the intrusion of foreign matter into the detection area using a feature amount obtained from the brightness reflected in the image data;
A power window device comprising: an execution unit that causes a lifting control unit that controls the lifting and lowering of the window to perform a safe operation that can prevent the pinching when the image processing unit detects the pinching. .

Images