JP2017175206A - Imaging apparatus, on-vehicle camera, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of correcting influence of a temperature change without using a temperature sensor, and to provide an on-vehicle camera and an image processing method.SOLUTION: An imaging apparatus includes an optical system 20, an imaging element 31 for capturing a subject imaged through the optical system 20, a processor 40 for extracting a specific value of a predetermined portion from the captured image including the subject, and a memory 30 for storing a reference value to be a reference of the specific value. The processor 40 performs correction or area setting of the image on the basis of a change of the specific value from the reference value. Further, the processor 40 may extract a position of a feature point from the image as a specific value and perform correction or area setting of the image on the basis of a change in the position of the feature point.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging device, an in-vehicle camera, and an image processing method.

  Conventionally, an imaging apparatus having an electronic circuit unit including an imaging element is known. In recent years, a small-sized image pickup apparatus packaged including an optical system (including an optical element such as a lens) for forming a subject image acquired by the image pickup element is often used. Imaging devices are widely used as, for example, in-vehicle cameras that support the driver's visual recognition in vehicles, surveillance cameras for security purposes, and the like.

  In general, in an imaging apparatus, a change in the refractive index of a lens or thermal expansion occurs due to a change in temperature, which affects the performance of the optical system. For an imaging device used in a vehicle or the like, a change in temperature in the usage environment may affect performance.

  For example, Patent Document 1 discloses an image pickup apparatus capable of correcting a change in distortion due to a temperature change of an optical system on image data based on temperature data measured by a temperature sensor.

JP 2009-225119 A

  However, including the temperature sensor in the imaging device leads to an increase in size and cost of the imaging device. If the temperature sensor is provided outside the imaging device, the measured temperature may not necessarily reflect the temperature of the lens of the imaging device.

  An object of the present invention made in view of such circumstances is to provide an imaging device, an in-vehicle camera, and an image processing method capable of correcting the influence of a temperature change without using a temperature sensor.

  In order to solve the above-described problem, an imaging apparatus according to the present invention includes an optical system, an imaging element that images a subject imaged via the optical system, and a predetermined image from an image that includes the subject. A processor for extracting a specific value of a part; and a memory for storing a reference value serving as a reference for the specific value, wherein the processor corrects the image based on a change in the specific value from the reference value. Set the area.

  In order to solve the above-described problem, an in-vehicle camera according to the present invention includes an optical system, an imaging element that images a part of a vehicle imaged through the optical system, and a part of the vehicle. A processor for extracting a specific value of a predetermined portion from the image captured in step (b), and a memory for storing a reference value serving as a reference for the specific value, wherein the processor changes the specific value from the reference value. The image is corrected or the area is set based on the above.

  In order to solve the above-described problem, an image processing method according to the present invention is an image processing method executed by an imaging apparatus including an optical system, an imaging element, a processor, and a memory, and the imaging element Imaging a subject imaged via the optical system, the processor extracting a specific value of a predetermined portion from an image captured including the subject, and the memory Storing a reference value serving as a reference for the specific value, and performing correction or region setting on the image based on a change in the specific value from the reference value.

  According to the imaging device, the in-vehicle camera, and the image processing method according to the embodiment of the present invention, it is possible to correct the influence of the temperature change without using the temperature sensor.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to a first embodiment of the present invention. It is the schematic which shows arrangement | positioning of the component of a vehicle-mounted camera system provided with the imaging device which concerns on 1st Embodiment of this invention. FIG. 3A is a diagram illustrating the position of the feature point in a state serving as a reference of the optical system. FIG. 3B is a diagram illustrating a change in the position of the feature point when the refractive index change or thermal expansion of the optical system occurs due to the temperature change. FIGS. 4A to 4C are diagrams illustrating the relationship among the optical system, the image sensor, and the image clipping region in the first embodiment. It is a flowchart which shows the process of the imaging device which concerns on 1st Embodiment of this invention. FIGS. 6A to 6C are diagrams illustrating the relationship among the optical system, the image sensor, and the image cut-out region in the second embodiment. 7A and 7B are diagrams illustrating examples of marker images. It is a flowchart which shows the process of the imaging device which concerns on 2nd Embodiment of this invention. FIG. 9A illustrates a modified example of the marker on the lens. FIG. 9B is a diagram illustrating a modification using a ring-shaped light shielding plate. FIG. 9C is a diagram illustrating an example in which feature points are provided away from the optical axis center.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
(Schematic configuration of imaging device)
As shown in FIG. 1, the imaging apparatus 10 according to the present embodiment includes an optical system 20, an imaging element 31, a memory 30, and a processor 40. The imaging device 10 according to the present embodiment is an in-vehicle camera for capturing a peripheral image behind the vehicle 1. The vehicle 1 is, for example, an automobile.

  Here, in the example of FIG. 2, the in-vehicle camera system includes the imaging device 10 and the display device 50. The imaging device 10 includes a lens with a wide angle of view, such as a fisheye lens, and can capture a peripheral region of the vehicle 1 at a wide angle. In the present embodiment, the imaging device 10 is fixed to the upper portion of the bumper 60 outside the vehicle 1.

  Further, as shown in FIG. 2, a display device 50 that displays an image from the imaging device 10 is mounted on the vehicle 1. The display device 50 is provided so as to be visible from the driver's seat. The display device 50 includes, for example, an LCD and can display a real-time moving image. The display device 50 may display a surrounding image of the vehicle 1 using, for example, an in-vehicle network (for example, CAN: Controller Area Network).

  The configuration of the imaging device 10 will be described with reference to FIG. 1 again. The optical system 20 forms an image of the subject on the image sensor 31. The optical system 20 includes at least a lens and a diaphragm, and forms a subject image. Since the optical system 20 includes a lens with a wide angle of view, a part of the vehicle 1, particularly the bumper 60, forms an image on the image sensor 31 as a subject. As will be described later, in the imaging apparatus 10 according to the present embodiment, a feature point (a point serving as an index in the image) is provided on the bumper 60 in the image and used when correcting the influence of the temperature change of the optical system 20.

  The image pickup device 31 is disposed behind the optical system 20, picks up a subject image formed on the light receiving surface via the optical system 20, converts it into an electrical signal, and outputs it. As the image sensor 31, for example, a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS) image sensor, or the like can be used. The imaging device 10 outputs an image signal based on an electrical signal from the imaging element 31 to the outside of the imaging device 10 by a processor 40 described later.

  The memory 30 stores various data used in the calculations and processes executed by the processor 40, and the results of the calculations and processes. In addition, the memory 30 stores a program for the processor 40 to function as the image acquisition unit 21, the specific value extraction unit 22, the calculation unit 23, the image processing unit 26, and the image output unit 27. Further, the memory 30 may be used when outputting image data of an area cut out by the processor 40 to a device (for example, the display device 50) other than the imaging device 10.

  The processor 40 includes an image acquisition unit 21, a specific value extraction unit 22, a calculation unit 23, an image processing unit 26, and an image output unit 27. The image acquisition unit 21, the specific value extraction unit 22, the calculation unit 23, the image processing unit 26, and the image output unit 27 are functional blocks of processing executed by the processor 40. In the present embodiment, the processor 40 is a CPU that realizes a specific function by reading a specific program. The processor 40 can be configured by a single CPU or a plurality of CPUs.

  The image acquisition unit 21 can perform necessary processing (for example, noise removal, luminance signal processing, and color signal processing) by the image processing unit 26, and the image output unit 27 can output image data of the extracted region. For example, an image is acquired at a cycle of 30 fps.

  The specific value extraction unit 22 extracts a specific value from an image of one frame based on the image signal acquired by the image acquisition unit 21 at a predetermined timing. The predetermined timing is, for example, when the key switch of the vehicle 1 is turned on. The specific value extraction unit 22 may extract the specific value from the image of one frame periodically (for example, every 20 minutes) after the key switch is turned on. The specific value is a value associated with a predetermined portion that is always included in the image. The specific value is, for example, the position, length, width, amount of curvature (curvature), or the like of a predetermined portion. The processor 40 can grasp at least one of a displacement and a shape change of a predetermined portion of the image based on the change of the specific value. In the present embodiment, the specific value is the position of a feature point (for example, xy coordinates in a two-dimensional image), and changes according to the refractive index change or thermal expansion of the optical system 20. In the present embodiment, the specific value extraction unit 22 extracts the position of one feature point. The number of feature points is not limited to one and may be plural.

  The calculation unit 23 calculates a change in the position of the feature point from the reference position in order to grasp the refractive index change and thermal expansion of the optical system 20. In the present embodiment, the reference position is the position of the feature point in a state where the refractive index change or thermal expansion of the optical system 20 has not occurred. In the present embodiment, the calculation unit 23 calculates the distance from the position where the optical axis center of the optical system 20 in the image is formed to the position of the feature point, and obtains the ratio between the distance and the reference value. The position of the optical axis center of the optical system 20 does not change even if a change in the performance of the lenses constituting the optical system 20 (for example, refractive index change, thermal expansion, etc.) occurs due to temperature. The change in the position of the feature point can be obtained by calculating the distance from the. In this embodiment, the reference value is the distance between the position of the optical axis center of the optical system 20 and the feature point at a predetermined reference temperature. For example, the reference value is acquired (or updated) when the imaging device 10 is shipped, at the time of setting when the imaging device 10 is first mounted on the vehicle 1, and when the calibration of the imaging device 10 is subsequently executed. Stored in the memory 30.

  The image processing unit 26 performs image correction or region setting based on the change in the position of the feature point. In the present embodiment, the area setting is an area setting for image clipping. The image processing unit 26 cuts out an image using the above ratio obtained by the calculation unit 23. In general, when the temperature rises and the performance of the lens constituting the optical system 20 changes, the imaging position fluctuates so that the subject shrinks around the optical axis. For this reason, if the image processing unit 26 cuts out the image in the same manner before and after the performance change occurs in the lens constituting the optical system 20, the imaging range changes. For example, when a performance change occurs in the lens constituting the optical system 20 of the rear camera, if the image processing unit 26 does not change the cutout region, it is combined with the image of the side camera and displayed on the display device 50. An unnatural boundary is visually recognized by the user. Therefore, the image processing unit 26 adjusts the cutout region so that an image in the same imaging range is output to the display device 50. The image processing unit 26 further performs noise removal, luminance signal processing (for example, brightness correction, gamma correction, etc.), color signal processing (for example, color interpolation, color correction, white balance, etc.), and other corrections on the image signal (for example). (Distortion correction, curvature correction) may be executed.

  The image output unit 27 outputs the image corrected or cut out by the image processing unit 26 to the display device 50 or the like. The image output unit 27 may output an image signal periodically (for example, at 30 fps).

(Extraction of feature points)
FIG. 3A is a diagram exemplifying the positions of feature points in a reference state of the optical system 20 (that is, a state in which no refractive index change or thermal expansion is caused by a temperature change). The image shown in FIG. 3A is an image based on an image signal from the image sensor 31 at the time of setting when the imaging device 10 is first mounted on the vehicle 1, for example. At this time, it is assumed that the refractive index of the lens provided in the optical system 20 is na. The image shown in FIG. 3A includes, as subjects, a moving region (road surface) of the vehicle 1 divided by a solid line (right side of the paper) and a broken line (left side of the paper), and a bumper 60a. Among these, the bumper 60a is always included in an image captured by the imaging device 10. Therefore, feature points can be set on the bumper 60a. In the present embodiment, the intersection of the boundary between the bumper 60a and the moving region and the left and right intermediate lines of the image shown in FIG. The position of the feature point 62 in FIG. 3A is a reference position and is stored in the memory 30. In the present embodiment, the distance DA between the position of the optical axis center of the optical system 20 in the image (hereinafter referred to as the optical axis center 61) and the position of the feature point 62 is stored in the memory 30 as a reference value. In the present embodiment, the optical axis center 61 also exists on the intermediate line. In this embodiment, a physical marker is not attached to the bumper 60a, but a feature point 62 is set on the image.

  FIG. 3B is a diagram illustrating a change in the position of the feature point when the refractive index change or thermal expansion of the optical system 20 occurs due to a temperature change. The image shown in FIG. 3B is an image based on an image signal from the image sensor 31 when the use environment temperature of the imaging device 10 rises due to the influence of sunlight while the vehicle 1 is traveling, for example. At this time, it is assumed that the refractive index of the lens provided in the optical system 20 changes to nb instead of na. Due to this change in the refractive index, the position of the bumper 60b changes as shown in the image of FIG. A bumper 60a indicated by a broken line in FIG. 3B is the same as the bumper 60a before the change shown in FIG. That is, the entire image is contracted toward the optical axis center 61. Therefore, the feature point 62 is also moved in the direction of the optical axis center 61 from the reference position in FIG. The computing unit 23 computes a distance DB between the optical axis center 61 and the position of the feature point 62, receives the distance DA from the memory 30, and a ratio between the distance DB and the distance DA (for example, a value calculated by DB / DA). )

  Here, as described above, the image processing unit 26 adjusts the cutout region by linear interpolation. By this adjustment, the influence of the lens performance change due to the temperature change (for example, the occurrence of an unnatural boundary when combined with the image of the side camera is displayed) is prevented from being visually recognized by the user. Here, the lens constituting the optical system 20 may be affected by distortion due to a temperature change. For example, in FIG. 3B, the bending amount (curvature) of the curved portion near the end of the bumper 60a may be different from the bending amount of the bumper 60b. The image processing unit 26 may correct the shape of the subject itself distorted by distortion. For example, the image processing unit 26 may read design data (characteristic table) of the lens 28 from the memory 30 and perform shape correction (distortion and curvature correction) of the subject. The design data may indicate how far the image is displayed from the optical axis, for example, according to the angle formed by the optical axis and the incident light. For example, the image processing unit 26 may adjust the cutout region after correcting the shape of the subject displayed at a position away from the optical axis center by a predetermined distance. Further, the image processing unit 26 may estimate the temperature of the optical system 20 by associating the design data with the change of the feature point. Then, the image processing unit 26 may perform image temperature correction by a known method based on the estimated temperature.

  FIG. 4A is a front view of the lens 28 provided in the optical system 20. The optical axis center 61 of the lens 28 in FIG. 4 (A) corresponds to the optical axis center 61 shown in FIGS. 3 (A) and 3 (B). FIG. 4B is a side view of the lens 28 and shows a positional relationship with the image sensor 31. The subject is present on the left side of the surface of the lens 28, and the subject forms an image on the image sensor 31. Here, the optical path La in FIG. 4B schematically shows a light path when the refractive index is na (when the optical system 20 is not affected by the temperature change). Also, the optical path Lb in FIG. 4B schematically shows a light path when the refractive index is nb (when the optical system 20 is affected by a temperature change). As shown in FIG. 4B, when the refractive index of the lens 28 changes from na to nb, the image formed on the image sensor 31 is distorted so as to shrink around the optical axis center 61. A large thick frame in FIG. 4C schematically shows an image of one frame based on an image signal from the image sensor 31. However, although an actual image formed on the image sensor 31 is an inverted image, in FIG. 4C, it is displayed as an erect image for ease of explanation. The feature point 62a in FIG. 4C is the position of the feature point when the refractive index is na, and the feature point 62b is the position of the feature point when the refractive index changes to nb. The image processing unit 26 uses the ratio to the reference value obtained by the calculation unit 23 to adjust the cutout region so as to cancel the influence of the temperature change of the optical system 20. By this adjustment, the imaging apparatus 10 can output an image in the same imaging range including the subject. A region Aa shown in FIG. 4C is a cut-out region when the refractive index is na. A region Ab shown in FIG. 4C is a cut-out region when the refractive index is nb. In the present embodiment, the boundary of the region Ab is the ratio calculated by the calculation unit 23 to the distance to the boundary of the region Aa with the optical axis center 61 as the origin (for example, a value calculated by DB / DA in the example of FIG. 3). Obtained by performing linear interpolation by multiplying by. In the example of FIG. 4C, when the distance from the optical axis center 61 to the point Ba on the boundary of the region Aa is multiplied by the ratio calculated by the calculation unit 23, the point Bb on the boundary of the region Ab is obtained.

(Image processing method)
Hereinafter, an image processing method executed by the processor 40 of the imaging apparatus 10 according to the present embodiment will be described with reference to FIG. Here, it is assumed that the reference positions and reference values of the feature points are stored in the memory 30 before the following steps S1 to S13 are executed.

  The image acquisition unit 21 acquires an image signal for one frame stored in the memory 30 (step S1).

  The specific value extraction unit 22 extracts feature points from one frame image based on the image signal acquired by the image acquisition unit 21 (step S2).

  The calculator 23 calculates the distance from the position of the optical axis center of the optical system 20 to the position of the feature point in the image (step S10). Here, the position of the center of the optical axis and the position of the feature point of the optical system 20 may be determined by coordinates on the xy plane, where the horizontal direction of the image is the x axis and the vertical direction is the y axis. At this time, the calculation unit 23 can efficiently calculate the distance based on the coordinates.

  The calculation unit 23 calculates the distance calculated in step S10 and the reference value acquired from the memory 30 (in this embodiment, the distance between the position of the optical axis center and the feature point in a state where the optical system 20 is not affected by the temperature change). Is obtained (step S11).

  The image processing unit 26 determines a cutout region according to the ratio obtained in step S11 (step S12). As described above, the image processing unit 26 performs linear interpolation with the optical axis center 61 as the origin and the distance to the boundary of the cutout region in the state where the optical system 20 is not affected by the temperature change by the above ratio. A new cutout area is determined.

  The image output unit 27 outputs the image of the cutout area determined by the image processing unit 26 to the display device 50 or the like (step S13).

  As described above, the imaging apparatus 10 of the present embodiment can correct the influence of refractive index change and thermal expansion that occur in the optical system 20 due to temperature change. That is, by using the ratio calculated by the calculation unit 23, the image processing unit 26 adjusts the cutout area so as to cancel the influence of the temperature change of the optical system 20, so that the imaging apparatus 10 of the present embodiment has the same imaging range. An image can be output. The imaging apparatus 10 of the present embodiment can perform correction without using a temperature sensor. Therefore, it is possible to avoid an increase in size and cost of the imaging device 10 due to the provision of the temperature sensor.

[Second Embodiment]
(Feature point)
The imaging device 10 according to the second embodiment will be described below. The configuration of the imaging device 10 according to the present embodiment and the arrangement of components of the vehicle-mounted camera system including the imaging device 10 according to the present embodiment on the vehicle 1 are the same as those in the first embodiment. In order to avoid redundant description, only the feature points and image processing methods that differ from the first embodiment will be described below.

  In the imaging apparatus 10 according to the present embodiment, a feature point is a point where the marker attached to the optical system 20 is imaged. Since the marker is attached to the optical system 20, it is always included in the image based on the image signal from the image sensor 31. Therefore, the point where the marker is imaged can be used as the feature point. In the imaging device 10 according to the present embodiment, a plurality of markers arranged symmetrically with respect to the optical axis center are attached to the optical system 20. Therefore, unlike the first embodiment, the position can be easily specified even if the position of the center of the optical axis is unknown.

  FIG. 6A is a front view of the lens 28 provided in the optical system 20. Unlike the first embodiment, the lens 28 is provided with a marker 63 and a marker 64. The marker 63 and the marker 64 may be printed on the surface, or may be provided by surface processing (for example, shaving). The marker 63 is provided at a position symmetrical to the marker 64 with the optical axis center 61 interposed therebetween.

  FIG. 6B is a side view of the lens 28 and shows a positional relationship with the image sensor 31. When the subject forms an image on the image sensor 31, the marker 63 and the marker 64 are also included in the image. A large thick frame in FIG. 6C schematically shows an image of one frame based on an image signal from the image sensor 31. However, the actual image formed on the image sensor 31 is an inverted image, but in FIG. 6C, it is displayed as an erect image for ease of explanation. Feature points 63a and 64a in FIG. 6C are the positions of the feature points when the refractive index is na. Also, the feature points 63b and 64b in FIG. 6C are the positions of the feature points when the refractive index changes to nb.

  Here, even when the optical axis center 61 is unknown, the calculation unit 23 sets an intermediate point of a straight line connecting the feature point 63b and the feature point 64b (or connecting the feature point 63a and the feature point 64a) to the optical axis center. It can be estimated to be 61.

  Here, the marker 63 and the marker 64 are attached to the periphery of the lens 28 so as not to interfere with the imaging of the subject. As shown in FIG. 7A, the feature point of the present embodiment is a blurred image spreading around. At this time, the calculation unit 23 obtains the luminance for each pixel in the range where the feature points are widened. The coordinates indicating the lowest value of the luminance may be used as the feature point position. In the example of FIG. 7B, the coordinate of the feature point in the x-axis direction is Px having the lowest luminance. The calculation unit 23 can determine the xy coordinates of the feature points corresponding to the marker 63 and the marker 64 by executing the same processing in the y-axis direction.

(Image processing method)
Hereinafter, an image processing method executed by the imaging apparatus 10 of the present embodiment will be described with reference to FIG. Here, steps other than step S3 in FIG. 8, that is, steps S1, S2, S10, S11, S12, and S13 are the same as those in the first embodiment, and thus the description thereof is omitted.

  Step S3 is executed after step S2. The computing unit 23 obtains the position of the optical axis center of the optical system 20, that is, the optical axis center of the lens 28, from the feature points 63b and 64b (or feature points 63a and 64a) (step S3). More specifically, the calculation unit 23 estimates that the midpoint of the straight line connecting the feature points 63 b and 64 b is the optical axis center 61. After step S3, the processes of steps S10 to S13 are executed as in the first embodiment. Here, also in the present embodiment, the image processing unit 26 may correct the shape of the subject itself distorted by distortion.

  As described above, in addition to the effects described in the first embodiment, the imaging apparatus 10 according to the present embodiment can easily estimate the position even when the position where the optical axis center is imaged is unknown. There is an effect. When the position where the optical axis center is imaged is unknown, for example, when the position where the optical axis center is imaged is not stored in the memory 30 in advance, the optical axis center is imaged by repairing the imaging device 10 or the like. For example, when the position changes. Further, when the feature point is provided on the subject (for example, bumper) of the imaging device 10, it is necessary to newly store the reference position and the reference value in the memory 30 when the shape of the subject changes (for example, the bumper is broken). . However, in this embodiment, it is not necessary to update the reference position and the reference value due to deformation of the subject.

  Although the present invention has been described based on the drawings and embodiments, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, the functions included in each means, each step, etc. can be rearranged so as not to be logically contradictory, and a plurality of means, steps, etc. can be combined into one or divided. .

  In the imaging device 10 according to the second embodiment, two markers arranged symmetrically with respect to the optical axis center are attached to the optical system 20. It is also possible to make the reference position and reference value different from those of the second embodiment as follows. For example, the computing unit 23 may use the positions of the feature points 63a and 64a as a reference position and the distance between the feature points 63a and 64a as a reference value. At this time, the calculation unit 23 calculates “distance between two feature points” instead of “distance between the feature point and the center of the optical axis of the lens” in step S10 of FIG. Since the calculation unit 23 compares the distance between the feature point 63b and the feature point 64b with the reference value, the calculation unit 23 performs comparison at a longer distance than in the case of the first embodiment. Therefore, it is expected that the accuracy of the required ratio is further increased.

  In the imaging device 10 according to the second embodiment, the number of markers attached to the optical system 20 is two, but the number may be further increased. For example, four markers may be attached to the optical system 20 as shown in FIG. At this time, not only the optical axis center 61 is estimated using the midpoint of the straight line connecting the feature points corresponding to the markers 63 and 64, but also the optical axis center 61 is estimated by the same method using the markers 65 and 66. be able to. For example, it is possible to estimate the position of the optical axis center 61 more accurately by obtaining the average value of the coordinates of the estimated optical axis centers 61.

  In the imaging device 10 according to the second embodiment, when the calculation unit 23 knows the position of the optical axis center 61, the number of markers attached to the optical system 20 may be one, Only one or more may be used. At this time, it is possible to use the image processing method of the first embodiment (see FIG. 5).

  Further, in the imaging apparatus 10 according to the second embodiment, as shown in FIG. 9B, a ring-shaped light shielding plate 70 may be used instead of attaching a marker to the optical system 20. In the optical system 20, the light shielding plate 70 is provided so as to be positioned between the lens 28 and the imaging device 31. The light shielding plate 70 generates an image including a circular imaging region including a subject and a surrounding ring-shaped dark region. At this time, one or more feature points can be set on the boundary between the imaging region and the ring-shaped dark region. At this time, if the distance between the two feature points is equal to the diameter of the imaging region, two markers arranged symmetrically across the center of the optical axis are attached to the optical system 20 (that is, the second embodiment). ) Can be used.

  Further, when the influence of distortion occurs on the image due to the temperature change, the image processing unit 26 may correct the shape of the subject as described above. When affected by the distortion, the image is distorted concentrically as the distance from the optical axis center increases. Therefore, by providing the feature point as far as possible from the center of the optical axis, the influence of distortion aberration can be grasped more accurately. As a specific example, it can be considered that the end of the bumper of the vehicle 1 in the image is a feature point. FIG. 9C shows a state in which the feature points 67a and 68a at both ends of the bumper 60a change to feature points 67b and 68b at both ends of the bumper 60b. Here, the bumper 60a is the position of the bumper when the refractive index is na. The bumper 60b is the position of the bumper when the refractive index is nb (when the distortion is changed in the optical system 20 due to a temperature change). Compared with the example of FIG. 3B in which the central portion of the bumper is a feature point, the feature point is greatly displaced both in the horizontal direction (x-axis direction) and in the vertical direction (y-axis direction). In addition, the calculation unit 23 may execute a process of comparing the distance D between the feature points 67b and 68b with a reference value (for example, the distance between the feature points 67a and 68a). Sensitivity of detection can be increased by providing a feature point at a location away from the center of the optical axis and detecting a change in the distance between them.

  In the first embodiment, the image processing unit 26 adjusts the cutout region by linear interpolation. However, the image processing unit 26 may shift (translate) the cutout region so as to correspond to the change in the position of the feature point.

  In the first embodiment described above, a part on the bumper 60 (see FIG. 2), which is the subject, is a feature point. However, the feature point is not limited to the bumper 60 and may be a part of the vehicle body. For example, when the imaging device 10 is provided on a side mirror of the vehicle 1, a part of a tire or a part of a door may be used as a feature point.

  In the above-described embodiment, the imaging device 10 includes the memory 30, but a storage device such as a hard disk drive may be used. The memory 30 includes a volatile memory and a nonvolatile memory, but may include only one of them (for example, a nonvolatile memory).

DESCRIPTION OF SYMBOLS 1 Vehicle 10 Imaging device 20 Optical system 21 Image acquisition part 22 Specific value extraction part 23 Calculation part 26 Image processing part 27 Image output part 28 Lens 30 Memory 31 Imaging element 40 Processor 50 Display apparatus 60, 60a, 60b Bumper 61 Optical axis center 62, 62a, 62b, 63a, 63b, 64a, 64b, 67a, 67b Feature point 63, 64, 65, 66 Marker 70 Light shielding plate

Claims (10)

Optical system,
An image sensor for imaging a subject imaged via the optical system;
A processor that extracts a specific value of a predetermined portion from an image captured including the subject;
A memory for storing a reference value serving as a reference for the specific value,
The imaging apparatus, wherein the processor corrects the image or sets an area based on a change in the specific value from the reference value.   The imaging apparatus according to claim 1, wherein the processor extracts a position of a feature point from the image as the specific value, and corrects or sets an area of the image based on a change in the position of the feature point.   The imaging apparatus according to claim 2, wherein the processor corrects the image or sets a region based on a change in a distance from a position where an optical axis center of the optical system in the image forms an image to a position of the feature point. .   The imaging device according to claim 3, wherein the processor determines a position at which the optical axis center forms an image by using a plurality of the feature points.   The imaging device according to claim 2, wherein the feature point is provided in a part on the subject in the image.   The imaging device according to claim 2, wherein the feature point is a point at which one or a plurality of markers attached to the optical system is imaged.   The imaging apparatus according to claim 1, wherein the processor extracts a bending amount of the predetermined portion as the specific value, and corrects or sets an area of the image based on a change in the bending amount.   The imaging device according to claim 7, wherein the processor corrects distortion and curvature of the image based on design data of the optical system stored in the memory. Optical system,
An image sensor for imaging a part of a vehicle imaged via the optical system;
A processor for extracting a specific value of a predetermined part from an image captured including a part of the vehicle;
A memory for storing a reference value serving as a reference for the specific value,
The processor is an in-vehicle camera that performs correction or region setting of the image based on a change in the specific value from the reference value. An image processing method executed by an imaging apparatus including an optical system, an imaging element, a processor, and a memory,
The imaging element imaging a subject imaged via the optical system;
The processor extracting a specific value of a predetermined portion from an image captured including the subject; and
The memory storing a reference value serving as a reference for the specific value;
And a step of correcting the image or setting an area based on a change in the specific value from the reference value.

Images