JP6502003B1 - Image correction apparatus and image correction method - Google Patents

Abstract

  The image correction device (1) is an image obtained by the first imaging device (21) for imaging a moving imaging target, and is an image formed at the final time of the time when the image is formed. The final time image is divided into an image and a non-final time image which is an image formed before the above-mentioned final time, and each of the final time image and the non-final time image is based on the position of the imaging target And a division unit (10) for dividing into a plurality of regions for each exposure time required when each portion of the non-final time image is formed, and a correction unit (12) for correcting the luminance value of each region. The correction unit (12) sets the luminance value of the region to be corrected to the first exposure time required when the region to be corrected is formed for a plurality of regions included in the final time image. With respect to a plurality of areas included in the non-final time image, correction is performed using the correction value, and the luminance value of the area to be corrected is a second correction value corresponding to the first correction value and the non-final time image Use and to correct.

Description

  The present invention relates to an image correction apparatus and an image correction method for correcting an image obtained by an imaging apparatus for imaging a moving imaging target.

  BACKGROUND Conventionally, there is known a technique for correcting blurring that occurs in an image captured by a camera for a moving imaging target. When the imaging target is moved, a partial image corresponding to at least a part of the imaging target is included in each of a plurality of portions in one frame, so that blurring is an integration of luminance values of a part of the imaging target It is caused by being done. For example, Patent Document 1 discloses a technique for correcting blurring that occurs in an image captured by an infrared camera with a ladle in a steel mill.

JP, 2017-187434, A

  The imaging target in the logistics industry moves faster than the ladle, which is the imaging target in a steel mill, and the size of the imaging target in the logistics industry is significantly smaller than that of the ladle. Therefore, even if the technique disclosed in Patent Document 1 is used, it is not possible to properly correct blurring that occurs in an image captured by an infrared camera as a moving imaging target in the physical distribution industry. It is required to provide a technique for appropriately correcting blurring that occurs in an image captured by an infrared camera as a moving imaging target.

  The present invention has been made in view of the above, and it is an object of the present invention to provide an image correction apparatus that appropriately corrects blurring that occurs in a captured image of a moving imaging target and that is generated by integration of luminance values. I assume.

  In order to solve the problems described above and to achieve the object, the present invention provides an image obtained at a time when one frame is formed by a first imaging device for imaging a moving imaging object, The image is divided into a final time image which is an image formed at the time of 1 and a non final time image which is an image formed before the final time of the above time, and each of the final time image and the non final time image is divided A division unit for dividing into a plurality of areas for each exposure time required when each portion of the final time image and the non-final time image is formed based on the position of the imaging target, and the final obtained by the division unit And a correction unit configured to correct the luminance value of each of a plurality of areas included in the time image and the non-final time image. The correction unit sets, for a plurality of areas included in the final time image, the luminance value of the area to be corrected, and the first correction value corresponding to the exposure time required when the area to be corrected is formed. For a plurality of areas included in the non-final time image, the correction is performed using the luminance value of the area to be corrected, and the first correction corresponding to the exposure time required when the area to be corrected is formed Correction is performed using the value and a second correction value corresponding to the non-final time image.

  The image correction apparatus according to the present invention has an effect that it is possible to appropriately correct a shake that occurs in a captured image of a moving imaging target and that is generated by integration of luminance values.

FIG. 1 shows a configuration of an imaging system according to a first embodiment. A diagram showing an example of an image obtained by imaging an imaging target on which each of a first imaging device and a second imaging device included in the imaging system according to the first embodiment moves FIG. 1 shows a configuration of an image correction apparatus according to a first embodiment. Diagram for explaining the function of the division unit of the image correction apparatus according to the first embodiment The figure for demonstrating the 1st correction value memorized by the correction value storage part which the image correction device concerning a 1st embodiment has. A flowchart showing the procedure of the operation of the image correction apparatus according to the first embodiment FIG. 5 schematically shows the distribution of luminance values obtained by performing correction by the correction unit of the image correction apparatus according to the first embodiment. FIG. 6 is a diagram for explaining correction performed by the image correction apparatus according to the first embodiment. Diagram for explaining the function of the division unit of the image correction apparatus according to the second embodiment The figure for demonstrating the 2nd correction value memorized by the correction value storage part which the image correction device concerning a 2nd embodiment has. FIG. 6 is a diagram showing the configuration of an image correction apparatus according to a third embodiment. Diagram for explaining the function of the interpolation image generation unit of the image correction device according to the third embodiment The figure which shows the condition where the image correction apparatus concerning Embodiment 1 to Embodiment 3 is connected to an encoder and a control apparatus.

  Hereinafter, an image correction apparatus and an image correction method according to an embodiment of the present invention will be described in detail based on the drawings. The present invention is not limited by the embodiment.

Embodiment 1
FIG. 1 is a diagram showing the configuration of an imaging system 50 according to the first embodiment. The imaging system 50 includes an image correction device 1 that corrects an image. Details of the image correction device 1 will be described later. The imaging system 50 has a first imaging device 21 for imaging the moving imaging object 23 at a first frame rate, and a first frame rate higher than the first frame rate at which the first imaging device 21 images the imaging object 23 And a second imaging device 22 for imaging the imaging target 23 at a frame rate of 2. The first imaging device 21 and the second imaging device 22 are arranged side by side. The imaging object 23 is also shown in FIG. The frame rate means the number of frames obtained by imaging per unit time.

  An example of the first imaging device 21 is an infrared camera. An example of the second imaging device 22 is a visible light camera. An example of the first frame rate is 60 fps and an example of the second frame rate is 1200 fps. For convenience of explanation, in the first embodiment, it is assumed that the first frame rate is 60 fps and the second frame rate is 180 fps. In the first embodiment, the angle of view of the first imaging device 21 and the angle of view of the second imaging device 22 are the same.

  The imaging target 23 is placed on the belt conveyor 24. A belt conveyor 24 is also shown in FIG. The imaging target 23 moves in the direction of the arrow 25 by the operation of the belt that constitutes the belt conveyor 24. An example of the imaging target 23 is a corrugated cardboard made by a hot melt process. In the first embodiment, it is assumed that the imaging target 23 performs constant velocity linear motion.

  FIG. 2 is a view showing an example of an image obtained by imaging the imaging target 23 on which each of the first imaging device 21 and the second imaging device 22 included in the imaging system 50 according to the first embodiment moves. It is. Specifically, FIG. 2A shows an example of an image obtained by imaging the imaging target 23 on which the first imaging device 21 moves. FIG. 2B illustrates an example of an image obtained by imaging the imaging target 23 on which the second imaging device 22 moves. In each of the drawings after FIG. 2, except for FIG. 13, for convenience of explanation, the imaging target 23 is shown by a triangle. In the following, it is assumed that the imaging target 23 is a triangle.

  As shown in FIG. 2 (B), the second imaging device 22 takes the imaging object 23 at each time after 1/180 seconds, 2/180 seconds and 3/180 seconds from the first time of imaging. Get an image. On the other hand, as shown in FIG. 2A, the first imaging device 21 obtains an image of the imaging target 23 1/60 seconds after the first time of imaging, that is, 3/180 seconds.

  The time 1/60 seconds after the first time of imaging, that is, the time after 3/180 seconds, is the time when one frame is formed by the first imaging device 21. That is, the time 3/180 seconds after the first time of imaging in FIG. 2 is the last time of the time when one frame is formed by the first imaging device 21. That is, the image 3/180 seconds after the first time of imaging shown in FIG. 2A is an image obtained at the time when one frame is formed by the first imaging device 21. Note that the image of the imaging target 23 described at the first time of imaging in FIGS. 2A and 2B is such that the second imaging device 22 images the imaging target 23 at the first time of imaging It is an image obtained by

  As described above, in the first embodiment, the first frame rate is 60 fps and the second frame rate is 180 fps. That is, the time when the first imaging device 21 obtains one image is three times the time when the second imaging device 22 obtains one image. As described with reference to FIG. 1, the imaging target 23 is moving. Therefore, as shown by the image 1/60 sec after the first time of imaging in FIG. 2A, the first imaging device 21 is the first imaging after 1/60 sec from the first time of imaging. Not only an image of the imaging target 23 at a position 1/60 seconds after the time is obtained, but also a blurred image 23a showing the imaging target 23 at each position up to 1/60 second after the first time of imaging is obtained. Furthermore, the first imaging device 21 obtains an image of the imaging target 23 at 1/60 seconds after the first time of imaging and moves to 1/60 seconds after the first time of imaging. A blurred image 23a resulting from the integration of the luminance values of a part of the object 23 is obtained.

  The use of the second imaging device 22 may be prohibited. As described above, the first imaging device 21 obtains an image of the imaging target 23 and obtains a blurred image 23 a 1/60 seconds after the first time of imaging. The blurred image 23a is an image that can not be obtained by the second imaging device 22 1/60 seconds after the first time of imaging. The image of the imaging target 23 obtained 1/60 seconds after the first imaging device 21 captures the first time of imaging includes a portion in which the luminance values of a part of the moving imaging target 23 are integrated. When the first imaging device 21 is used without using the second imaging device 22, the image correction device 1 obtains an image similar to an image obtained by the second imaging device 22 every 1/60 seconds. The purpose is

  The image correction apparatus 1 obtains an image of the imaging target 23 from an image including the image of the imaging target 23 such as an image 1/60 seconds after the first time of imaging in FIG. 2A and the blurred image 23a. Make corrections for That is, the image correction device 1 corrects an image obtained by the first imaging device 21 that images the moving imaging target 23.

  FIG. 3 is a diagram showing the configuration of the image correction device 1 according to the first embodiment. The image correction device 1 includes a trigger generation unit 2 that generates an imaging trigger for causing each of the first imaging device 21 and the second imaging device 22 to start imaging. The first imaging device 21 and the second imaging device 22 are also shown in FIG. The image correction device 1 has a communication unit 3 that simultaneously transmits the imaging trigger generated by the trigger generation unit 2 to each of the first imaging device 21 and the second imaging device 22.

  The first imaging device 21 and the second imaging device 22 receive an imaging trigger and start imaging of the imaging target 23. The communication unit 3 has a function of receiving from the first imaging device 21 the image data obtained when the first imaging device 21 images the imaging target 23, and the second imaging device 22 images the imaging target 23 And a function of receiving image data obtained at the time of image formation from the second imaging device 22.

  The image correction device 1 further includes an image storage unit 4 that stores the image data received by the communication unit 3 from the first imaging device 21 and the image data received from the second imaging device 22. The image correction apparatus 1 extracts the feature points of the image of the imaging target 23 based on the image data which is the image data stored in the image storage unit 4 and which the communication unit 3 receives from the second imaging device 22. It further has a feature extraction unit 5. An example of the feature point is an edge of the imaging target 23. In the first embodiment, the edge is each of three vertices of the triangular imaging target 23. The shape and size of the triangular imaging target 23 are identified based on the three vertices. The image correction device 1 further includes a feature storage unit 6 that stores data indicating feature points extracted by the feature extraction unit 5.

  The image correction device 1 further includes a movement amount calculation unit 7 that calculates the movement amount per unit time of the feature point indicated by the data stored in the feature storage unit 6. Specifically, in the movement amount calculation unit 7, one frame is formed by the second imaging device 22 based on the corresponding feature points in two adjacent frames captured by the second imaging device 22. The movement amount of the feature point in the time is calculated. The image correction apparatus 1 further includes a movement amount storage unit 8 that stores data indicating the movement amount per unit time of the feature point obtained by the movement amount calculation unit 7 performing calculation.

  The image correction apparatus 1 extracts the feature points of the correction target image which is an image based on the image data which is the image data stored in the image storage unit 4 and which the communication unit 3 receives from the first imaging device 21. The correction image specifying unit 9 further specifies a correction target image. Specifically, the correction image specifying unit 9 sets a trapezoidal image including the image of the imaging target 23 and the blurred image 23a 1/60 seconds after the first time of imaging in FIG. 2A as the correction target image. Identify. The feature storage unit 6 also stores data indicating feature points extracted by the corrected image identification unit 9. Furthermore, the feature storage unit 6 also stores data indicating the correction target image.

  The image correction device 1 forms the correction target image, which is an image obtained at the time when one frame is formed by the first imaging device 21 for imaging the moving imaging object 23, at the final time of the time The image processing apparatus further includes a division unit 10 that divides an image into a final time image and a non-final time image that is an image formed before the final time of the time. The dividing unit 10 sets each of the final time image and the non-final time image on the basis of the position of the imaging target 23, and for each exposure time required when each portion of the final time image and the non-final time image is formed. Divide into multiple areas. The position of the imaging target 23 is specified by the image data received by the communication unit 3 from the second imaging device 22. Furthermore, the position of the imaging target 23 is specified by the data stored in the movement amount storage unit 8.

  Specifically, the dividing unit 10 also uses the shape and size of the imaging target 23 of the photographed image based on the data stored in the feature storage unit 6 and the data stored in the movement amount storage unit 8. The correction target image specified by the correction image specifying unit 9 based on the position of the imaging target 23 determined in step It is divided into a final time image which is an image and a non-final time image which is a blurred image 23a.

  FIG. 4 is a diagram for explaining the function of the dividing unit 10 of the image correction apparatus 1 according to the first embodiment. Further, FIG. 4 is a view showing a correction target image including the image of the imaging target 23 and the blurred image 23a 1/60 seconds after the first time of imaging in FIG. 2 (A). As described above, the imaging target 23 moves in a uniform linear motion. Therefore, after 1/60 seconds from the first time of imaging, a correction target image such as a triangular image having point P0, point Q0 and point R0 at three vertices in FIG. 4 sequentially moved in the direction of arrow 25a. Is obtained by the first imaging device 21.

  At the first time of imaging, a first image pickup device 21 obtains an image of a triangle having the point P0, the point Q0 and the point R0 in FIG. After 1/180 seconds from the first time of imaging, the first imaging device 21 obtains a triangular image having the point P1, the point Q1 and the point R1 in FIG. 2/180 seconds after the first time of imaging, the first imaging device 21 obtains a triangular image having the point P2, the point Q2 and the point R2 in FIG. 4 at three apexes. After 1/60 seconds, which is 3/180 seconds after the first time of imaging, a first image pickup device 21 obtains an image of a triangle having three points, point P3, point Q3 and point R3 in FIG. Be The images obtained at each of the above four times are combined, and point P0, point Q0, point R3 and point P3 are 4 at 1/60 seconds after 3/180 seconds from the first time of imaging. The first image pickup device 21 obtains a trapezoidal correction target image possessed at each vertex.

  In FIG. 4, “y0” indicates the movement amount of the above-mentioned triangular image in each 1/180 second when dividing 1/60 second from first time of imaging to 1/60 second in half. ing. Data indicating the movement amount is stored in the movement amount storage unit 8. Since the imaging target 23 moves in a uniform linear motion, the movement amount y0 of the triangular image every 1/180 seconds is constant.

  As described above, after 1/60 seconds, which is 3/180 seconds after the first time of imaging, a triangular image having point P3, point Q3 and point R3 at three vertices in FIG. 4 is the first imaging. The image is captured by the device 21. The code B is assigned to four regions constituting a triangular image having the point P3, the point Q3 and the point R3 at three vertices. The image constituted by all the regions to which the code B is assigned is the final time image formed at the last time of the time when the correction target image is formed. However, the exposure time taken when each of the four areas to which the code B is assigned is formed is different from the exposure time required when the other areas are formed. In FIG. 4, the image constituted by all the regions to which the code A is assigned is a non-final time image formed before the final time of the exposure time.

  A part of the triangle image of the first time of imaging is a triangle image 1/180 second after the first time of imaging, a triangle image 2/180 seconds after the first time of imaging, and Overlap with the triangle image 3/180 seconds after the first time. A part of the triangle image 1/180 sec after the first time of imaging is a triangle image 2/180 sec after the first time of imaging and 3/180 seconds after the first time of imaging Overlap with the triangle image. A part of the triangle image 2/180 seconds after the first time of imaging overlaps with the triangle image 3/180 seconds after the first time of imaging.

  In FIG. 4, the numbers added on the right side of the code A or the code B are each 1/180 seconds obtained by dividing 1/60 seconds from the first time of imaging to 1/60 seconds into three equal parts. "1" is added to the number of times that a part of the above triangular image overlaps. The number of times of duplication corresponds to the exposure time. Specifically, the exposure time required when the area to which the code A1 is assigned is formed is shorter than the exposure time required when the area to which the code A2 is assigned is formed. The exposure time required when the area to which the code A2 is assigned is formed is shorter than the exposure time required when the area to which the code A3 is assigned is formed.

  The exposure time required when the area to which the code B1 is assigned is formed is shorter than the exposure time required when the area to which the code B2 is assigned is formed. The exposure time required when the area to which the code B2 is assigned is formed is shorter than the exposure time required when the area to which the code B3 is assigned is formed. The exposure time required when the area to which the code B3 is assigned is formed is shorter than the exposure time required when the area to which the code B4 is assigned is formed.

  The dividing unit 10 sets each of the final time image and the non-final time image on the basis of the position of the imaging target 23, and for each exposure time required when each portion of the final time image and the non-final time image is formed. Divide into multiple areas. The position of the imaging target 23 is specified by the image data received by the communication unit 3 from the second imaging device 22. Specifically, the position of the imaging target 23 is specified by the data stored in the movement amount storage unit 8.

  That is, as shown in FIG. 4, the dividing unit 10 determines the final time image as an area to which the code B1 is allocated, an area to which the code B2 is allocated, an area to which the code B3 is allocated, and Divide into the area to which B4 is allocated. In addition, the dividing unit 10 divides the non-final time image into an area to which the code A1 is allocated, an area to which the code A2 is allocated, and an area to which the code A3 is allocated.

  Return to FIG. The image correction apparatus 1 further includes a correction value storage unit 11 that stores a first correction value and a second correction value used when correcting an image. FIG. 5 is a diagram for explaining the first correction value stored in the correction value storage unit 11 of the image correction apparatus 1 according to the first embodiment. The first correction value corresponds to the exposure time required when the area to be corrected is formed. FIG. 5 shows an example in which the first correction value decreases in proportion to the exposure time.

  FIG. 5 shows four first correction values of α1, α2, α3 and α4. α1 is a first correction value used when correcting the luminance value of the area to which the code A1 and the code B1 are assigned. α2 is a first correction value used when correcting the luminance value of the area to which the code A2 and the code B2 are assigned. α3 is a first correction value used when correcting the luminance value of the area to which the code A3 and the code B3 are assigned. α4 is a first correction value used when correcting the luminance value of the area to which the code B4 is assigned.

  Return to FIG. The image correction device 1 further includes a correction unit 12 that corrects the luminance value of each of a plurality of areas included in the final time image and the non-final time image obtained by the dividing unit 10. The correction unit 12 sets, for a plurality of areas included in the final time image, a first correction value corresponding to the exposure time required when the area to be corrected is formed. Use to correct. The first correction value is stored in the correction value storage unit 11.

  When the area to be corrected is the area to which the code B1 is assigned, the correction unit 12 generates the luminance value of the area to which the code B1 is assigned and the area to which the code B1 is assigned is formed. Correction is performed using the first correction value α1 corresponding to the required exposure time. When the area to be corrected is the area to which the code B2 is allocated, the correction unit 12 generates the luminance value of the area to which the code B2 is allocated, and the area to which the code B2 is allocated is formed. Correction is performed using the first correction value α2 corresponding to the required exposure time.

  The correction unit 12 performs, for a plurality of areas included in the non-final time image, the first correction corresponding to the exposure time required when the area to be corrected is formed. Correction is performed using the value and the second correction value corresponding to the non-final time image. The first correction value and the second correction value are stored in the correction value storage unit 11. For example, the second correction value is a value smaller than one.

  When the area to be corrected is the area to which the code A1 is assigned, the correction unit 12 generates the luminance value of the area to which the code A1 is assigned and the area to which the code A1 is assigned is formed. Correction is performed using the first correction value α1 corresponding to the required exposure time and the second correction value corresponding to the non-final time image. When the area to be corrected is the area to which the code A2 is allocated, the correction unit 12 generates the luminance value of the area to which the code A2 is allocated, and the area to which the code A2 is allocated is formed. Correction is performed using the first correction value α2 corresponding to the required exposure time and the second correction value corresponding to the non-final time image.

Furthermore, for example, when the luminance value before correction of the area to which the code Xn is allocated is Xn and the luminance value after correction of the area to which the code Xn is allocated is Xn ′, for example, the correction unit 12 Corrects the luminance value before correction according to the following equation (1). Here, X is A or B, and n is an integer of 1 to 4. In the following equation, β0 is a second correction value.
A1 ′ = α1 × β0 × A1
A2 '= α2 × β0 × A2
A3 '= α3 × β0 × A3
B1 '= α1 × B1
B2 '= α2 × B2
B3 '= α3 × B3
B4 '= α4 × B4 (1)

  The image correction apparatus 1 further includes a corrected image generation unit 13 that generates an image after correction based on the luminance value obtained by the correction unit 12. The image correction apparatus 1 further includes a display unit 14 that displays the image after correction generated by the correction image generation unit 13. An example of the display unit 14 is a liquid crystal display device.

  An example of the image storage unit 4, the feature storage unit 6, the movement amount storage unit 8, and the correction value storage unit 11 included in the image correction device 1 is a semiconductor memory.

  FIG. 6 is a flowchart showing the procedure of the operation of the image correction apparatus 1 according to the first embodiment. The trigger generation unit 2 generates an imaging trigger for causing each of the first imaging device 21 and the second imaging device 22 to start imaging. The communication unit 3 simultaneously transmits the imaging trigger generated by the trigger generation unit 2 to each of the first imaging device 21 and the second imaging device 22 (S1). The communication unit 3 receives, from the first imaging device 21, image data obtained when the first imaging device 21 images the imaging target 23, and when the second imaging device 22 images the imaging target 23. The obtained image data is received from the second imaging device 22 (S2).

  The feature extraction unit 5 extracts feature points of the image of the imaging target 23, which is an image based on the image data received by the communication unit 3 from the second imaging device 22 (S3). The movement amount calculation unit 7 calculates the movement amount per unit time of the feature point extracted by the feature extraction unit 5 (S4). The correction image specifying unit 9 specifies feature points of an image based on the image data received by the communication unit 3 from the first imaging device 21 and specifies a correction target image (S5).

  The dividing unit 10 is an image formed at the final time of the time when the correction target image is formed, which is the correction target image which is an image obtained by the first imaging device 21 that picks up the moving imaging target 23. The image is divided into a final time image and a non-final time image which is an image formed before the final time of the exposure time (S6). The dividing unit 10 sets each of the final time image and the non-final time image on the basis of the position of the imaging target 23, and for each exposure time required when each portion of the final time image and the non-final time image is formed. It is divided into a plurality of areas (S6).

  The correction unit 12 corrects the luminance value of each of the plurality of areas included in the final time image and the non-final time image obtained by the dividing unit 10. That is, the correction unit 12 corrects the correction target image (S7). The corrected image generation unit 13 generates a corrected image, which is an image after correction, based on the luminance value obtained by the correction unit 12 (S8). The display unit 14 displays a corrected image which is an image after correction generated by the corrected image generation unit 13 (S9).

  As described above, the image correction device 1 corrects the correction target image, which is an image obtained at the time when one frame is formed by the first imaging device 21 for imaging the moving imaging object 23, at the last time of the time And a non-final time image which is an image formed before the final time of the time. The image correction device 1 sets each of the final time image and the non-final time image based on the position of the imaging target 23 for each exposure time required when each portion of the final time image and the non-final time image is formed. Divide into multiple areas of

  The image correction apparatus 1 performs the first correction corresponding to the exposure time required when the area to be corrected is formed for the luminance values of the area to be corrected for a plurality of areas included in the final time image. Make corrections using the values. The image correction apparatus 1 applies the first luminance value of the area to be corrected to the exposure time required when the area to be corrected is formed for a plurality of areas included in the non-final time image. Correction is performed using the correction value and the second correction value corresponding to the non-final time image.

  FIG. 7 is a view schematically showing a distribution of luminance values obtained by performing correction by the correction unit 12 of the image correction apparatus 1 according to the first embodiment. FIG. 7 shows that the luminance values of the area to which the code B1, the code B2, the code B3 and the code B4 are assigned are the same. In addition, FIG. 7 shows that the luminance values of the regions to which the symbols A1, A2 and A3 are assigned are smaller than the luminance values of the regions to which the symbols B1, B2, B3 and B4 are assigned, and It shows that it is the same.

  FIG. 8 is a diagram for explaining the correction performed by the image correction device 1 according to the first embodiment. FIG. 8A is the same as FIG. 2A, and shows the correction target image at the final time of imaging. FIG. 8B shows an image obtained by the image correction apparatus 1 correcting the correction target image at the final time of imaging. The image shown at the last time of imaging in FIG. 8B is an image obtained based on the distribution of luminance values in FIG. 7. FIG. 8C is the same as FIG. 2B, and shows the first time of imaging, the time after 1/180 seconds from the beginning of imaging, and the time after 2/180 seconds from the beginning of imaging , The image obtained by the second imaging device 22 at a time 3/180 seconds after the beginning of imaging.

  As shown in FIG. 8B, the image correction apparatus 1 according to the first embodiment can obtain an image of the imaging target 23 not including the blurred image 23a at a time 3/180 seconds after the start of imaging. . That is, the image correction apparatus 1 can appropriately correct the shake that occurs in the image in which the moving imaging target is captured and that is generated by the integration of the luminance values. Furthermore, in the image correction apparatus 1, when the use of the second imaging device 22 is prohibited, the first imaging device 21 is used without the second imaging device 22 being used. An image similar to the image obtained by the second imaging device 22 can be obtained every 1/60 seconds.

  In addition, as can be understood from FIG. 7, the image correction device 1 is an imaging target in which the luminance value after one frame is formed by the first imaging device 21 is corrected in consideration of the exposure time. 23 images can be obtained. That is, in the case where the first imaging device 21 is an infrared camera used to determine whether or not a corrugated cardboard boxed by the hot melt process is appropriately boxed, the image correction apparatus 1 is configured by the hot melt process. An image can be generated to determine whether the boxed cardboard is properly boxed. The image correction device 1 does not perform the correction for each pixel, but performs the correction for each region where a plurality of pixels are integrated, so that the correction can be performed relatively quickly.

  Data indicating the shape and size of the imaging target 23 of the captured image may be stored in the feature storage unit 6 in advance. Furthermore, data indicating the shape and size of the imaging target 23 of the image captured by the second imaging device 22 may be stored in the feature storage unit 6 in advance.

Second Embodiment
Next, an image correction apparatus 1 according to a second embodiment will be described. In the first embodiment, it is assumed that the imaging target 23 performs constant velocity linear motion. In the second embodiment, it is assumed that the imaging target 23 performs an acceleration motion. The configuration of the image correction device 1 according to the second embodiment is the same as the configuration of the image correction device 1 according to the first embodiment. In the second embodiment, parts different from the first embodiment will be described.

  In the second embodiment, division unit 10 divides the non-final time image into a plurality of regions corresponding to the exposure time when each portion of the non-final time image is formed and the acceleration of imaging target 23. FIG. 9 is a diagram for explaining the function of the dividing unit 10 of the image correction apparatus 1 according to the second embodiment. At the first time of imaging, the first imaging device 21 obtains an image of a triangle having the point P0, the point Q0, and the point R0 in FIG. After 1/180 seconds from the first time of imaging, the first imaging device 21 obtains a triangular image having the point P1, the point Q1 and the point R1 in FIG.

  At 2/180 seconds after the first time of imaging, the first imaging device 21 obtains a triangular image having the point P2, the point Q2 and the point R2 in FIG. After 1/60 seconds, which is 3/180 seconds after the first time of imaging, a first image pickup device 21 obtains an image of a triangle having three points, point P3, point Q3 and point R3 in FIG. Be The images obtained at each of the above four times are combined, and point P0, point Q0, point R3 and point P3 are 4 at 1/60 seconds after 3/180 seconds from the first time of imaging. The first image pickup device 21 obtains a trapezoidal correction target image possessed at each vertex.

  In FIG. 9, the moving amount of the above-mentioned triangular image is y1 between 1/180 seconds after the first time of imaging and y2 between 1/180 seconds and 2/180 seconds. And y3 between 2/180 seconds and 3/180 seconds. In the second embodiment, the imaging target 23 performs acceleration motion. Therefore, each of the movement amount y1, the movement amount y2, and the movement amount y3 is different from the other two movement amounts. Each of the movement amount y1, the movement amount y2, and the movement amount y3 corresponds to the acceleration of the imaging target 23.

  In FIG. 9, in the same way as in the first embodiment, a symbol B is assigned to four regions constituting a triangular image having points P 3, Q 3 and R 3 at three vertices. The image constituted by all the regions to which the code B is assigned is the final time image formed at the last time of the time when the correction target image is formed. In FIG. 9, the image constituted by all the areas to which the code A is assigned is a non-final time image formed before the final time of the time when the correction target image is formed.

  As described above, the dividing unit 10 divides the non-final time image into a plurality of regions corresponding to the exposure time when each portion of the non-final time image is formed and the acceleration of the imaging target. That is, since each of the movement amount y1, the movement amount y2 and the movement amount y3 corresponds to the acceleration of the imaging target 23, the division unit 10 does not have any one of the non-final time images from A4 to A9. Divide into 6 areas to which is assigned. FIG. 10 is a diagram for explaining a second correction value stored in the correction value storage unit 11 of the image correction apparatus 1 according to the second embodiment. The second correction value of the second embodiment corresponds to the non-final time image and differs depending on the acceleration of the imaging target 23. FIG. 10 shows an example in which the second correction value decreases as the amount of movement of the triangular image per 1/180 second increases.

  The second correction value when the movement amount of the triangular image per 1/180 sec is y1 is β1. The second correction value is β2 when the movement amount of the triangular image per 1/180 sec is y2. The second correction value is β3 when the movement amount of the triangular image per 1/180 sec is y3.

  For the area to which the code A4 is assigned, the first correction value is α1 and the second correction value is β1. For the area to which the code A5 is assigned, the first correction value is α1, and the second correction value is β2. For the area to which the code A6 is assigned, the first correction value is α1 and the second correction value is β3. For the area to which the code A7 is assigned, the first correction value is α2 and the second correction value is β2. For the area to which the code A8 is assigned, the first correction value is α2, and the second correction value is β3. For the area to which the code A9 is assigned, the first correction value is α3 and the second correction value is β3.

  The correction unit 12 performs, for a plurality of areas included in the non-final time image, the first correction corresponding to the exposure time required when the area to be corrected is formed. The correction is performed using the value and the second correction value corresponding to the acceleration of the imaging target 23 when the area to be corrected is formed. The correction unit 12 corrects the luminance value of the area to be corrected according to the method described in the first embodiment for a plurality of areas included in the final time image.

When the luminance value before correction of the area to which the code Xn is assigned is Xn and the luminance value after correction of the area to which the code Xn is assigned is Xn ′, for example, the correction unit 12 The luminance value before correction is corrected according to equation (2). Here, X is A or B, and n is an integer of 1 to 9.
A4 '= α1 × β1 × A4
A5 '= α1 × β2 × A5
A6 '= α1 × β3 × A6
A7 '= α2 × β2 × A7
A8 '= α2 × β3 × A8
A9 '= α3 × β3 × A9
B1 '= α1 × B1
B2 '= α2 × B2
B3 '= α3 × B3
B4 '= α4 × B4 (2)

  As described above, the image correction device 1 according to the second embodiment includes a plurality of regions corresponding to the exposure time and the acceleration of the imaging target 23 when each portion of the non-final time image is formed. Divide into The second correction value of the second embodiment corresponds to the non-final time image and differs depending on the acceleration of the imaging target 23. The correction unit 12 performs, for a plurality of areas included in the non-final time image, the first correction corresponding to the exposure time required when the area to be corrected is formed. The correction is performed using the value and the second correction value corresponding to the acceleration of the imaging target 23 when the area to be corrected is formed. The correction unit 12 corrects the luminance value of the area to be corrected according to the method described in the first embodiment for a plurality of areas included in the final time image.

  That is, as in the first embodiment, the image correction apparatus 1 according to the second embodiment corrects the correction target image at the final time of imaging even when the imaging target 23 performs acceleration motion. The image of the imaging object 23 which does not contain 23a can be obtained. Furthermore, the image correction device 1 according to the second embodiment takes the image of the imaging target 23 whose luminance value has been corrected after the first imaging device 21 has formed one frame in consideration of the exposure time. You can get it.

  The image correction device 1 according to the second embodiment corrects the correction target image according to the method described in the first embodiment without considering the acceleration of the imaging target 23 even when the imaging target 23 performs acceleration motion. May be Even in such a case, the image correction apparatus 1 can generate a corrected image in which the influence of the blurred image 23a is smaller than that in the related art and the exposure time is taken into consideration.

Third Embodiment
Next, an image correction apparatus 1A according to the third embodiment will be described. FIG. 11 is a diagram showing the configuration of the image correction apparatus 1A according to the third embodiment. The image correction apparatus 1A has all the components of the image correction apparatus 1 according to the first or second embodiment. The image correction apparatus 1A has components other than the components of the image correction apparatus 1. In the third embodiment, parts different from the first or second embodiment will be described. The image correction apparatus 1A further includes a correction image storage unit 15 that stores data indicating the image after correction generated by the correction image generation unit 13. An example of the correction image storage unit 15 is a semiconductor memory.

  The image correction apparatus 1A generates an interpolation image generation unit 16 that generates an interpolation image based on the first image based on the data stored in the correction image storage unit 15 and the position of the imaging target 23. Furthermore, it has. The first image is an image obtained by the correction unit 12 performing the correction. The position of the imaging target 23 is identified based on the data stored in the movement amount storage unit 8. Furthermore, the position of the imaging target 23 is specified by the image data received by the communication unit 3 from the second imaging device 22.

  Specifically, the first image is an image generated by the corrected image generation unit 13 after the correction unit 12 performs the correction. The interpolation image is a first time at which an image before correction of the first image is obtained, and an image before correction of a second image obtained next to the first image by the correction unit 12 performing correction. It is an image of the imaging object 23 of the time between the obtained 2nd time. The display unit 14 also displays the interpolation image generated by the interpolation image generation unit 16.

  FIG. 12 is a diagram for explaining the function of the interpolation image generation unit 16 included in the image correction device 1A according to the third embodiment. FIG. 12A shows a first image at a first time, and shows a second image at a second time. FIG. 12B shows the interpolation image 23 b generated by the interpolation image generation unit 16 at an intermediate time between the first time and the second time in addition to the first image and the second image. Is shown.

  It is assumed that the image of the imaging target 23 is imaged by the first imaging device 21 imaging the imaging target 23 at an intermediate time. In addition, the image of the imaging target 23 at an intermediate time points the imaging target 23 of the first image from the position of the imaging target 23 of the first image to the direction of the position of the imaging target 23 of the second image. It is assumed that the image is moved by the movement amount Z from the position of the imaging target 23 of the one image.

  FIG. 12A shows an image 23 b of the imaging target 23 in which the imaging target 23 of the first image has been moved by the movement amount Z at a middle time by a broken line. The movement amount Z is specified based on the position of the imaging target 23. Specifically, the movement amount Z is specified based on the data stored in the movement amount storage unit 8. Furthermore, the movement amount Z is specified by the image data received by the communication unit 3 from the second imaging device 22.

  Based on the first image and the movement amount Z, the interpolation image generation unit 16 generates an interpolation image 23b which is an image of the imaging target 23 at an intermediate time, as shown in FIG. 12 (B). Specifically, the interpolation image generation unit 16 moves the imaging target 23 of the first image from the position of the imaging target 23 of the first image to the direction of the position of the imaging target 23 of the second image by the movement amount Z It is moved to generate an interpolated image 23 b which is an image of the imaging target 23 at an intermediate time.

  As described above, the image correction apparatus 1A determines, based on the first image and the position of the imaging target 23, more specifically, based on the first image and the movement amount Z, as shown in FIG. As shown in, the interpolation image 23b which is an image of the imaging target 23 at an intermediate time is generated. That is, in addition to the first image and the second image, the image correction apparatus 1A can present the image of the imaging target 23 at an intermediate time to the user. The user can confirm the movement of the imaging target 23 by visually recognizing the interpolation image 23 b which is an image of the imaging target 23 at an intermediate time.

  In the first to third embodiments, the image correction devices 1 and 1A may be connected to an encoder and a control device. FIG. 13 is a diagram showing a state in which the image correction devices 1 and 1A according to the first to third embodiments are connected to the encoder 26 and the control device 27. The encoder 26 is a device that detects the position of the imaging target 23. The control device 27 is a device that controls the moving speed of the imaging target 23 by controlling the belt conveyor 24.

  In this case, the communication unit 3 receives information related to the position of the imaging target 23 from one or both of the encoder 26 and the control device 27. The position of the imaging target 23 or the acceleration of the imaging target 23 may be specified by the information received by the communication unit 3. The encoder 26 may be replaced by a sensor that detects the position or velocity of the imaging target 23. When the encoder 26 is replaced with a sensor that detects the position or velocity of the imaging target 23, the communication unit 3 receives information related to the position of the imaging target 23 from the sensor.

  Further, the first imaging device 21 may be a device that generates image data based on a result of detecting and detecting depth or sound.

  The trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12, and the correction image generation unit included in the image correction apparatus 1 according to the first embodiment. Some or all of the functions of 13 may be realized by a processor that executes a program stored in a memory.

  The processor is a central processing unit (CPU), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a digital signal processor (DSP).

  The processor realizes all or some of the functions of the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12 and the correction image generation unit 13 When it is performed, the part or all of the functions are realized by a combination of a processor and software, firmware, or software and firmware. Software or firmware is described as a program and stored in a memory. The processor reads and executes the program stored in the memory, thereby generating the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, and the correction unit 12. And implement all or some of the functions of the corrected image generation unit 13.

  The processor realizes all or some of the functions of the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12 and the correction image generation unit 13 When the image correction device 1 includes the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12 and the correction image generation unit 13. It has a memory for storing a program that results in the steps performed by some or all being executed. The program stored in the memory is a part of the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12 and the correction image generation unit 13 Alternatively, it may be said that the computer is made to execute a procedure or method that is entirely performed.

  The memory is, for example, nonvolatile or random access memory (RAM), read only memory (ROM), flash memory, erasable programmable read only memory (EPROM), EEPROM (registered trademark) (electrically erasable programmable read only memory) or the like. It is volatile semiconductor memory, magnetic disk, flexible disk, optical disk, compact disk, mini disk, or DVD (Digital Versatile Disk).

  The trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12, and the correction image generation unit included in the image correction apparatus 1 according to the first embodiment. Some or all of the functions of 13 may be realized by the processing circuit.

  The processing circuit is dedicated hardware. The processing circuit may be, for example, a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. is there. The trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12 and a part of the correction image generation unit 13 are dedicated Hardware may be used.

  The plurality of functions of the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12 and the correction image generation unit 13 A part may be realized by software or firmware, and the rest of the plurality of functions may be realized by dedicated hardware. As described above, the plurality of functions of the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12 and the correction image generation unit 13 It can be realized by hardware, software, firmware, or a combination thereof.

  The trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12, and the correction image generation unit of the image correction device 1A according to the third embodiment 13 and some or all of the functions of the interpolation image generation unit 16 may be realized by a processor. When a part or all of the functions are realized by the processor, the image correction device 1A includes the trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, and the division unit 10. And a memory for storing a program that results in execution of steps executed by part or all of the correction unit 12, the correction image generation unit 13, and the interpolation image generation unit 16.

  The trigger generation unit 2, the communication unit 3, the feature extraction unit 5, the movement amount calculation unit 7, the correction image specification unit 9, the division unit 10, the correction unit 12, and the correction image generation unit of the image correction device 1A according to the third embodiment 13 and some or all of the functions of the interpolation image generation unit 16 may be realized by a processing circuit.

  The configuration shown in the above embodiment shows an example of the contents of the present invention, and can be combined with another known technique, and one of the configurations is possible within the scope of the present invention. It is also possible to omit or change parts.

  1, 1A image correction device, 2 trigger generation unit, 3 communication unit, 4 image storage unit, 5 feature extraction unit, 6 feature storage unit, 7 movement amount calculation unit, 8 movement amount storage unit, 9 correction image identification unit, 10 A division unit, 11 correction value storage unit, 12 correction unit, 13 correction image generation unit, 14 display unit, 15 correction image storage unit, 16 interpolation image generation unit, 21 first imaging device, 22 second imaging device, 23 Imaging target, 23a blurred image, 24 belt conveyors, 26 encoders, 27 controllers, 50 imaging systems.

Claims (7)

An image obtained at a time when one frame is formed by the first imaging device for imaging a moving imaging target, a final time image which is an image formed at the final time of the time, and the final of the time And dividing the final time image and the non-final time image into the final time image and the non-final time image based on the position of the imaging target. A division unit for dividing into a plurality of regions for each exposure time required when each portion of the non-final time image is formed;
And a correction unit that corrects the luminance value of each of a plurality of areas included in the final time image and the non-final time image obtained by the division unit,
The correction unit is configured to, for a plurality of areas included in the final time image, have a first luminance value corresponding to an exposure time required when the area to be corrected is formed. Correction is performed using a correction value, and for a plurality of areas included in the non-final time image, the luminance value of the area to be corrected corresponds to the exposure time required when the area to be corrected is formed. An image correction apparatus characterized by performing correction using a first correction value to be corrected and a second correction value corresponding to the non-final time image. Detecting a position or a speed of the imaging target, which receives image data from a second imaging apparatus that captures the imaging target at a frame rate higher than a frame rate at which the first imaging device captures the imaging target The communication unit further includes a communication unit that receives information related to the position of the imaging target from a sensor, or receives information related to the position of the imaging target from a control device that controls the moving speed of the imaging target.
The dividing unit divides each of the final time image and the non-final time image into the plurality of areas based on the position of the imaging target specified by the image data or information received by the communication unit. The image correction apparatus according to claim 1, further comprising: The imaging target is performing an acceleration motion,
The division unit divides the non-final time image into a plurality of regions corresponding to an exposure time when each portion of the non-final time image is formed and an acceleration of the imaging target.
The second correction value corresponds to the non-final time image and differs depending on the acceleration of the imaging target,
The correction unit corresponds to the exposure time required when the area to be corrected is formed for the plurality of areas included in the non-final time image, the luminance value of the area to be corrected being formed. The image correction apparatus according to claim 1, wherein the correction is performed using the first correction value and the second correction value corresponding to the acceleration of the imaging target when the area to be corrected is formed. . Detecting a position or a speed of the imaging target, which receives image data from a second imaging apparatus that captures the imaging target at a frame rate higher than a frame rate at which the first imaging device captures the imaging target The communication unit further includes a communication unit that receives information related to the position of the imaging target from a sensor, or receives information related to the position of the imaging target from a control device that controls the moving speed of the imaging target.
The division unit is configured to form the non-final time image and each portion of the non-final time image based on a position and an acceleration of the imaging target specified by the image data or information received by the communication unit. Divided into a plurality of areas corresponding to the exposure time at the time of image
When the correction unit corrects the luminance value of each of the plurality of areas included in the non-final time image, the correction unit uses the acceleration of the imaging target specified by the image data or information received by the communication unit. The image correction apparatus according to claim 3, wherein the correction is performed using the corresponding second correction value. The first time at which the image before correction of the first image is obtained based on the first image obtained by performing the correction by the correction unit and the position of the imaging target, and the correction unit The interpolation image generation unit further generates an image of the imaging target at a time between a second time before the correction of the second image obtained next to the first image by performing the correction and a second time before the correction. The image correction apparatus according to claim 1, comprising: Detecting a position or a speed of the imaging target, which receives image data from a second imaging apparatus that captures the imaging target at a frame rate higher than a frame rate at which the first imaging device captures the imaging target The communication unit further includes a communication unit that receives information related to the position of the imaging target from a sensor, or receives information related to the position of the imaging target from a control device that controls the moving speed of the imaging target.
The interpolation image generation unit is configured to calculate the time between the first time and the second time based on the position of the imaging target specified by the image data or information received by the communication unit. The image correction apparatus according to claim 5, wherein an image of an imaging target is generated. An image obtained at a time when one frame is formed by the first imaging device for imaging a moving imaging target, a final time image which is an image formed at the final time of the time, and the final of the time And dividing the final time image and the non-final time image into the final time image and the non-final time image based on the position of the imaging target. A division step of dividing into a plurality of areas for each exposure time required when each portion of the non-final time image is formed;
Correcting the luminance value of each of a plurality of areas included in the final time image and the non-final time image obtained in the division step;
In the correction step, for a plurality of areas included in the final time image, a first luminance value of the area to be corrected corresponds to an exposure time required when the area to be corrected is formed. Correction is performed using a correction value, and for a plurality of areas included in the non-final time image, the luminance value of the area to be corrected corresponds to the exposure time required when the area to be corrected is formed. And correcting using the first correction value and the second correction value corresponding to the non-final time image.

Images