WO2023234356A1 - 撮像システム、及び、それを備えた移動体 - Google Patents

撮像システム、及び、それを備えた移動体 Download PDF

Info

Publication number
WO2023234356A1
WO2023234356A1 PCT/JP2023/020299 JP2023020299W WO2023234356A1 WO 2023234356 A1 WO2023234356 A1 WO 2023234356A1 JP 2023020299 W JP2023020299 W JP 2023020299W WO 2023234356 A1 WO2023234356 A1 WO 2023234356A1
Authority
WO
WIPO (PCT)
Prior art keywords
imaging
optical axis
image
imaging device
imaging system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/020299
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
智司 松井
範一 勝山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2024524914A priority Critical patent/JPWO2023234356A1/ja
Publication of WO2023234356A1 publication Critical patent/WO2023234356A1/ja
Priority to US18/961,954 priority patent/US20250097581A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00—Special procedures for taking photographs; Apparatus therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • G03B17/02—Bodies
    • G03B17/17—Bodies with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B30/00—Camera modules comprising integrated lens units and imaging units, specially adapted for being embedded in other devices, e.g. mobile phones or vehicles
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B5/00—Adjustment of optical system relative to image or object surface other than for focusing
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B5/00—Adjustment of optical system relative to image or object surface other than for focusing
    • G03B5/06—Swinging lens about normal to the optical axis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation

Definitions

  • the present disclosure relates to an imaging system that is fixed to a moving body and captures an image while the moving body is moving, and a moving body equipped with the same.
  • inspection efficiency is significantly improved by capturing images of infrastructure equipment while moving with a moving body and detecting defective locations through image processing of the captured images.
  • Patent Document 1 an image of a target area is captured while moving using a camera installed in a vehicle. Further, when the vehicle travels at a high speed, camera movement causes camera shake, but in Patent Document 1, a saccade mirror technique is used to correct the movement blur. Shake is reduced by irradiating light onto the object to be imaged and having the light reflected by the object be reflected by a mirror that rotates for a predetermined exposure time and then incident on the camera.
  • a lens with a long focal length is used, which reduces the angle of view and narrows the imaging range.
  • the present disclosure provides an imaging system that can expand the imaging range and a mobile object equipped with the same.
  • the imaging system of the present disclosure includes an imaging device disposed on a moving body, and a first optical axis of the imaging device when capturing a first image when the imaging device captures an image while the moving body is moving in a first direction.
  • the optical axis of the imaging device is moved in two or more directions so that the optical axis of the imaging device is changed from the state of
  • the present invention includes an optical axis changing mechanism that sequentially changes the optical axis of the imaging device in two directions by changing the state of the optical axis of the number k, which is a number, and a control device that operates the optical axis changing mechanism.
  • the control device captures the r-th image between the first image capturing timing and the second image capturing timing, and after the image capturing device captures the r-th image that satisfies the condition 2 ⁇ r ⁇ n in the total number of captured images n.
  • the optical axis changing mechanism is operated between the imaging timing of the rth image and the imaging timing of the r+1th image.
  • the mobile object of the present disclosure includes the above-described imaging system.
  • the imaging system of the present disclosure and a moving object equipped with the same, it is possible to provide an imaging system that can expand the imaging range and a moving object equipped with the same.
  • a diagram for explaining a vehicle equipped with an imaging system in Embodiment 1 Block diagram showing the internal configuration of the imaging system in Embodiment 1 Explanatory diagram illustrating the state of each imaging device in two optical axis states Explanatory diagram showing a captured image in the first imaging mode Explanatory diagram showing a captured image in the second imaging mode Explanatory diagram illustrating the imaging magnification range by the imaging device in two optical axis states Flowchart showing imaging processing in Embodiment 1 Graph showing the relationship between change in movement speed, timing of exposure time, and optical axis change in Embodiment 1 Diagram showing a modification of the imaging mode Diagram showing a modification of the imaging mode Explanatory diagram illustrating the imaging enlarged range by the imaging device in two optical axis states in Embodiment 2 Diagram for explaining a vehicle equipped with an imaging system in Embodiment 3 Explanatory diagram illustrating the state of each imaging device in two optical axis states in Embodiment 3 Block diagram showing the internal configuration of the imaging system in Embodiment 3 Explanatory diagram explaining
  • Embodiment 1 will be described below with reference to the drawings.
  • the moving object is a vehicle 3 such as a car
  • the imaging system 1 is attached to the upper part of the vehicle 3.
  • the imaging system 1 of Embodiment 1 is arranged to take an image of a road, for example.
  • FIG. 1 is a diagram for explaining an imaging system 1.
  • FIG. 2 is a block diagram showing the internal configuration of the imaging system 1.
  • a vehicle 3 is traveling on a road 4, for example.
  • holes 4b and cracks 4c occur on the road 4.
  • potholes, ruts, etc. that have occurred on the road surface can be detected from the captured images through image processing.
  • the imaging target of the imaging system 1 is at least a part of the structure around the vehicle 3, and is an object that moves relatively according to the moving speed of the vehicle 3 as the vehicle 3 moves.
  • the imaging target area 9 is an area of this imaging target that is acquired as an image.
  • images may be taken of the inner walls of tunnels, the sides and bottoms of overpasses, utility poles, and electric wires. With this, it is possible to detect holes, cracks, lifting, peeling, seams, inclinations of telephone poles, and deflections of electric wires in the imaged object through image processing from the acquired images.
  • An imaging system 1 is installed on the top surface of the vehicle 3.
  • the imaging system 1 is fixed so as to capture an image of the road 4 below the vehicle 3 in FIG.
  • the imaging system 1 includes a speed detection device 3a, an imaging device 11, an optical axis changing mechanism 12, and a control device 15.
  • the imaging device 11 captures an image of the surroundings of the vehicle 3, and in the first embodiment captures an image of the road surface of the road 4.
  • the imaging device 11 includes a camera body 21, a lens 23, a shutter 24, an imaging element 25, and a camera control section 27.
  • the speed detection device 3a is placed in the vehicle 3 and detects the moving speed of the vehicle 3. Thereby, it is also possible to detect that the vehicle 3 is moving.
  • the speed detection device 3a detects the moving speed based on a vehicle speed pulse signal in which ON/OFF of the pulse signal is switched for each constant rotation amount (rotation angle) of the axle of the vehicle 3.
  • the speed detection device 3a sends a vehicle speed pulse signal to the control device 15 together with the detected moving speed.
  • the speed detection device 3a may be, for example, a vehicle speed sensor that detects the moving speed from the rotational speed of the axle of the vehicle 3.
  • the detection of the moving speed may be performed by the control device 15 based on the vehicle speed pulse signal.
  • the camera body 21 has a replaceable lens 23 attached thereto, and houses an image sensor 25 and a camera control section 27.
  • An image sensor 25 is arranged at a focal length F of the lens 23.
  • the lens 23 is oriented directly toward the road 4, which is the subject.
  • the camera body 21 and the lens 23 may be integrated.
  • the image sensor 25 converts the received light into an electrical signal according to the intensity, and is, for example, a solid-state image sensor such as a CCD image sensor, a CMOS image sensor, or an infrared image sensor.
  • the camera control unit 27 opens the shutter 24 while receiving the exposure instruction signal from the control device 15.
  • the shutter 24 may have a configuration in which a plurality of blade diaphragms open and close, or may be an electronic shutter.
  • the camera body 21 is supported by a base 61.
  • the base 61 is rotatably supported on the upper surface of the vehicle 3.
  • the optical axis changing mechanism 12 includes a base 61 and a rotation drive section 63. In addition to rotating the optical path to change the optical axis, the optical axis changing mechanism 12 may also rotate the imaging device 11 around a point different from the principal point 23b of the lens 23, or The device 11 may also be translated in parallel.
  • the base 61 supports the camera body 21.
  • the imaging device 11 may have a configuration in which the camera body 21, the lens 23, and the optical axis changing mechanism 12 are integrated.
  • the imaging device 11 may have a panning function that horizontally rotates the camera body 21 and the lens 23.
  • the rotation drive unit 63 rotates the base 61 based on a rotation instruction from the control device 15.
  • the rotation drive unit 63 includes, for example, a motor and gears.
  • the optical axis changing mechanism 12 may include, for example, a rotation stage, and the imaging device 11 may be rotated by the rotation stage.
  • FIG. 3 is an explanatory diagram illustrating the imaging device 11 in two types of optical axis states
  • FIG. 3(a) shows the imaging device in the first state C1 in which the lens 23 is in the first optical axis 23ab state
  • FIG. 3B is an explanatory diagram showing the imaging device 11 in a second state C2 in which the lens 23 is on the second optical axis 23ac.
  • the optical axis changing mechanism 12 rotates the imaging device 11, for example, with the principal point 23b of the lens 23 as the rotation center. Thereby, as shown in FIG. 4, the imaging range of the imaging device 11 can be expanded in the second direction.
  • the imaging device 11 when the imaging device 11 is in the first state C1, the first image Im1 (see FIG. 4) is captured, and after the imaging of the image Im1 is completed, the imaging device 11 changes to the second state C2 and the image is captured. Image Im2. After the image Im2 is captured, the optical axis 23a of the lens 23 is moved toward the first optical axis 23ab in the third direction, which is the Y-axis direction, so that the imaging device 11 changes to the first state C1, and the image Im3 is moved to the first state C1. Take an image.
  • the fourth image Im4 By imaging the road 4 while the imaging device 11 alternately changes between the first state C1 and the second state C2, the fourth image Im4, the fifth image Im5, and the sixth image
  • the mode in which the imaging device 11 is changed in this way to take an image is referred to as a first imaging mode.
  • an end region Im3a on the opposite side to the movement direction in the third image Im3 is imaged to overlap with an end region Im1a in the movement direction in the first image Im1.
  • an end region Im1b in the second direction (+Y-axis direction) of the first image Im1 and an end region Im2b in the third direction (-Y-axis direction) of the second image Im2.
  • the images are taken so that they overlap.
  • an end region Im2b in the third direction (-Y-axis direction) of the second captured image Im2, and an end region Im3b in the second direction (+Y-axis direction) of the third captured image Im3. are imaged so that they overlap.
  • the first image Im1 and the third image Im3 have a common imaging area
  • the first image Im1 and the second image Im2 have a common imaging area
  • the third image Im3 each have a common imaging area.
  • adjacent images can have overlapping imaging regions.
  • the road 4 may be imaged without any gaps. In either case, it is possible to prevent omission of imaging between images.
  • the imaging system 1 may have a second imaging mode in which images Im1 to Im6 are captured while maintaining the first state C1 without changing the imaging device 11.
  • captured images can be continuously acquired in a line.
  • the widths of the images Im1 to Im6 in the Y-axis direction are different for clarity, but the actual widths are all the same.
  • the imaging area of the image Im2 is entirely included in the imaging areas of the image Im1 and the image Im3, so it is not necessary to image the image Im2.
  • images Im1, Im3, and Im5 may be captured one after another.
  • FIG. 6 is an explanatory diagram illustrating the amount of expansion of the imaging range
  • the imaging target surface is the road surface of the road 4.
  • the enlargement amount WL2 is calculated by the following equation (2) using the optical axis change angle ⁇ .
  • the optical axis change angle ⁇ is an angle rotated by the optical axis change from the first optical axis 23ab.
  • WL2 D1 ⁇ tan ⁇ ( ⁇ +(a/2) ⁇ D1 ⁇ tan(a/2)...Equation (2)
  • the optical axis change angle ⁇ is 5 [deg]
  • the enlargement amount WL2 is 0.15 [m]
  • the imaging range is expanded by about 44% in the horizontal direction.
  • optical axis change angle ⁇ field angle a optical axis change angle ⁇ field angle a.
  • the optical axis change angle ⁇ may be a predetermined angle designated by the user from the operation unit 19, or may be determined by the control device 15 based on the distance to the imaging target area 9 and the angle of view a of the imaging device 11. .
  • the control device 15 controls the exposure time of the imaging device 11 and the driving of the optical axis changing mechanism 12, and operates the optical axis changing mechanism 12 between the timing of capturing the first image and the timing of capturing the second image.
  • the control device 15 is a circuit that can be realized using a semiconductor element or the like.
  • the control device 15 can be configured with, for example, a microcomputer, CPU, MPU, GPU, DSP, FPGA, or ASIC.
  • the functions of the control device 15 may be configured only by hardware, or may be realized by a combination of hardware and software.
  • the control device 15 realizes predetermined functions by reading data and programs stored in the storage unit 17 and performing various calculation processes.
  • the control device 15 has an optical axis change instruction section 71.
  • the optical axis change instruction unit 71 controls the rotation drive unit of the optical axis change mechanism 12, for example, in accordance with the timing at which the speed detection of the vehicle 3 is received by the speed detection device 3a or the imaging timing of the imaging device 11 at a constant frame rate. 63 to change the state of the optical axis.
  • the control device 15 transmits an exposure control signal to the camera control section 27.
  • the exposure control signal has two types of signals: a Hi signal as an ON signal that instructs exposure, and a Low signal as an OFF signal that does not instruct exposure. Note that when capturing an image at a constant frame rate, the camera control section 27 may control the exposure instead of the control device 15.
  • the storage unit 17 is a storage medium that stores programs and data necessary to realize the functions of the control device 15.
  • the storage unit 17 can be realized by, for example, a hard disk (HDD), SSD, RAM, DRAM, ferroelectric memory, flash memory, magnetic disk, or a combination thereof.
  • the operation unit 19 is an input device for a user to give instructions to the control device 15.
  • the operation unit 19 may be an input device dedicated to the imaging system 1, or may be a mobile terminal such as a smartphone. When a mobile terminal is used as the operation unit 19, the operation unit 19 and the control device 15 transmit and receive data through wireless communication.
  • the user may use the operation unit 19 to instruct the control device 15 whether the imaging target area is an indoor dark area such as a tunnel, or an outdoor bright area such as a mountain slope or a road, and may also specify the imaging interval Tf. may be instructed.
  • the imaging interval Tf is the time between when the current image is captured and when the next image is captured, and is the time of one frame in the case of video capture, and the time of one frame in the case of still image capture.
  • the imaging interval Tf1, Tf2 for each image is illustrated as the imaging interval Tf
  • the exposure time Tp1, Tp2, Tp3 for each image is illustrated as the exposure time Tp.
  • the exposure times Tp1, Tp2, and Tp3 are the times from times t1, t3, and t5 when imaging starts to times t2, t4, and t6 when imaging ends, respectively.
  • the frame rate (number of images captured per second) may be specified.
  • the user may also use the operation unit 19 to instruct the control device 15 to switch between the first imaging mode in which the optical axis changing mechanism 12 is operated and the second imaging mode in which the optical axis changing mechanism 12 is not operated. good.
  • FIG. 7 is a flowchart showing the imaging process performed by the imaging system 1.
  • FIG. 8 is a graph showing the relationship between exposure time and timing of changing the optical axis.
  • FIG. 8(a) is a graph showing the temporal change in vehicle speed
  • FIG. 8(b) is a graph showing the output timing of the exposure control signal
  • FIG. 8(c) is a graph showing the temporal change in the optical axis change angle ⁇ . It is a graph.
  • the imaging process shown in FIG. 7 is started, for example, when an instruction to start imaging is received from the operation unit 7 while the vehicle 3 is moving.
  • step S1 the user measures in advance the object distance from the image sensor 25 of the camera body 21 to the road surface of the road 4 to be imaged, and sets the measured object distance in the control device 15 using the operation unit 7. do. Further, by setting the section of the road to be imaged, for example, the control device 15 can determine whether the vehicle has traveled on the set section of road based on the GPS information and the travel distance.
  • step S2 the vehicle 3 starts traveling, and the speed detection device 3a detects the moving speed of the vehicle 3.
  • the detected moving speed is sent to the control device 15.
  • the speed detection device 3a performs imaging in synchronization with detecting the speed, but the imaging may be performed at a predetermined cycle, or the user may set the imaging cycle using the operation unit 19. good.
  • the exposure is synchronized with the vehicle speed pulse, for example, if the moving distance of the vehicle 3 is detected at intervals of 40 cm, the frame rate will be approximately 40 fps if the traveling speed is 60 km/h.
  • the speed detection device 3a does not need to detect an accurate moving speed, and may be configured to detect that the vehicle is moving.
  • the detected movement state is then sent to the control device 15.
  • the control device 15 may perform imaging after detecting that the vehicle 3 is in a moving state. Note that the distance interval for speed detection may be changed depending on the specifications of the speed detection device 3a.
  • step S3 the optical axis change instruction unit 71 of the control device 15 instructs the rotation drive unit 63 of the optical axis change mechanism 12 to rotate if the number of images to be captured next is the second or more. As a result, the imaging direction of the imaging device 11 is changed.
  • the optical axis change instruction section 71 does not need to instruct the rotation drive section 63 to rotate.
  • the optical axis of the imaging device 11 is fixed during the first time Tm1, and the optical axis of the imaging device 11 is rotated during the second time Tm2. ing.
  • Tm1a and Tm1b are illustrated as the first time Tm1, and Tm2a and Tm2b are illustrated as the second time Tm2.
  • Changing the optical axis of the imaging device 11 starts at time t2 when the first image Im1 has been captured, and ends at time ta.
  • the period from time t2 to ta is the second time Tm2a.
  • the imaging device 11 is operated during a first time Tm1a from time ta to time tb, which is after time t4 at which the imaging of the second image Im2 ends.
  • the optical axis is fixed.
  • the optical axis change instruction unit 71 controls the optical axis change mechanism 12 so that the exposure time Tp ⁇ first time Tm1 ⁇ imaging interval Tf and the exposure time Tp is included within the period of the first time Tm1. According to this control, while the rotation drive section 63 is changing the optical axis of the imaging device 11, the control device 15 does not issue an exposure instruction to the camera control section 27, so that the influence of the optical axis change on image blur is reduced. can be avoided with high precision.
  • the range of possible values for the second time Tm2 is 0 ⁇ Tm2 ⁇ Tf ⁇ Tp. Therefore, in consideration of the responsiveness of the optical axis changing mechanism 12, if it is necessary to secure the second time Tm2 as much as possible, the first time Tm1 may be made smaller. In this case, the first time Tm1 does not necessarily have to satisfy the condition of exposure time Tp ⁇ first time Tm1.
  • the control device 15 continues to send, for example, a Hi signal to the camera control unit 27 of the imaging device 11 for an exposure time Tp2.
  • the image sensor 25 images the object to be imaged during the exposure time Tp2.
  • the image captured by the image sensor 25 is recorded from the camera control unit 27 to the storage unit 17, thereby obtaining a captured image.
  • step S5 the control device 15 determines whether the vehicle 3 has traveled a predetermined section. When the control device 15 determines that the vehicle 3 has finished traveling in the predetermined section, it has finished acquiring images of the road in this section, and thus ends the moving imaging. Alternatively, the control device 15 may end the moving imaging in response to an instruction from the operating section 7 by the user operating the operating section 7 . When the control device 15 determines that the vehicle 3 has not finished traveling in the predetermined section, the control device 15 returns to step S2 and performs moving imaging again.
  • step S4 the control device 15 continues to send, for example, a Hi signal to the camera control section 27 of the imaging device 11 for an exposure time Tp3. Thereby, the image sensor 25 images the object to be imaged during the exposure time Tp3.
  • the optical axis of the imaging device 11 has two optical axis states, the first optical axis 23ab and the second optical axis 23ac, but the present invention is not limited to this.
  • the optical axis changing mechanism 12 may change the optical axis of the imaging device 11 to k optical axis states, which is a number of two or more, and change the optical axis of the imaging device 11 in two directions sequentially. .
  • the imaging device 11 has three or more optical axis states, and the optical axis changing mechanism 12 changes the imaging device 11 to each optical axis state to capture an image in each optical axis state.
  • the object may be imaged. Thereby, the imaging range can be further expanded.
  • FIG. 9 shows the relationship between the imaging order and the captured images when the imaging device 11 has three optical axis states.
  • the optical axis of the imaging device 11 is changed from the second direction to the third direction opposite to the second direction.
  • the control device 15 operates the optical axis changing mechanism 12 between the imaging timing of the first image and the imaging timing of the second image, and between the imaging timing of the r-th image and the imaging timing of the r+1-th image.
  • the th captured image is an image taken in the first direction when the rth captured image is captured. Thereby, the imaging range can be further expanded compared to the case where two images are taken in the second direction.
  • the control device 15 calculates the interval Tfk between the r-th image and the r+k-th image based on the moving speed of the vehicle 3. If the shooting range of one image in the traveling direction is Lx [m], the overlapping range of images in the traveling direction is Ly [m], and the moving speed is V [km/h], then the r-th image and the r+k-th image are taken.
  • the imaging time interval Tx [sec] satisfies the following formula.
  • the coefficient c is a coefficient for converting the units of distance [m] and [km] and the units of time [hour] and [sec], and is 3.6.
  • Lymin is the minimum necessary overlapping range of images in the traveling direction, which is the range necessary for joining consecutive images into one image by image recognition.
  • Lymin may be 20% or more of the imaging range Lx.
  • the number k of changeable optical axis states can be calculated from the set time interval Tx and imaging interval Tf. k is calculated as an integer value that satisfies the following formula.
  • the control device 15 may calculate the imaging interval Tf between the r-th imaging and the r+1-th imaging based on the moving speed and the number k of changeable optical axes. As described above, the imaging interval Tf may be calculated from the time interval Tx set based on the moving speed V and the number k of optical axis states to be changed. The imaging interval Tf is set to a value that satisfies the following formula.
  • the fourth image Im4 and the seventh image Im7 have a common imaging area.
  • the fifth image Im5 has a common imaging area, and the fifth image Im5 and the seventh image Im7 have a common imaging area.
  • FIG. 10 shows the relationship between the imaging order and the captured images when the imaging device 11 has four optical axis states.
  • the fifth image Im5 and the sixth image Im5 and the sixth image Im9 have a common imaging area.
  • the eye image Im6 has a common imaging area
  • the sixth image Im6 and the ninth image Im9 have a common imaging area.
  • the imaging system 1 uses the imaging device 11 disposed in the vehicle 3 to capture the first image when the imaging device 11 captures an image while the vehicle 3 is moving in the first direction, which is the +X-axis direction.
  • the optical axis 23a of the imaging device 11 is changed from the state of the first optical axis 23ab of the imaging device 11 at the time of imaging to the state of the second optical axis 23ac displaced in the +Y-axis direction intersecting the +X-axis direction at the time of capturing the second image.
  • an optical axis changing mechanism 12 that sequentially changes the optical axis of the imaging device 11 in two directions by changing the optical axis of the imaging device 11 into k optical axes that are an integer of 2 or more so as to change the optical axis of the imaging device 11; , and a control device 15 that operates the optical axis changing mechanism 12.
  • the control device 15 sets the condition of 2 ⁇ r ⁇ n (n is an integer of 2 or more) between the first image capturing timing and the second image capturing timing and when the image capturing device 11 captures a total number of images n. After capturing the r-th image that satisfies the requirements, the optical axis changing mechanism is operated between the r-th image capturing timing and the r+1-th image capturing timing.
  • the range of images to be captured can be expanded. Further, since no image is captured while the optical axis is being changed, it is possible to prevent the captured image from being blurred.
  • the control device 15 adjusts the optical axis of the image-capturing device 11 to an optical axis based on the imaging interval Tf and exposure time Tp between the r-th image and the r+1-th image of the imaging device 11.
  • the optical axis of the imaging device 11 is fixed for 1 hour Tm1, and is changed at a second time Tm2 based on the imaging interval Tf and the first time Tm1 between the r-th imaging timing and the r+1-th imaging timing.
  • the optical axis changing mechanism 12 is operated as follows. Thereby, since the optical axis of the imaging device 11 is positive while capturing an image, it is possible to prevent image blurring due to rotation of the optical axis.
  • the optical axis change angle ⁇ that changes the optical axis of the imaging device 11 from the r-th image to the r+1-th imaging is less than or equal to the angle of view a of the imaging device 11. .
  • the r-th image and the r+1-th image can be continuous without a gap, or can have an overlapping area with each other.
  • the optical axis changing mechanism 12 moves the optical axis of the imaging device 11 in a second direction from the time of capturing the rth image with respect to the r+1th image capturing. and when r/k is an integer, the optical axis of the imaging device is changed from the time of capturing the r-th image to the third direction opposite to the second direction with respect to the r+1-th image.
  • the optical axis changing mechanism 12 changes the imaging device 11 to a state in which the first optical axis is displaced in the third direction when the r+1th image is captured with respect to the rth image. Change the optical axis.
  • the r+kth captured image is an image obtained by capturing the imaging target region 9 located in the first direction from the imaging target region 9 when the rth captured image was captured, and the r+kth captured image The end region on the opposite side to the first direction overlaps the end region in the first direction in the r-th captured image. Thereby, it is possible to prevent omission of imaging between images in the first direction.
  • control device 15 controls the optical axis changing mechanism 12 between the first image capturing timing and the second image capturing timing, and between the rth image capturing timing and the r+1th image capturing timing. It has a first imaging mode in which the imaging device 11 takes images while operating, and a second imaging mode in which the imaging device 11 continuously takes images without operating the optical axis changing mechanism 12. Thereby, the imaging mode can be selected according to the imaging range.
  • the imaging system 1A in the second embodiment images the imaging target by changing the states of the two optical axes, but in the first state C1 and the second state C2, the respective optical axes are tilted with respect to the imaging target. There is. Regarding the configuration other than this point and the points described below, the imaging system 1A in the second embodiment is common to the imaging system 1 in the first embodiment.
  • the imaging system 1 of Embodiment 1 achieves the first state C1 by placing the lens 23 in a position directly facing the imaging target, that is, by placing the optical axis in parallel with the axis perpendicular to the imaging target surface.
  • the object distance within the imaging plane became uniform, and an in-focus image could be captured.
  • the imaging device 11 is moved to the second state C2 in order to take pictures by changing the imaging direction of the camera so that a wide range can be taken in one traveling imaging, the position of the lens 23 changes to the facing position. In some cases, the subject distance varies within the imaging plane, resulting in a slightly blurred image.
  • the installation position of the imaging device 11 is tilted in consideration of changing the optical axis.
  • the imaging direction moves within a plane intersecting the moving direction of the vehicle 3 by a predetermined angle smaller than the optical axis changing angle ⁇ of the optical axis changing mechanism 12.
  • the imaging device 11 is installed with an installation inclination so as to be inclined around an axis parallel to the direction.
  • the moving direction of the vehicle 3 is the X-axis direction perpendicular to the plane of the paper, and the imaging direction is tilted around the X-axis parallel to the moving direction in the YZ plane that intersects with the moving direction of the vehicle 3.
  • a position where the imaging direction is tilted by half of the set optical axis change angle ⁇ /2 is defined as the position of the first state C1a.
  • the subject distance D1a in the first state C1a and the subject distance D2a in the second state C2 can be made the same distance. Since it is possible to suppress changes in the subject distance of images captured in two positions, the first state C1 and the second state C2, the initial position (first state C1a) with respect to the imaging target can be suppressed. and the optical axis change position (second state C2), it is possible to image symmetrically. Therefore, it is possible to both equalize the imaging accuracy at both positions and expand the angle of view.
  • FIG. 12A is an explanatory diagram for explaining a vehicle 3 equipped with an imaging system 1B according to the third embodiment.
  • FIG. 12B is an explanatory diagram illustrating the states of the imaging device in two optical axis states in Embodiment 3.
  • FIG. 13 is a block diagram showing the internal configuration of the imaging system 1B in the third embodiment.
  • FIG. 14 is an explanatory diagram illustrating blur correction of the imaging system 1B.
  • An imaging system 1B according to the third embodiment has a configuration in which the imaging system 1 according to the first embodiment is provided with a blur correction mechanism 31. Regarding the configuration other than this point and the points described below, the imaging system 1B in the third embodiment is common to the imaging system 1 in the first embodiment.
  • the vehicle 3 is traveling in a tunnel 5, for example.
  • holes 5b and cracks 5c are formed on the wall surface 5a within the tunnel 5.
  • the blur correction mechanism 31 corrects the optical path of light that enters the imaging system 1 so that even if the imaging device 11B captures an image while the vehicle 3 is moving, the movement blur of the image of the imaging target area 9 is reduced.
  • the camera body 21 is arranged on the vehicle 3 so that the lens 23 is oriented parallel to the direction of movement of the vehicle 3.
  • the camera body 21 is arranged so that the lens 23 faces the front or rear of the vehicle 3.
  • the blur correction mechanism 31 corrects the optical path of the light L1, which is the environmental light reflected by the imaging target area 9, in accordance with the movement of the vehicle 3.
  • the blur correction mechanism 31 aligns the direction of light L1, which is the ambient light reflected by the imaging target area 9, with the imaging direction of the imaging element 25.
  • the blur correction mechanism 31 includes, for example, a mirror 41 and a mirror drive section 43.
  • the mirror 41 totally reflects the ambient light reflected by the imaging target toward the imaging device 11 .
  • the optical axis changing mechanism 12B when the imaging device 11 takes an image while the vehicle 3 is moving in the first direction, which is the +X-axis direction, the optical axis changing mechanism 12B is configured to In the lens 23, from a first optical axis 23ad that is perpendicular to the ceiling of the tunnel 5 when taking the first image, to a second direction that is the +Y axis direction that intersects the first direction when taking the second image.
  • the optical axis 23a of the lens 23 is changed to the tilted second optical axis 23ae.
  • the optical axis changing mechanism 12B rotates both the blur correction mechanism 31 supported by the base 61 and the imaging device 11B around the rotation axis 61a. Therefore, by driving the optical axis changing mechanism 12B, the optical axis of the blur correction mechanism 31 is also changed together with the optical axis of the imaging device 11.
  • the shake correction mechanism 31 and the optical axis changing mechanism 12B are not limited to this configuration.
  • a pan-tilt rotation mechanism that rotates the camera body 21 and the lens 23 around a rotation axis may be used.
  • the blur correction mechanism 31 corresponds to a mechanism that rotates in the tilt direction
  • the optical axis changing mechanism 12B corresponds to a mechanism that rotates in the pan direction.
  • the blur correction mechanism 31 becomes a mechanism that rotates in the panning direction
  • the optical axis changing mechanism 12B becomes a mechanism that rotates in the tilting direction.
  • the lens 23 when the lens 23 is directed directly toward the subject, it can be rotated in the panning direction corresponding to the direction of movement.
  • the traveling direction can be set to the long side direction of the image sensor 25, and even if the vehicle 3 becomes high speed, it becomes easy to ensure the overlap of the captured images in the moving direction.
  • the mechanism that rotates in a uniaxial direction may be used as the optical axis changing mechanism 12B.
  • the blur correction mechanism 31 includes a mirror 41 and a mirror drive section 43.
  • the blur correction mechanism 31 and the optical axis changing mechanism 12B may be configured by two mirrors whose rotation axes are perpendicular to each other, and a motor.
  • the entire imaging device 11B may be rotated in two orthogonal directions.
  • a mechanism that rotates in the panning direction may be used as the optical axis changing mechanism 12B, and a mechanism that rotates the entire imaging device 11B in the tilting direction may be used as the blur correction mechanism 31.
  • the mirror 41 is rotatably arranged to face the lens 23.
  • the mirror 41 can be rotated, for example, in either a clockwise forward direction or a reverse direction, and the rotatable angular range may be less than 360 degrees or more than 360 degrees. Good too.
  • the mirror 41 totally reflects the ambient light reflected by the imaging target toward the imaging device 11 .
  • the mirror drive unit 43 rotates the mirror 41 from the initial angle to the specified angle, and after rotating the mirror 41 to the specified angle, returns the mirror 41 to the initial angle.
  • the mirror drive unit 43 is, for example, a motor.
  • the rotation angle of the mirror 41 is limited by mechanical constraints of the mirror drive unit 43, and the mirror 41 can be rotated up to the maximum swing angle of the mirror 41 determined by this limitation.
  • the moving blur correction angle ⁇ at which moving blur correction is possible is less than or equal to the maximum swing angle of the mirror 41.
  • blur correction by the blur correction mechanism 31 will be described. For example, assume that the imaging system 1 located at position A moves to position B together with the vehicle 3 during the exposure time. Assume that imaging is started at position A and an image is acquired at this timing. For example, the hole 5b in the imaging target area 9 is captured in the image acquired at position A, but the image is dark and not clear because the exposure time is not sufficient.
  • the imaging target area 9 will move relatively in the opposite direction to the moving direction of the vehicle 3, resulting in an image in which the hole 5b has moved relatively.
  • the amount of pixel movement is detected as the amount of blur. In this way, the image captured by the imaging device 11 while the vehicle 3 is moving becomes a blurred image.
  • the end 41a of the mirror 41 on the moving direction side rotates the mirror 41 in a direction that offsets the relative movement of the imaging target during the exposure time.
  • the imaging system 1 can capture the same imaging target region 9 in the captured image during the exposure time, and can obtain an image with significantly reduced movement blur.
  • the mirror 41 is rotated clockwise so that the end 41a of the mirror 41 on the moving direction side rotates toward the imaging target side during the exposure time. By rotating the mirror 41, the amount of pixel movement in the captured image is corrected to zero.
  • the control device 15B includes an optical axis change instruction section 71, a correction mechanism swing angle calculation section 73, and a correction mechanism rotation speed calculation section 75.
  • the correction mechanism swing angle calculation unit 73 calculates the mirror swing of the mirror 41 during imaging based on the moving speed V of the vehicle 3, the set exposure time Tp, the subject magnification M, and the focal length F of the lens 23. Angle ⁇ is calculated as follows. The mirror swing angle ⁇ corresponds to the correction mechanism swing angle.
  • the focal length F is a value determined by the lens 23.
  • the subject magnification M is a value determined by the focal length F and the subject distance D.
  • the subject distance D is the distance from the principal point 23b of the lens 23 disposed between the imaging target as the subject and the imaging device 25 to the imaging target area 9.
  • a known value measured in advance may be used, or a value measured by a rangefinder during imaging may be used.
  • the amount of movement P of a pixel on the image sensor 25 from the time when imaging starts to the time when imaging ends is calculated from the amount of movement L of the vehicle 3 and the subject magnification M using the following equation (4).
  • P [mm] L [mm] ⁇ M... (4) formula
  • the movement blur correction angle ⁇ is calculated from the pixel movement amount P and the focal length F using the following equation (5).
  • ⁇ [deg] arctan(P/F)...Equation (5)
  • the subject magnification M is calculated from the focal length F[mm] and the subject distance D[m] using the following equation (6). be done.
  • the mirror swing angle ⁇ required for movement blur correction during exposure is half the movement blur correction angle ⁇ , it is calculated by the following equation (8).
  • ⁇ ⁇ /q...Equation (8)
  • the coefficient q It is 2.
  • the coefficient q is 1.
  • the correction mechanism swing angle calculation unit 73 calculates the mirror swing angle ⁇ of the mirror 41.
  • the rotational speed Vm corresponding to each moving speed V of the vehicle 3 can be calculated. Therefore, by rotating the mirror 41 in the opposite direction to the moving direction at the rotation speed Vm after starting imaging, the imaging device 11B can capture light from the same imaging target area 9 during the exposure time. It is possible to receive light, and it is possible to suppress movement blur from occurring in a captured image.
  • FIG. 15 is a flowchart showing the imaging processing performed by the imaging system 1B.
  • FIG. 16 is a graph showing the relationship between exposure time, movement blur correction angle, and optical axis change angle.
  • step S11 the correction mechanism swing angle calculation unit 73 calculates the mirror swing angle ⁇ as the shake correction amount. Further, the correction mechanism rotation speed calculation unit 75 calculates the rotation speed Vm of the mirror 41 based on the mirror swing angle ⁇ .
  • step S12 the control device 15B causes the mirror drive unit 43 to rotate the mirror 41 at the calculated rotation speed Vm, so that the mirror 41 starts rotating from a predetermined initial angle that is a rotation start position. .
  • movement blur correction is performed during imaging by the imaging device 11B.
  • the control device 15B continues to send a Hi signal instructing exposure to the camera control section 27 during the exposure time Tp.
  • the camera control unit 27 acquires an image by opening the shutter 24 and exposing it while receiving the Hi signal (step S12), and stores the acquired image in the storage unit 17.
  • the control device 15B continues to send a Low signal to the camera control unit 27 as an OFF signal instructing to stop exposure.
  • a Low signal may be used as an ON signal for instructing exposure
  • a Hi signal may be used as an OFF signal for instructing to stop exposure.
  • the shutter 24 While the camera control unit 27 is receiving the Low signal, the shutter 24 is closed, and the control device 15B causes the mirror drive unit 43 to rotate the mirror 41 in the opposite direction to return the mirror 41 to its initial angle. Note that the mirror drive unit 43 may return the mirror 41 to its initial angle by rotating the mirror 41 in the forward direction. Note that this initial angle varies depending on the moving speed of the vehicle 3.
  • FIG. 16 is a graph showing the relationship between changes in movement speed, timing of exposure time, and movement blur correction angle.
  • FIG. 16(a) is a graph showing the moving speed of the vehicle 3 that changes over time.
  • FIG. 16(b) is a graph showing the timing of exposure time for each frame.
  • FIG. 16(c) is a graph showing the movement blur correction angle calculated for each frame.
  • FIG. 16(d) is a graph showing the optical axis change angle.
  • a Hi signal indicating an imaging instruction is transmitted and the first image Im1 is captured.
  • the speed detection device 3a detects the speed of the vehicle 3, and the movement blur correction angle of the next frame is calculated.
  • the optical axis change instruction unit 71 of the control device 15B instructs the optical axis change mechanism 12B to change the optical axis.
  • the mirror drive unit 43 drives the mirror 41 to the rotation start angle ⁇ 1 in the movement blur correction direction.
  • the dashed line extending diagonally from FIG. 16(a) to FIG.
  • the mirror drive section 43 begins to rotate the mirror 41 in the direction of correcting blurring at the rotation speed calculated by the correction mechanism rotation speed calculation section 75.
  • the control device 15B transmits a Hi signal indicating an imaging instruction to the camera control unit 27 during the first time Tm1 when the optical axis is fixed, and be done.
  • the optical axis changing mechanism 12B is in a state where the optical axis changing operation is completed.
  • the second image is corrected using the moving blur correction angle ⁇ 1 according to the speed V0 when the previous image was taken, and the third image is corrected using the moving blur correction angle ⁇ 1 when the second image was taken.
  • the blur is corrected using a moving blur correction angle ⁇ 2 corresponding to the speed V1.
  • the movement blur correction angle for the fourth image may be calculated according to the average speed of the speed V1 at the time of capturing the second image and the speed V2 at the time of capturing the third image.
  • the angle of view can be expanded while correcting blur according to the moving speed of the vehicle 3, so it is possible to acquire a wide-range image with high resolution.
  • FIG. 17 is a block diagram showing the internal configuration of an imaging system 1C in the fourth embodiment. Note that, in the first embodiment, FIG. 6 has been described assuming that the surface to be imaged is the road surface of the road 4, but in the fourth embodiment, the surface to be imaged is described as being the wall surface of the tunnel 5, particularly the ceiling.
  • An imaging system 1C in the fourth embodiment has a configuration in which the control device 15B of the imaging system 1B in the third embodiment includes a subject distance calculation unit 77.
  • the imaging system 1C in the fourth embodiment calculates the blur correction amount in accordance with the subject distance that changes as the optical axis changes.
  • the imaging system 1C in the fourth embodiment is common to the imaging system 1B in the third embodiment.
  • the subject distance calculation unit 77 calculates the subject distance on the second optical axis 23ae when the optical axis changes from the initial position of the first optical axis 23ad to the second optical axis 23ae.
  • a method of calculating the subject distance on the second optical axis 23ae will be described with reference to FIG. 12B and FIG. 6.
  • the third object distance D3 of the outer edge of the optical path of the angle of view a on the optical axis 23ae is calculated as follows.
  • the subject distance D3 is the length of a perpendicular line drawn from the end of the optical path at the angle of view a to the extension of the imaging plane after the optical axis has been changed.
  • the second subject distance D2 is calculated using the following equation (10).
  • D2 D1/cos( ⁇ ) (10)
  • the second subject distance D2 is 1.706 [m].
  • the third subject distance D3 is calculated using the following equation (11).
  • D3 [D1/cos ⁇ +(a/2) ⁇ ] ⁇ sin(90- ⁇ )...Formula (11)
  • the third subject distance D3 is 1.73[ m].
  • FIG. 18 is a flowchart showing the imaging process in the fourth embodiment.
  • the operation of the imaging system 1C in the fourth embodiment is the same as the operation of the imaging system 1B in the third embodiment with step S21 added.
  • steps S1 to S3, S5, S11, and S12 are the same as those of the imaging system 1C of the third embodiment, so the description thereof will be omitted.
  • the object distance calculation unit 77 calculates a second object distance D2 based on the optical axis change angle ⁇ in step S21, and sets the calculated second object distance D2 to the imaging target. Set a new subject distance.
  • the control device 15C calculates the second subject distance D2 from the imaging device 11B to the imaging target area, which changes before and after changing the optical axis, based on the optical axis changing angle ⁇ by the optical axis changing mechanism 12B, and A mirror swing angle ⁇ , which is a blur correction amount for correcting blur in the first direction, is set based on the subject distance D2.
  • the accuracy of the correction mechanism swing angle (shake amount) calculated by the correction mechanism swing angle calculation unit 73 can be improved. This makes it possible to achieve more accurate tracking and perform blur correction with higher accuracy.
  • FIG. 19 is a block diagram showing the internal configuration of the imaging system 1D in the fifth embodiment.
  • An imaging system 1D in the fifth embodiment has a configuration in which the control device of the imaging system 1C in the fourth embodiment is provided with a subject distance detection device 81. Regarding the configuration other than this point and the points described below, the imaging system 1D in the fifth embodiment is common to the imaging system 1C in the fourth embodiment.
  • the subject distance detection device 81 detects the distance from the principal point of the lens 23 to the subject.
  • the subject distance detection device 81 is, for example, a laser measuring device.
  • Information on the subject distance detected by the subject distance detection device 81 is sent to the control device 15D.
  • the correction mechanism swing angle calculation unit 73 of the control device 15D calculates the correction mechanism swing angle at the first optical axis 23ad based on the detected first subject distance D1.
  • the object distance detection device 81 detects the distance from the principal point of the lens 23 to the object, thereby correcting blur even if the distance from the wall surface 5a in the tunnel 5 of the imaging device 11B is changed according to the on-site situation.
  • the amount can be calculated with high accuracy. Thereby, the angle of view of the imaging device 11B can be easily adjusted by adjusting the distance between the imaging device 11B and the subject.
  • FIG. 20 is a flowchart showing the imaging process in the fifth embodiment.
  • the operation of the imaging system 1D in the fifth embodiment differs from the operation of the imaging system 1C in the fourth embodiment by omitting step S1 and adding steps S31 and S32.
  • the subject distance detection device 81 instead of measuring the subject distance in advance, measures the first subject distance D1 when the optical axis is at the first optical axis 23ad, which is the initial position.
  • step S31 the control device 15D determines whether the optical axis is at the first optical axis 23ad, which is the initial position.
  • the control device 15D determines that the optical axis is on the first optical axis 23ad (Yes in step S31)
  • the subject distance detection device 81 detects the first subject distance D1 in step S32, and the control device 15D detects this detected value. This is set as the subject distance on the first optical axis 23ad.
  • step S11 the accuracy of the correction mechanism swing angle calculated by the correction mechanism swing angle calculation unit 73 can be improved.
  • the subject distance calculation unit 77 calculates the second subject distance D2 based on the optical axis change angle ⁇ in step S21.
  • the calculated second subject distance D2 is set as a new subject distance to the imaging target.
  • the control device 15D calculates the mirror swing angle ⁇ based on the first subject distance D1 before the optical axis change and the optical axis change angle ⁇ .
  • the control device 15D causes the object distance detection device 81 to detect the object distance in the state of the first optical axis 23ad before changing the optical axis, and changes the optical axis to the second optical axis 23ae using the optical axis changing mechanism 12B.
  • subject distance detection by the subject distance detection device 81 is not performed. Therefore, by detecting the object distance using the object distance detection device 81 only when the optical axis is at the initial position, when changing the optical axis from the wall image of the tunnel 5 directly above to oblique information, it is possible to This makes it possible to prevent erroneous distance detection when a tall vehicle in a lane enters the object distance detection range.
  • FIG. 21 is a flowchart showing imaging processing in a modification of the fifth embodiment.
  • the operation of the imaging system 1D in the modification of the fifth embodiment is such that step S33 is added to the operation of the imaging system 1D in the fifth embodiment.
  • step S33 the control device 15D sets the optical axis change angle ⁇ according to the subject distance detected by the subject distance detection device 81. For example, as the subject distance becomes closer, the detection value in step S32 becomes smaller, and with the same amount of optical axis change, there is no overlap area for the captured images in the second direction, which may result in missing images. In order to avoid this, the amount of optical axis change is optimally set based on the detected object distance. Therefore, in step S21, the subject distance at the changed position is calculated based on the set optical axis change amount. Thereby, the amount of view angle expansion can be made in accordance with the subject distance.
  • the imaging system 1 may include a speed detection device that detects the moving speed of the imaging system 1. Further, the speed detection device may be one that utilizes a GPS (Global Positioning System) system.
  • GPS Global Positioning System
  • the imaging system 1 images the upper and lower walls of the vehicle 3, but the invention is not limited thereto.
  • the imaging system 1 may image a wall surface on the side of the vehicle 3.
  • the moving object is a vehicle 3 such as a car
  • the moving object is not limited to the vehicle 3, but may be a vehicle that runs on the ground such as a train or a motorcycle, a ship that moves on the sea, or a flying object such as an airplane or a drone that flies in the sky.
  • the imaging system 1 images the bottom of a bridge pier or bridge girder, or a structure built along the shore.
  • the position and wear of the overhead wire can be detected by imaging the overhead wire.
  • an image is captured using the light that is reflected by the ambient light on the imaging target area 9, but the present invention is not limited to this.
  • Light may be emitted from a moving object or an imaging system toward the imaging target area 9, and an image may be captured using reflected light of the emitted light.
  • the imaging system of the present disclosure includes an imaging device disposed on a moving body, and when the imaging device captures an image while the moving body is moving in a first direction, a first image of the imaging device at the time of capturing a first image.
  • the optical axis of the imaging device is changed from the state of the first optical axis to the state of the second optical axis displaced in a second direction intersecting the first direction when capturing a second image.
  • An optical axis changing mechanism that sequentially changes the optical axis of the imaging device in two directions by changing the states of k optical axes, which are an integer of 2 or more, and a control device that operates the optical axis changing mechanism. .
  • the control device captures the r-th image between the first image capturing timing and the second image capturing timing, and after the image capturing device captures the r-th image that satisfies the condition 2 ⁇ r ⁇ n in the total number of captured images n.
  • the optical axis changing mechanism is operated between the imaging timing of the rth image and the imaging timing of the r+1th image.
  • the imaging system can expand the imaging range in a direction intersecting the direction in which the moving body moves, and can capture images over a wide range.
  • the control device adjusts the optical axis of the imaging device to the imaging interval between the r-th image and the r+1-th image and the exposure.
  • the optical axis of the imaging device is fixed for a first time based on time, and is changed at a second time based on the imaging interval and the first time between the imaging timing of the rth image and the imaging timing of the r+1th image. Then, operate the optical axis changing mechanism.
  • the optical axis of the imaging device is changed in the second direction from the time of capturing the r-th image with respect to the r+1-th image, and the optical axis of the imaging device is
  • the axis changing mechanism changes the optical axis of the imaging device from the time of capturing the r-th image to the third direction opposite to the second direction with respect to the r+1-th image.
  • the optical axis changing mechanism is configured such that when r/k is an integer, the first optical axis is displaced in the third direction when the r+1th image is captured relative to the rth image.
  • the optical axis of the imaging device is changed to the state shown in FIG.
  • the r+kth captured image is an image obtained by capturing an imaging target area located in the first direction from the imaging target area when the rth captured image was captured. , the end region on the opposite side to the first direction in the r+kth captured image overlaps with the end region in the first direction in the rth captured image.
  • the imaging system of (4) or (5) includes a speed detection device that detects the moving speed of the moving object, and the control device determines whether the r-th image is captured or the r+k image is captured based on the moving speed of the moving object. Calculate the eye imaging interval. Thereby, the imaging interval in the first direction can be defined.
  • the control device calculates the interval between capturing the r-th image and the r+1-th image based on the moving speed and the number k of optical axes. Thereby, the imaging interval in the second direction can be defined.
  • the optical axis change angle that changes the optical axis of the imaging device from the r-th image to the r+1-th image. is smaller than the angle of view of the imaging device.
  • the optical axis change angle at which the optical axis of the imaging device is changed from the r-th image to the r+1-th imaging is less than or equal to the field of view of the imaging device. Since the r-th captured image and the r+k-th captured image have a common imaging area, the r-th captured image, the r+1-th captured image, and the r+k-th captured image , each has a common imaging area.
  • the control device controls the timing between the first and second imaging timings, and between the rth imaging timing and r+1st imaging timing.
  • the imaging system includes a blur correction mechanism that corrects blur in the first direction when an image is captured by the imaging device while the moving object is moving.
  • the optical axis of the blur correction mechanism is also changed together with the optical axis of the imaging device by driving the optical axis changing mechanism.
  • the imaging direction intersects the movement direction of the imaging target of the imaging device by a predetermined angle smaller than the optical axis changing angle of the optical axis changing mechanism.
  • the imaging device is installed with an installation inclination so that it is tilted around the axis.
  • the installation inclination is an angle that is half the optical axis change angle.
  • the control device calculates the subject distance from the imaging device to the imaging target area, which changes before and after changing the optical axis, based on the optical axis changing angle by the optical axis changing mechanism. Then, a blur correction amount for correcting blur in the first direction is set based on the subject distance.
  • control device calculates the blur correction amount based on the subject distance and the optical axis change angle before changing the optical axis.
  • the imaging system of (18) and (17) includes a subject distance measuring device that measures the distance from the imaging device to the object to be imaged.
  • the control device detects the object distance using the object distance measuring device in the state of the first optical axis before changing the optical axis, and uses the optical axis changing mechanism to detect the object distance in the state of the second optical axis. After changing the optical axis to , the object distance measurement device does not detect the object distance.
  • control device sets the optical axis change angle according to the subject distance detected by the subject distance measuring device.
  • a mobile object according to the present disclosure includes the imaging system according to any one of (1) to (20). Thereby, the imaging system can expand the imaging range while the mobile object is moving, and can capture images over a wide range.
  • the present disclosure is applicable to an imaging system installed on a moving body.
  • Imaging target area 11 Imaging device 12, 12B Optical axis changing mechanism 15 Control Device 17 Storage section 19 Operation section 21 Camera body 23 Lens 23a Optical axis 23ab, 23ad First optical axis 23ac, 23ae Second optical axis 24 Shutter 25 Image sensor 27 Camera control section 31 Shake correction mechanism 41, 41B Mirror 43 Mirror drive section 45 Arm 51 Maximum exposure time calculation section 53 Exposure time setting section 61 Base 63 Rotation drive section 71 Optical axis change instruction section 73 Correction mechanism swing angle calculation section 75 Correction mechanism rotation speed calculation section 81 Subject distance detection device ⁇ Mirror swing angle F Focal length C1 First state C2 Second state M Subject magnification LE1 Extension line of principal point of lens LE2 Extension line of imaging target surface ⁇ Optical axis change angle Tf Imaging interval V1, V2, V3 Travel speed

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Studio Devices (AREA)
PCT/JP2023/020299 2022-06-01 2023-05-31 撮像システム、及び、それを備えた移動体 Ceased WO2023234356A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024524914A JPWO2023234356A1 (https=) 2022-06-01 2023-05-31
US18/961,954 US20250097581A1 (en) 2022-06-01 2024-11-27 Imaging system and mobile object provided with same

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022089866 2022-06-01
JP2022-089866 2022-06-01

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/961,954 Continuation US20250097581A1 (en) 2022-06-01 2024-11-27 Imaging system and mobile object provided with same

Publications (1)

Publication Number Publication Date
WO2023234356A1 true WO2023234356A1 (ja) 2023-12-07

Family

ID=89024921

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/020299 Ceased WO2023234356A1 (ja) 2022-06-01 2023-05-31 撮像システム、及び、それを備えた移動体

Country Status (3)

Country Link
US (1) US20250097581A1 (https=)
JP (1) JPWO2023234356A1 (https=)
WO (1) WO2023234356A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011033569A1 (ja) * 2009-09-17 2011-03-24 富士通株式会社 画像処理装置及び画像処理方法
JP2011097639A (ja) * 2011-01-17 2011-05-12 Takenaka Komuten Co Ltd 画像検査方法および画像検査装置
JP2015195569A (ja) * 2014-03-25 2015-11-05 パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 移動体用撮影装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011033569A1 (ja) * 2009-09-17 2011-03-24 富士通株式会社 画像処理装置及び画像処理方法
JP2011097639A (ja) * 2011-01-17 2011-05-12 Takenaka Komuten Co Ltd 画像検査方法および画像検査装置
JP2015195569A (ja) * 2014-03-25 2015-11-05 パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 移動体用撮影装置

Also Published As

Publication number Publication date
JPWO2023234356A1 (https=) 2023-12-07
US20250097581A1 (en) 2025-03-20

Similar Documents

Publication Publication Date Title
KR101339193B1 (ko) 카메라 플랫폼 시스템
JP2015195569A (ja) 移動体用撮影装置
CN103676405A (zh) 光学成像装置、光学系统和移动终端
US12141947B2 (en) Image processing method, electronic device, and computer-readable storage medium
JP6528771B2 (ja) 撮影装置
JP2019080226A (ja) 撮像装置、撮像装置の制御方法、及びプログラム
JP2010250156A (ja) 電子カメラ
US12581190B2 (en) Imaging system and moving body provided with same
JP4134869B2 (ja) 撮像装置
WO2023234356A1 (ja) 撮像システム、及び、それを備えた移動体
US12432450B2 (en) Imaging system and moving body provided with same
JP4031646B2 (ja) 撮像装置
JP5277600B2 (ja) 架空線撮影システム及び方法
JPWO2023234356A5 (https=)
WO2023234360A1 (ja) 撮像システム、及び、それを備えた移動体
JPH11155096A (ja) 撮像装置、コンピュータ読み取り可能な記憶媒体及び画像処理装置
JPWO2023234360A5 (https=)
EP3930305B1 (en) Control apparatus, control method, and program
WO2026004723A1 (ja) 撮像システム
JP2019092036A (ja) 撮像装置及び制御方法
WO2023243656A1 (ja) 撮像システム、及び、それを備えた移動体
US20250350836A1 (en) Imaging system
JPWO2023243656A5 (https=)
JP2005175852A (ja) 撮影装置及び撮影装置の制御方法
KR100664811B1 (ko) 프라이버시 마스크 표시 제어 방법 및 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23816113

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024524914

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23816113

Country of ref document: EP

Kind code of ref document: A1