US11895403B2 - Imaging support device that performs registration control for setting a detected subject image position, imaging support system, imaging system, imaging support method, and program - Google Patents

Imaging support device that performs registration control for setting a detected subject image position, imaging support system, imaging system, imaging support method, and program Download PDF

Info

Publication number
US11895403B2
US11895403B2 US17/728,993 US202217728993A US11895403B2 US 11895403 B2 US11895403 B2 US 11895403B2 US 202217728993 A US202217728993 A US 202217728993A US 11895403 B2 US11895403 B2 US 11895403B2
Authority
US
United States
Prior art keywords
imaging
shake correction
focal length
lens
case
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US17/728,993
Other languages
English (en)
Other versions
US20220256086A1 (en
Inventor
Tetsuya Fujikawa
Shinichiro Fujiki
Junichi Tanaka
Atsushi KAWANAGO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Assigned to FUJIFILM CORPORATION reassignment FUJIFILM CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FUJIKI, SHINICHIRO, KAWANAGO, Atsushi, FUJIKAWA, TETSUYA, TANAKA, JUNICHI
Publication of US20220256086A1 publication Critical patent/US20220256086A1/en
Priority to US18/539,310 priority Critical patent/US20240114245A1/en
Application granted granted Critical
Publication of US11895403B2 publication Critical patent/US11895403B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation
    • H04N23/687—Vibration or motion blur correction performed by mechanical compensation by shifting the lens or sensor position
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B5/00—Adjustment of optical system relative to image or object surface other than for focusing
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/02—Mountings, adjusting means, or light-tight connections, for optical elements for lenses
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/02—Mountings, adjusting means, or light-tight connections, for optical elements for lenses
    • G02B7/04—Mountings, adjusting means, or light-tight connections, for optical elements for lenses with mechanism for focusing or varying magnification
    • G02B7/08—Mountings, adjusting means, or light-tight connections, for optical elements for lenses with mechanism for focusing or varying magnification adapted to co-operate with a remote control mechanism
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00—Special procedures for taking photographs; Apparatus therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • G03B17/56—Accessories
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/64—Imaging systems using optical elements for stabilisation of the lateral and angular position of the image
    • G02B27/646—Imaging systems using optical elements for stabilisation of the lateral and angular position of the image compensating for small deviations, e.g. due to vibration or shake
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B2205/00—Adjustment of optical system relative to image or object surface other than for focusing
    • G03B2205/0007—Movement of one or more optical elements for control of motion blur
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6811—Motion detection based on the image signal

Definitions

  • the disclosed technology relates to an imaging support device, an imaging support system, an imaging system, an imaging support method, and a program.
  • JP2016-90973A discloses an imaging apparatus comprising a first anti-vibration control portion that controls a first anti-vibration portion which corrects a camera shake by moving a lens, a second anti-vibration control portion that controls a second anti-vibration portion which corrects the camera shake by moving an imaging element, a wide angle image generation portion that generates a wide angle image based on a plurality of images which are sequentially captured within a moved imaging range by moving the imaging range for wide angle imaging by at least the first anti-vibration control portion or the second anti-vibration control portion, and a setting portion that performs setting of whether to use the first anti-vibration portion and the second anti-vibration portion for correcting the camera shake or for moving the imaging range.
  • One embodiment according to the disclosed technology provides an imaging support device, an imaging support system, an imaging system, an imaging support method, and a program that can set a subject image position to a specific position in a captured image with high accuracy compared to a case where a registration control is performed depending on only a lens moving mechanism and an image region movement portion regardless of a focal length.
  • a first aspect according to the disclosed technology is an imaging support device comprising an acquisition portion that acquires a focal length of an imaging apparatus which includes a lens and an imaging element and includes a lens moving mechanism which moves the lens in a direction intersecting with an optical axis of the lens, and an image region movement portion which moves a region of a captured image in the imaging element in the direction intersecting with the optical axis, a detection portion that detects a subject image position of a target subject image showing a target subject in the captured image obtained by capturing an imaging region including the target subject by the imaging apparatus, and a control portion that performs a registration control of setting the subject image position detected by the detection portion to a specific position in the captured image by increasing a ratio of application of the image region movement portion compared to the lens moving mechanism in a case where the focal length acquired by the acquisition portion is greater than or equal to a threshold value, and increasing a ratio of application of the lens moving mechanism compared to the image region movement portion in a case where the focal length is less than the threshold value.
  • a second aspect according to the disclosed technology is the imaging support device according to the first aspect, in which the lens moving mechanism is a lens side shake correction mechanism that corrects a shake occurring due to a vibration exerted on the imaging apparatus, by moving the lens, the image region movement portion is an imaging element side shake correction portion that corrects the shake by moving the imaging element or performing image processing on the captured image obtained by imaging performed by the imaging element, and the control portion performs the registration control by increasing a ratio of application of the imaging element side shake correction portion compared to the lens side shake correction mechanism in a case where the focal length is greater than or equal to the threshold value, and increasing a ratio of application of the lens side shake correction mechanism compared to the imaging element side shake correction portion in a case where the focal length is less than the threshold value.
  • the lens moving mechanism is a lens side shake correction mechanism that corrects a shake occurring due to a vibration exerted on the imaging apparatus, by moving the lens
  • the image region movement portion is an imaging element side shake correction portion that corrects the shake by moving the imaging element or performing image
  • a third aspect according to the disclosed technology is the imaging support device according to the second aspect, in which the control portion causes the lens side shake correction mechanism to correct the shake in a case where the focal length acquired by the acquisition portion is greater than or equal to the threshold value.
  • a fourth aspect according to the disclosed technology is the imaging support device according to the second or third aspect, in which the control portion causes the imaging element side shake correction portion to correct the shake in a case where the focal length acquired by the acquisition portion is less than the threshold value.
  • a fifth aspect according to the disclosed technology is the imaging support device according to any one of the first to fourth aspects, in which the imaging apparatus includes a variable magnification mechanism that enables the focal length to change, and the control portion performs the registration control using the imaging element side shake correction portion and the lens side shake correction mechanism together in a case where the focal length acquired by the acquisition portion is changing.
  • a sixth aspect according to the disclosed technology is the imaging support device according to any one of the first to fifth aspects, in which the control portion causes at least one of the imaging element side shake correction portion or the lens side shake correction mechanism to correct the shake in a case where the focal length acquired by the acquisition portion is not changing.
  • a seventh aspect according to the disclosed technology is the imaging support device according to any one of the first to sixth aspects, in which the control portion performs the registration control using the lens side shake correction mechanism and the imaging element side shake correction portion together in a case where the focal length acquired by the acquisition portion is within a predetermined range including the threshold value.
  • An eighth aspect according to the disclosed technology is the imaging support device according to the seventh aspect, in which the registration control is a control including a control of gradually increasing the ratio of application of the imaging element side shake correction portion compared to the lens side shake correction mechanism in accordance with increasing of the focal length in a case where the focal length acquired by the acquisition portion is within the predetermined range.
  • a ninth aspect according to the disclosed technology is the imaging support device according to the seventh or eighth aspect, in which the control portion performs the registration control using the lens side shake correction mechanism out of the lens side shake correction mechanism and the imaging element side shake correction portion in a case where the focal length acquired by the acquisition portion is less than a lower limit value of the predetermined range.
  • a tenth aspect according to the disclosed technology is the imaging support device according to any one of the seventh to ninth aspects, in which the control portion performs the registration control using the imaging element side shake correction portion out of the lens side shake correction mechanism and the imaging element side shake correction portion in a case where the focal length acquired by the acquisition portion exceeds an upper limit value of the predetermined range.
  • An eleventh aspect according to the disclosed technology is the imaging support device according to any one of the first to tenth aspects, in which the control portion performs the registration control using a registration portion including a revolution mechanism that enables the imaging apparatus to revolve, the imaging element side shake correction portion, and the lens side shake correction mechanism, and registration accuracy for adjusting the subject image position in the captured image is higher for the imaging element side shake correction portion and the lens side shake correction mechanism than for the revolution mechanism.
  • a twelfth aspect according to the disclosed technology is an imaging support system comprising the imaging support device according to the eleventh aspect, and the revolution mechanism, in which the imaging support device supports imaging performed by the imaging apparatus in a case where the revolution mechanism causes the imaging apparatus to revolve.
  • a thirteenth aspect according to the disclosed technology is an imaging system comprising the imaging support device according to any one of the first to eleventh aspects, and the imaging apparatus, in which the imaging support device supports imaging for the imaging apparatus.
  • a fourteenth aspect according to the disclosed technology is the imaging system according to the thirteenth aspect, further comprising a revolution mechanism that causes the imaging apparatus to revolve.
  • a fifteenth aspect according to the disclosed technology is an imaging support method comprising acquiring a focal length of an imaging apparatus which includes a lens and an imaging element and includes a lens moving mechanism which moves the lens in a direction intersecting with an optical axis of the lens, and an image region movement portion which moves a region of a captured image in the imaging element in the direction intersecting with the optical axis, detecting a subject image position of a target subject image showing a target subject in the captured image obtained by capturing an imaging region including the target subject by the imaging apparatus, and performing a registration control of setting the detected subject image position to a specific position in the captured image by increasing a ratio of application of the image region movement portion compared to the lens moving mechanism in a case where the acquired focal length is greater than or equal to a threshold value, and increasing a ratio of application of the lens moving mechanism compared to the image region movement portion in a case where the focal length is less than the threshold value.
  • a sixteenth aspect according to the disclosed technology is a program causing a computer to execute a process comprising acquiring a focal length of an imaging apparatus which includes a lens and an imaging element and includes a lens moving mechanism which moves the lens in a direction intersecting with an optical axis of the lens, and an image region movement portion which moves a region of a captured image in the imaging element in the direction intersecting with the optical axis, detecting a subject image position of a target subject image showing a target subject in the captured image obtained by capturing an imaging region including the target subject by the imaging apparatus, and performing a registration control of setting the detected subject image position to a specific position in the captured image by increasing a ratio of application of the image region movement portion compared to the lens moving mechanism in a case where the acquired focal length is greater than or equal to a threshold value, and increasing a ratio of application of the lens moving mechanism compared to the image region movement portion in a case where the focal length is less than the threshold value.
  • FIG. 1 is a schematic configuration diagram illustrating an example of a configuration of a surveillance system according to a first embodiment
  • FIG. 2 is a perspective view illustrating an example of an exterior of a surveillance camera in a state where the surveillance camera revolves in a pitch direction according to the first embodiment
  • FIG. 3 is a perspective view illustrating an example of the exterior of the surveillance camera in a state where the surveillance camera revolves in a yaw direction according to the first embodiment
  • FIG. 4 is a block diagram illustrating an example of a configuration of an optical system and an electric system of the surveillance camera according to the first embodiment
  • FIG. 5 is a block diagram illustrating an example of a configuration of an electric system of a management apparatus and a revolution mechanism according to the first embodiment
  • FIG. 6 is a function block diagram illustrating an example of functions of a CPU included in the surveillance camera according to the first embodiment
  • FIG. 7 is a function block diagram illustrating an example of functions of a CPU included in the management apparatus according to the first embodiment
  • FIG. 8 is a function block diagram illustrating an example of the functions of the CPU included in the management apparatus according to the first embodiment
  • FIG. 9 is a function block diagram illustrating an example of functions of the management apparatus in a case where the management apparatus uses a ratio derivation table according to the first embodiment
  • FIG. 10 A is a flowchart illustrating an example of a flow of registration control processing according to the first embodiment
  • FIG. 10 B is a continuation of the flowchart illustrated in FIG. 10 A ;
  • FIG. 11 A is a conceptual diagram illustrating an example of an aspect in which a target subject is present in a center portion of an imaging region according to the first embodiment
  • FIG. 11 B is a schematic image diagram illustrating an example of a captured image obtained by capturing the imaging region illustrated in FIG. 11 A by the surveillance camera;
  • FIG. 12 A is a conceptual diagram illustrating an example of an aspect in which the target subject is present in an end portion of the imaging region according to the first embodiment
  • FIG. 12 B is a schematic image diagram illustrating an example of the captured image obtained by capturing the imaging region illustrated in FIG. 12 A by the surveillance camera;
  • FIG. 13 A is a conceptual diagram illustrating an example of an aspect in which the target subject image deviates from the imaging region in accordance with increasing of a focal length of the surveillance camera according to the first embodiment
  • FIG. 13 B is a schematic image diagram illustrating an example of the captured image obtained by capturing the imaging region illustrated in FIG. 13 A by the surveillance camera;
  • FIG. 14 is a schematic image diagram illustrating an example of the captured image in which a target subject image is caused to fall within a center region of the captured image by operating a shake correction portion according to the first embodiment
  • FIG. 15 is a function block diagram illustrating an example of functions of a CPU included in a management apparatus according to a second embodiment
  • FIG. 16 A is a flowchart illustrating an example of a flow of registration control processing according to the second embodiment
  • FIG. 16 B is a continuation of the flowchart illustrated in FIG. 16 A ;
  • FIG. 17 is a schematic image diagram illustrating an example of the captured image obtained by performing coarse adjustment and the captured image obtained by performing fine adjustment after the coarse adjustment in the registration control processing;
  • FIG. 18 is a graph illustrating an example of the ratio derivation table different from
  • FIG. 9 for an application ratio of a lens side shake correction mechanism and an imaging element side shake correction mechanism in an imaging support device.
  • FIG. 19 is a conceptual diagram illustrating an example of an aspect in which a display control processing program and a variable magnification mechanism control processing program are installed on a computer in the surveillance camera from a storage medium storing a registration control processing program, the display control processing program, and the variable magnification mechanism control processing program according to the embodiments.
  • CPU is an abbreviation for “Central Processing Unit”.
  • RAM is an abbreviation for “Random Access Memory”.
  • ROM is an abbreviation for “Read Only Memory”.
  • ASIC is an abbreviation for “Application Specific Integrated Circuit”.
  • PLD is an abbreviation for “Programmable Logic Device”.
  • FPGA is an abbreviation for “Field-Programmable Gate Array”.
  • AFE is an abbreviation for “Analog Front End”.
  • DSP is an abbreviation for “Digital Signal Processor”.
  • ISP is an abbreviation for “Image Signal Processor”.
  • SoC is an abbreviation for “System-on-a-chip”.
  • CMOS is an abbreviation for “Complementary Metal Oxide Semiconductor”.
  • CCD is an abbreviation for “Charge Coupled Device”.
  • SWIR is an abbreviation for “Short-Wavelength Infrared”.
  • SSD is an abbreviation for “Solid State Drive”.
  • USB is an abbreviation for “Universal Serial Bus”.
  • HDD is an abbreviation for “Hard Disk Drive”.
  • EEPROM is an abbreviation for “Electrically Erasable and Programmable Read Only Memory”.
  • EL is an abbreviation for “Electro-Luminescence”.
  • A/D is an abbreviation for “Analog/Digital”.
  • I/F is an abbreviation for “Interface”.
  • UI is an abbreviation for “User Interface”.
  • WAN is an abbreviation for “Wide Area Network”.
  • CRT is an abbreviation for “Cathode Ray Tube”.
  • OIS is an abbreviation for “Optical Image Stabilizer”.
  • BIS is an abbreviation for “Body Image Stabilizer”.
  • “horizontal” refers to being horizontal in a sense of not only being completely horizontal but also including an error generally allowed in the technical field to which the disclosed technology belongs.
  • “parallel” refers to being parallel in a sense of not only being completely parallel but also including an error generally allowed in the technical field to which the disclosed technology belongs.
  • “perpendicular” refers to being perpendicular in a sense of not only being completely perpendicular but also including an error generally allowed in the technical field to which the disclosed technology belongs.
  • “same” refers to being the same in a sense of not only being completely the same but also including an error generally allowed in the technical field to which the disclosed technology belongs.
  • a surveillance system 2 comprises a surveillance camera 10 , a revolution mechanism 16 , and a management apparatus 11 .
  • the surveillance system 2 is an example of an “imaging system” or an “imaging support system” according to the embodiment of the disclosed technology.
  • the surveillance camera 10 is an example of an “imaging apparatus” according to the embodiment of the disclosed technology.
  • the surveillance camera 10 is installed in an indoor or outdoor post or wall, a part (for example, a rooftop) of a building, or the like through a revolution mechanism 16 , described later, images a surveillance target (hereinafter, referred to as an “imaging region”) that is a subject, and generates a motion picture image by the imaging.
  • the motion picture image includes images of a plurality of frames obtained by imaging.
  • the surveillance camera 10 transmits the motion picture image obtained by imaging to the management apparatus 11 through a communication line 12 .
  • the management apparatus 11 comprises a display 13 and a secondary storage device 14 .
  • Examples of the display 13 include a liquid crystal display, a plasma display, an organic EL display, and a CRT display.
  • Examples of the secondary storage device 14 include an HDD.
  • the secondary storage device 14 may be a non-volatile memory such as a flash memory, an SSD, or an EEPROM instead of the HDD.
  • the motion picture image transmitted by the surveillance camera 10 is received, and the received motion picture image is displayed on the display 13 or stored in the secondary storage device 14 .
  • the surveillance camera 10 is attached to the revolution mechanism 16 .
  • the revolution mechanism 16 enables the surveillance camera 10 to revolve.
  • the revolution mechanism 16 is a 2-axis revolution mechanism that enables the surveillance camera 10 to revolve in a revolution direction (hereinafter, referred to as a “pitch direction”) that intersects with a yaw direction and is about a pitch axis PA as a central axis as illustrated in FIG. 2 as an example, and a revolution direction (hereinafter, referred to as the “yaw direction”) about a yaw axis YA as a central axis as illustrated in FIG. 3 as an example.
  • the revolution mechanism 16 is an example of a “revolution mechanism” according to the embodiment of the disclosed technology. While an example of the 2-axis revolution mechanism is illustrated for the revolution mechanism 16 according to the present embodiment, the disclosed technology is not limited thereto, and a 3-axis revolution mechanism may be used.
  • the surveillance camera 10 comprises an optical system 15 and an imaging element 25 .
  • the imaging element 25 is positioned on a rear stage of the optical system 15 .
  • the imaging element 25 comprises a light-receiving surface 25 A.
  • An image of light showing the imaging region is formed on the light-receiving surface 25 A of the imaging element 25 by the optical system 15 , and the imaging region is captured by the imaging element 25 .
  • the surveillance camera 10 comprises a computer 19 , a displacement driver 22 , a displacement driver 23 , a zoom driver 28 , a DSP 31 , an image memory 32 , an electronic shake correction portion 33 , a communication I/F 34 , position detection sensors 39 and 47 , a shake amount detection sensor 40 , and a UI system device 43 .
  • the computer 19 comprises a memory 35 , a storage 36 , and a CPU 37 .
  • the computer 19 is an example of a “computer” according to the embodiment of the disclosed technology.
  • the electronic shake correction portion 33 and the CPU 37 are an example of an “electronic shake correction portion (electronic shake correction component)” according to the embodiment of the disclosed technology.
  • the displacement driver 22 , the displacement driver 23 , the imaging element 25 , the zoom driver 28 , the DSP 31 , the image memory 32 , the electronic shake correction portion 33 , the communication I/F 34 , the memory 35 , the storage 36 , the CPU 37 , the position detection sensors 39 and 47 , the shake amount detection sensor 40 , and the UI system device 43 are connected to a bus 38 .
  • a bus 38 In the example illustrated in FIG. 4 , while one bus is illustrated as the bus 38 for convenience of illustration, a plurality of buses may be used.
  • the bus 38 may be a serial bus or may be a parallel bus including a data bus, an address bus, a control bus, and the like.
  • the memory 35 temporarily stores various information and is used as a work memory.
  • Examples of the memory 35 include a RAM. However, the disclosed technology is not limited thereto. A storage device of other types may be used.
  • the storage 36 stores various programs for the surveillance camera 10 .
  • the CPU 37 controls the entire surveillance camera 10 by reading out various programs from the storage 36 and executing the read various programs on the memory 35 .
  • Examples of the storage 36 include a flash memory, an SSD, an EEPROM, or an HDD.
  • various non-volatile memories such as a magnetoresistive memory and a ferroelectric memory may be used instead of the flash memory or together with the flash memory.
  • the imaging element 25 is a CMOS image sensor.
  • the imaging element 25 images the target subject at a predetermined frame rate under an instruction of the CPU 37 .
  • the “predetermined frame rate” refers to a few tens of frames/second to a few hundred frames/second.
  • the imaging element 25 may incorporate a control device (imaging element control device).
  • the imaging element control device performs detailed controls inside imaging element 25 in accordance with an imaging instruction output by the CPU 37 .
  • the imaging element 25 may image a target subject at the predetermined frame rate under an instruction of the DSP 31 .
  • the imaging element control device performs the detailed controls inside the imaging element 25 in accordance with the imaging instruction output by the DSP 31 .
  • the DSP 31 may be referred to as an ISP.
  • the light-receiving surface 25 A of the imaging element 25 is formed with a plurality of photosensitive pixels (not illustrated) arranged in a matrix.
  • photoelectric conversion is performed for each photosensitive pixel by exposing each photosensitive pixel. Charges obtained by performing the photoelectric conversion for each photosensitive pixel correspond to an analog imaging signal indicating the target subject.
  • a plurality of photoelectric conversion elements for example, photoelectric conversion elements in which color filters are arranged
  • having sensitivity to visible light are employed as the plurality of photosensitive pixels.
  • a photoelectric conversion element having sensitivity to light of red (R) for example, a photoelectric conversion element in which an R filter corresponding to R is arranged
  • a photoelectric conversion element having sensitivity to light of green (G) for example, a photoelectric conversion element in which a G filter corresponding to G is arranged
  • a photoelectric conversion element having sensitivity to light of blue (B) for example, a photoelectric conversion element in which a B filter corresponding to B is arranged
  • Imaging based on infrared light may be performed.
  • a plurality of photoelectric conversion elements having sensitivity to the infrared light may be used as the plurality of photosensitive pixels.
  • an InGaAs sensor and/or a type-2 quantum well (T2SL; Simulation of Type-II Quantum Well) sensor may be used for imaging for SWIR.
  • the imaging element 25 generates a digital image that is a digital imaging signal by performing signal processing such as A/D conversion on the analog imaging signal.
  • the imaging element 25 is connected to the DSP 31 through the bus 38 and outputs the generated digital image to the DSP 31 in units of frames through the bus 38 .
  • the digital image is an example of a “captured image” according to the embodiment of the disclosed technology.
  • a CCD image sensor may be applied as the imaging element 25 .
  • the imaging element 25 is connected to the bus 38 through an AFE (not illustrated) that incorporates a CCD driver.
  • the AFE generates the digital image by performing the signal processing such as the A/D conversion on the analog imaging signal obtained by the imaging element 25 and outputs the generated digital image to the DSP 31 .
  • the CCD image sensor is driven by the CCD driver incorporated in the AFE.
  • the CCD driver may be independently provided.
  • the DSP 31 performs various digital signal processing on the digital image.
  • the various digital signal processing refers to demosaicing, noise removal processing, gradation correction processing, and color correction processing.
  • the DSP 31 outputs the digital image after the digital signal processing to the image memory 32 for each frame.
  • the image memory 32 stores the digital image from the DSP 31 .
  • the digital image stored in the image memory 32 will be referred to as the “captured image”.
  • the shake amount detection sensor 40 is, for example, a device including a gyro sensor and detects the shake amount of the surveillance camera 10 . In other words, the shake amount detection sensor 40 detects the shake amount for each of a pair of axial directions.
  • the gyro sensor detects an amount of a rotational shake about each axis (refer to FIG. 1 ) of the pitch axis PA, the yaw axis YA, and a roll axis RA (axis parallel to the optical axis OA).
  • the shake amount detection sensor 40 detects the shake amount of the surveillance camera 10 by converting the amount of the rotational shake about the pitch axis PA and the amount of the rotational shake about the yaw axis YA detected by the gyro sensor into a shake amount in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA.
  • the shake amount detection sensor 40 may be an acceleration sensor.
  • the acceleration sensor detects the shake amount in the two-dimensional plane parallel to the pitch axis PA and the yaw axis YA.
  • the shake amount detection sensor 40 outputs the detected shake amount to the CPU 37 .
  • the disclosed technology is not limited thereto.
  • a movement vector obtained by comparing captured images that are stored in the image memory 32 and are adjacent to each other in time series may be used as the shake amount.
  • a finally used shake amount may be derived based on the shake amount detected by the physical sensor and the movement vector obtained by the image processing.
  • the CPU 37 acquires the shake amount detected by the shake amount detection sensor 40 and controls the lens side shake correction mechanism 29 , the imaging element side shake correction mechanism 45 , and the electronic shake correction portion 33 based on the acquired shake amount.
  • the shake amount detected by the shake amount detection sensor 40 is used for correction of the shake by each of the lens side shake correction mechanism 29 and the electronic shake correction portion 33 .
  • the electronic shake correction portion 33 is a device including an ASIC.
  • the electronic shake correction portion 33 corrects the shake by performing the image processing on the captured image obtained by imaging performed by the imaging element 25 in the image memory 32 based on the shake amount detected by the shake amount detection sensor 40 .
  • the disclosed technology is not limited thereto.
  • the shake may be corrected by the electronic shake correction portion 33 in accordance with a movement vector calculated based on a difference between the frames of the captured image.
  • the disclosed technology is not limited thereto.
  • a device including an FPGA or a PLD may be used.
  • the electronic shake correction portion 33 may be a device including a plurality out of the ASIC, the FPGA, and the PLD.
  • a computer including a CPU, a storage, and a memory may be employed as the electronic shake correction portion 33 .
  • the number of CPUs may be singular or plural.
  • the electronic shake correction portion 33 may be implemented by a combination of a hardware configuration and a software configuration.
  • the communication I/F 34 is, for example, a network interface and controls transfer of various information with respect to the management apparatus 11 through a network.
  • Examples of the network include a WAN such as the Internet or a public communication network. Communication between the surveillance camera 10 and the management apparatus 11 is controlled.
  • the UI system device 43 comprises a reception device 43 A and a display 43 B.
  • the reception device 43 A includes, for example, a hard key and a touch panel and receives various instructions from the user.
  • the CPU 37 acquires the various instructions received by the reception device 43 A and operates in accordance with the acquired instructions.
  • the display 43 B displays various information under control of the CPU 37 .
  • Examples of the various information displayed on the display 43 B include contents of the various instructions received by the reception device 43 A and the captured image.
  • the optical system 15 comprises an objective lens 15 A and a lens group 15 B.
  • the objective lens 15 A and the lens group 15 B are arranged in an order of the objective lens 15 A and the lens group 15 B along the optical axis OA of the optical system 15 from the target subject side to the light-receiving surface 25 A side of the imaging element 25 .
  • the lens group 15 B includes an anti-vibration lens 15 B 1 , a focus lens (not illustrated), a zoom lens 15 B 2 , and the like.
  • Various lenses (not illustrated) may be comprised as the optical system 15 in addition to the objective lens 15 A and the lens group 15 B.
  • the optical system 15 may comprise a stop. Positions of the lenses, the lens group, and the stop included in the optical system 15 are not limited. For example, the disclosed technology is also established for positions different from the positions illustrated in FIG. 4 .
  • the surveillance camera 10 comprises actuators 17 and 21 and an actuator 27 .
  • the zoom lens 15 B 2 is movably supported along the optical axis OA by the actuator 21 , described later.
  • the anti-vibration lens 15 B 1 is movably supported in a direction perpendicular to the optical axis OA by the actuator 17 .
  • the imaging element 25 is movably supported in the direction perpendicular to the optical axis OA by the actuator 27 .
  • the actuator 17 applies motive power to the anti-vibration lens 15 B 1 to change in a direction perpendicular to an optical axis of the anti-vibration lens 15 B 1 .
  • the actuator 17 is controlled by the CPU 37 through the displacement driver 23 .
  • the displacement driver 23 changes the position of the anti-vibration lens 15 B 1 in the direction perpendicular to the optical axis OA by operating the actuator 17 in accordance with an instruction from the CPU 37 .
  • the actuator 21 applies motive power to the zoom lens 15 B 2 to move along the optical axis OA of the optical system 15 .
  • the actuator 21 is controlled by the CPU 37 through the zoom driver 28 .
  • the zoom driver 28 moves the position of the zoom lens 15 B 2 along the optical axis OA by operating the actuator 21 in accordance with the instruction from the CPU 37 .
  • the actuator 27 applies motive power to the imaging element 25 to change the imaging element 25 in the direction perpendicular to the optical axis of the anti-vibration lens 15 B 1 .
  • the actuator 27 is controlled by the CPU 37 through the displacement driver 22 .
  • the displacement driver 22 changes a position of the imaging element 25 in the direction perpendicular to the optical axis OA by operating the actuator 27 in accordance with the instruction from the CPU 37 .
  • the surveillance camera 10 includes a variable magnification mechanism 18 .
  • the variable magnification mechanism 18 is a mechanism that comprises the zoom lens 15 B 2 , the actuator 21 , and the zoom driver 28 and can change a focal length.
  • the CPU 37 changes the focal length by operating the actuator 21 through the zoom driver 28 .
  • increasing the focal length from the current focal length positions the zoom lens 15 B 2 on a telephoto side from the current position.
  • Narrowing of the angle of view means narrowing of an imaging range.
  • decreasing the focal length from the current focal length positions the zoom lens 15 B 2 on a wide angle side from the current position.
  • the angle of view is widened. Widening of the angle of view means widening of an imaging range.
  • the diagram of the variable magnification mechanism 18 illustrated in the embodiment of the disclosed technology is merely a conceptual diagram.
  • the variable magnification mechanism 18 may employ various configurations.
  • Examples of a vibration exerted on the surveillance camera 10 include, in a case of an outdoor space, a vibration caused by traffic of an automobile, a vibration caused by wind, a vibration caused by construction work, and the like and, in a case of an indoor space, a vibration caused by an operation of an air conditioner, a vibration caused by entrance and exit of a person, and the like.
  • a shake occurs due to the vibration exerted on the surveillance camera 10 (hereinafter, simply referred to as the “vibration”).
  • the shake that occurs due to the vibration will be simply referred to as the “shake”.
  • the “shake” refers to a phenomenon in which a subject image on the light-receiving surface 25 A of the imaging element 25 changes due to a change in positional relationship between the optical axis OA and the light-receiving surface 25 A in the surveillance camera 10 .
  • the “shake” is said to be a phenomenon in which an optical image obtained by forming the image on the light-receiving surface 25 A changes by inclination of the optical axis OA due to the vibration exerted on the surveillance camera 10 .
  • changing of the optical axis OA means inclination of the optical axis OA with respect to a reference axis (for example, the optical axis OA before the shake occurs).
  • the shake is included in the captured image as a noise component and affects image quality of the captured image. Therefore, in order to remove the noise component included in the captured image due to the shake, the surveillance camera 10 comprises a shake correction portion 24 .
  • the shake correction portion 24 is an example of a “registration portion (registration component)” according to the embodiment of the disclosed technology.
  • the shake correction portion 24 includes the lens side shake correction mechanism 29 , the imaging element side shake correction mechanism 45 , and the electronic shake correction portion 33 . All of the lens side shake correction mechanism 29 , the imaging element side shake correction mechanism 45 , and the electronic shake correction portion 33 are used for correcting the shake.
  • the lens side shake correction mechanism 29 is a mechanism that corrects the shake by applying motive power generated by a driving source such as a motor (for example, a voice coil motor) to the anti-vibration lens to move the anti-vibration lens in the direction perpendicular to the optical axis of the imaging optical system.
  • the imaging element side shake correction mechanism 45 is a mechanism that corrects the shake by applying motive power generated by a driving source such as a motor (for example, a voice coil motor) to the imaging element to move the imaging element in the direction perpendicular to the optical axis of the imaging optical system.
  • the electronic shake correction portion 33 corrects the shake by performing image processing on a captured image based on a shake amount.
  • the shake correction portion 24 mechanically or electronically corrects the shake using a hardware configuration and/or a software configuration.
  • mechanical correction of the shake refers to correction of the shake implemented by mechanically moving a shake correction element such as an anti-vibration lens and/or an imaging element using motive power generated by a driving source such as a motor (for example, a voice coil motor).
  • Electronic correction of the shake refers to correction of the shake implemented by performing the image processing by a processor.
  • “correction of the shake” includes a meaning of removing the shake and also a meaning of reducing the shake.
  • the lens side shake correction mechanism 29 is a mechanism that moves the anti-vibration lens 15 B 1 in the direction perpendicular to the optical axis OA, and is an example of a “lens moving mechanism” according to the embodiment of the disclosed technology.
  • the imaging element side shake correction mechanism 45 and the electronic shake correction portion 33 are mechanisms that move the region of the captured image in the imaging element 25 in the direction perpendicular to the optical axis OA.
  • the imaging element side shake correction mechanism 45 and the electronic shake correction portion 33 are an example of an “image region movement portion (image region movement component)” and an “imaging element side shake correction portion (imaging element side shake correction component)” according to the embodiment of the disclosed technology.
  • the lens side shake correction mechanism 29 comprises the anti-vibration lens 15 B 1 , the actuator 17 , the displacement driver 23 , and the position detection sensor 39 .
  • a method of correcting the shake by the lens side shake correction mechanism 29 can be employed as a method of correcting the shake by the lens side shake correction mechanism 29 .
  • a method of correcting the shake by moving the anti-vibration lens 15 B 1 based on the shake amount detected by the shake amount detection sensor 40 is employed as the method of correcting the shake.
  • the shake is corrected by moving the anti-vibration lens 15 B 1 in a direction of canceling the shake by an amount with which the shake is canceled.
  • the actuator 17 is attached to the anti-vibration lens 15 B 1 .
  • the actuator 17 is a shift mechanism in which a voice coil motor is mounted, and changes the anti-vibration lens 15 B 1 in the direction perpendicular to the optical axis of the anti-vibration lens 15 B 1 by driving the voice coil motor.
  • the shift mechanism in which the voice coil motor is mounted is employed as the actuator 17
  • the disclosed technology is not limited thereto.
  • Other motive power sources such as a stepping motor or a piezo element may be applied instead of the voice coil motor.
  • the actuator 17 is controlled by the displacement driver 23 .
  • Driving the actuator 17 under control of the displacement driver 23 mechanically changes the position of the anti-vibration lens 15 B 1 in a two-dimensional plane perpendicular to the optical axis OA.
  • the position detection sensor 39 detects the current position of the anti-vibration lens 15 B 1 and outputs a position signal indicating the detected current position.
  • a device including a hall element is employed as an example of the position detection sensor 39 .
  • the current position of the anti-vibration lens 15 B 1 refers to the current position in a two-dimensional plane of the anti-vibration lens.
  • the two-dimensional plane of the anti-vibration lens refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15 B 1 .
  • the device including the hall element is employed as an example of the position detection sensor 39
  • the disclosed technology is not limited thereto.
  • a magnetic sensor, a photosensor, or the like may be employed instead of the hall element.
  • the lens side shake correction mechanism 29 corrects the shake by moving the anti-vibration lens 15 B 1 along at least one of a direction of the pitch axis PA or a direction of the yaw axis YA. That is, the lens side shake correction mechanism 29 corrects the shake by moving the anti-vibration lens 15 B 1 by a movement amount corresponding to the shake amount detected by the shake amount detection sensor 40 in the two-dimensional plane of the anti-vibration lens.
  • the imaging element side shake correction mechanism 45 comprises the imaging element 25 , the displacement driver 22 , the actuator 27 , and a position detection sensor 47 .
  • various well-known methods can be employed as a method of correcting the shake by the imaging element side shake correction mechanism 45 .
  • a method of correcting the shake by moving the imaging element 25 by the displacement driver 22 through the actuator 27 in accordance with the instruction from the CPU 37 based on the shake amount detected by the shake amount detection sensor 40 is employed as the method of correcting the shake.
  • the shake is corrected by moving the imaging element 25 in the direction of canceling the shake by the amount with which the shake is canceled.
  • the actuator 27 is a shift mechanism in which a voice coil motor is mounted, and changes the imaging element 25 in the direction perpendicular to the optical axis of the anti-vibration lens 15 B 1 by driving the voice coil motor.
  • the shift mechanism in which the voice coil motor is mounted is employed as the actuator 27
  • the disclosed technology is not limited thereto.
  • Other motive power sources such as a stepping motor or a piezo element may be applied instead of the voice coil motor.
  • the position detection sensor 47 detects the current position of the imaging element 25 and outputs a position signal indicating the detected current position.
  • a device including a hall element is employed as an example of the position detection sensor 47 .
  • the current position of the imaging element 25 refers to the current position in a two-dimensional plane of the imaging element.
  • the two-dimensional plane of the imaging element refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15 B 1 .
  • the device including the hall element is employed as an example of the position detection sensor 47
  • the disclosed technology is not limited thereto.
  • a magnetic sensor, a photosensor, or the like may be employed instead of the hall element.
  • the management apparatus 11 comprises the display 13 , a control device 60 , a reception device 62 , and a communication I/F 66 .
  • the revolution mechanism 16 comprises a yaw axis revolution mechanism 71 , a pitch axis revolution mechanism 72 , a motor 73 , a motor 74 , a driver 75 , and a driver 76 .
  • the control device 60 comprises a CPU 60 A, a storage 60 B, and a memory 60 C.
  • Each of the reception device 62 , the display 13 , the secondary storage device 14 , the CPU 60 A, the storage 60 B, the memory 60 C, and the communication I/F 66 is connected to a bus 70 .
  • a bus 70 In the example illustrated in FIG. 5 , while one bus is illustrated as the bus 70 for convenience of illustration, a plurality of buses may be used.
  • the bus 70 may be a serial bus or may be a parallel bus including a data bus, an address bus, a control bus, and the like.
  • the memory 60 C temporarily stores various information and is used as a work memory. Examples of the memory 60 C include a RAM. However, the disclosed technology is not limited thereto. A storage device of other types may be used.
  • the storage 60 B stores various programs for the management apparatus 11 (hereinafter, simply referred to as a “management apparatus program”).
  • the CPU 60 A controls the entire management apparatus 11 by reading out the management apparatus program from the storage 60 B and executing the read management apparatus program on the memory 60 C.
  • the communication I/F 66 is, for example, a network interface.
  • the communication I/F 66 is communicably connected to the communication I/F 34 of the surveillance camera 10 through a network and controls transfer of various information with respect to the surveillance camera 10 .
  • the communication I/F 66 requests the surveillance camera 10 to transmit the captured image and receives the captured image transmitted from the communication I/F 34 of the surveillance camera 10 in response to the request for transmission of the captured image.
  • Communication I/Fs 67 and 68 are, for example, network interfaces.
  • the communication I/F 67 is communicably connected to the driver 75 of the revolution mechanism 16 through a network.
  • the CPU 60 A controls a revolution operation of the yaw axis revolution mechanism 71 by controlling the motor 73 through the communication I/F 67 and the driver 75 .
  • the communication I/F 68 is communicably connected to the driver 76 of the revolution mechanism 16 through a network.
  • the CPU 60 A controls a revolution operation of the pitch axis revolution mechanism 72 by controlling the motor 74 through the communication I/F 68 and the driver 76 .
  • the reception device 62 includes, for example, a keyboard, a mouse, and a touch panel and receives various instructions from the user.
  • the CPU 60 A acquires the various instructions received by the reception device 62 and operates in accordance with the acquired instructions. For example, in a case where a processing content for the surveillance camera 10 and/or the revolution mechanism 16 is received by the reception device 62 , the CPU 60 A operates the surveillance camera 10 and/or the revolution mechanism 16 in accordance with an instruction content received by the reception device 62 .
  • the display 13 displays various information under control of the CPU 60 A. Examples of the various information displayed on the display 13 include contents of the various instructions received by the reception device 62 and the captured image received by the communication I/F 66 .
  • the management apparatus 11 comprises the secondary storage device 14 .
  • the secondary storage device 14 is a non-volatile memory and stores various information under control of the CPU 60 A. Examples of the various information stored in the secondary storage device 14 include the captured image received by the communication I/F 66 .
  • control device 60 performs a control of displaying the captured image received by the communication I/F 66 on the display 13 and a control of storing the captured image received by the communication I/F 66 in the secondary storage device 14 .
  • control device 60 displays the captured image on the display 13 and stores the captured image received by the communication I/F 66 in the secondary storage device 14
  • the disclosed technology is not limited thereto.
  • any of the display of the captured image on the display 13 and the storage of the captured image in the secondary storage device 14 may be performed.
  • the motor 73 is driven to generate motive power under control of the driver 75 .
  • the yaw axis revolution mechanism 71 causes the surveillance camera 10 to revolve in the yaw direction by receiving the motive power generated by the motor 73 .
  • the motor 74 is driven to generate motive power under control of the driver 76 .
  • the pitch axis revolution mechanism 72 causes the surveillance camera 10 to revolve in the pitch direction by receiving the motive power generated by the motor 74 .
  • the storage 60 B of the management apparatus 11 stores a registration control processing program 36 A, a display control processing program 36 B, and a variable magnification mechanism control processing program 36 C.
  • the CPU 60 A operates as a variable magnification mechanism control portion 37 S by reading out the variable magnification mechanism control processing program 36 C from the storage 60 B and executing the read variable magnification mechanism control processing program 36 C on the memory 60 C (refer to FIG. 5 ).
  • the reception device 62 receives an instruction (hereinafter, referred to as a “magnification changing instruction”) to cause the variable magnification mechanism 18 to change magnification, and the variable magnification mechanism control portion 37 S controls the variable magnification mechanism 18 in accordance with the magnification changing instruction received by the reception device 62 .
  • the memory 35 of the surveillance camera 10 includes a focal length storage region 35 A. The most recent focal length is stored in an overwriting manner in the focal length storage region 35 A.
  • the CPU 60 A operates as a display control portion 37 T by reading out the display control processing program 36 B from the storage 60 B and executing the read display control processing program 36 B on the memory 60 C.
  • the display control portion 37 T reads out the captured image from the image memory 32 of the surveillance camera 10 and displays the read captured image on the display 13 as a live view image.
  • the live view image is illustrated as the image displayed on the display 13 , the disclosed technology is not limited thereto.
  • the image displayed on the display 13 may be a motion picture image for recording or a still picture image for recording.
  • the CPU 60 A is an example of a “processor” according to the embodiment of the disclosed technology.
  • the memory 60 C is an example of a “memory” according to the embodiment of the disclosed technology.
  • the CPU 60 A reads out the registration control processing program 36 A from the storage 60 B and executes the read registration control processing program 36 A on the memory 60 C.
  • the CPU 60 A operates as an acquisition portion 37 P, a focal length change determination portion 37 Q, a subject image detection portion 37 A, an image position determination portion 37 B, a shift amount calculation portion 37 D, a shift determination portion 37 E, a shake correction execution portion 37 F, and a registration control portion 37 G
  • the subject image detection portion 37 A and the image position determination portion 37 B are an example of a “detection portion” according to the embodiment of the disclosed technology.
  • the acquisition portion 37 P acquires the focal length from the focal length storage region 35 A.
  • the focal length acquired by the acquisition portion 37 P the angle of view that can be imaged by the surveillance camera 10 is narrowed compared to the focal length before the increase.
  • the focal length change determination portion 37 Q determines whether or not the focal length acquired by the acquisition portion 37 P is changed. Specifically, an internal memory 37 M of the CPU 60 A in the management apparatus 11 stores the focal length (hereinafter, referred to as a “previous focal length”) acquired at a previous acquisition timing by the acquisition portion 37 P. The focal length change determination portion 37 Q compares the previous focal length stored in the internal memory 37 M with the focal length (hereinafter, referred to as the “most recent focal length”) most recently acquired by the acquisition portion 37 P. In a case where the previous focal length and the most recent focal length are different, it is determined that the focal length is changed. In a case where there is no difference, it is determined that the focal length is not changed.
  • the focal length change determination portion 37 Q updates a value of the previous focal length in the internal memory 37 M by storing a value of the most recent focal length in an overwriting manner in the internal memory 37 M.
  • the updated previous focal length is used by the focal length change determination portion 37 Q as a comparison target with the most recent focal length in a case where the determination is subsequently performed by the focal length change determination portion 37 Q.
  • the subject image detection portion 37 A acquires the captured image of one frame from the image memory 32 . Then, the subject image detection portion 37 A detects a target subject image in the captured image acquired from the image memory 32 . In addition, in a case where the target subject image is detected, the subject image detection portion 37 A detects a position of the target subject image (hereinafter, referred to as a “subject image position”).
  • the image position determination portion 37 B determines whether or not the subject image position detected by the subject image detection portion 37 A is in a center region in the captured image. As illustrated in FIG. 11 A and FIG. 11 B as an example, the subject image position may be in the center region in the captured image. Meanwhile, as illustrated in FIG. 12 A and FIG. 12 B as an example, the target subject may be present in the imaging region but not present in the center region due to revolution of the surveillance camera 10 by the revolution mechanism 16 or movement of the target subject.
  • a positive determination is made by the image position determination portion 37 B.
  • “present” means an aspect in which at least a part of the target subject image overlaps with the center region. That is, in a case where at least a part of the target subject image overlaps with the center region, a determination “present” is made by the image position determination portion 37 B.
  • Examples of at least the part of the target subject image include a specific part in the subject image (for example, in a case where the target subject is a person, a face of the person).
  • An image recognition function for example, a face recognition function
  • the image position determination portion 37 B may determine that the target subject image is present in the center region.
  • the center region is an example of a “specific position” according to the embodiment of the disclosed technology.
  • the shift amount calculation portion 37 D calculates a shift amount in a case where the image position determination portion 37 B determines that the subject image position is not in a center region CE.
  • the shift amount is an amount indicating shifting, that is, a difference in relative position, of the subject image position with respect to the center region CE and is a vector having two values in the pitch direction and the yaw direction.
  • Examples of the shift amount include a shift amount between a center of the target subject image and a center of the center region CE.
  • the center of the target subject image can be obtained as two values of a center coordinate in the pitch direction and a center coordinate in the yaw direction of the target subject image.
  • the shift determination portion 37 E determines whether or not the subject image position is shifted from the center region based on the shift amount calculated by the shift amount calculation portion 37 D. For example, whether or not the shift is present can be determined by determining whether or not the shift amount calculated by the shift amount calculation portion 37 D exceeds a predetermined threshold value. That is, in a case where the shift amount does not exceed the threshold value, it is determined that the shift is not present. Meanwhile, in a case where the shift amount is greater than or equal to the threshold value, it is determined that the shift is present.
  • the threshold value may be a fixed value that is derived in advance as an upper limit value allowable as the shift amount by a sensory test using an actual apparatus and/or computer simulation or the like.
  • the threshold value may be a variable value that can be changed in accordance with an instruction received by the reception device 62 or the like, or may be a variable value that can be changed in accordance with an imaging scene or the like.
  • the shake correction execution portion 37 F operates the shake correction portion 24 based on the shake amount detected by the shake amount detection sensor 40 .
  • the storage 60 B stores a ratio derivation table 60 L.
  • the ratio derivation table 60 L is used by the registration control portion 37 G in a registration control.
  • the ratio derivation table 60 L is a table indicating whether to increase a ratio of application of the imaging element side shake correction mechanism 45 to the lens side shake correction mechanism 29 or increase a ratio of application of the lens side shake correction mechanism 29 to the imaging element side shake correction mechanism 45 , in accordance with the focal length acquired by the acquisition portion 37 P.
  • the ratio of application of the imaging element side shake correction mechanism 45 to the lens side shake correction mechanism 29 will be referred to as a “first application ratio”, and the ratio of application of the lens side shake correction mechanism 29 to the imaging element side shake correction mechanism 45 will be referred to as a “second application ratio”.
  • first application ratio and the second application ratio will be referred to as the “application ratio” unless otherwise necessary to distinguish therebetween.
  • the imaging element side shake correction mechanism 45 is illustrated as an element defining the application ratio, the disclosed technology is not limited thereto.
  • the electronic shake correction portion 33 may be applied instead of the imaging element side shake correction mechanism 45 or together with the imaging element side shake correction mechanism 45 .
  • a ratio of use of the lens side shake correction mechanism 29 is higher than the imaging element side shake correction mechanism 45 .
  • a ratio of use of the imaging element side shake correction mechanism 45 is higher than the lens side shake correction mechanism 29 .
  • the focal length in a case where the focal length is within a predetermined range including the threshold value th, increasing the focal length gradually decreases the first application ratio and gradually increases the second application ratio.
  • the predetermined range include, as illustrated in FIG. 8 and FIG. 9 , a range of x on a negative side and a positive side from the threshold value th, that is, a range of greater than or equal to “threshold value th ⁇ x” and less than or equal to “threshold value th+x” (hereinafter, referred to as the “predetermined range”).
  • the first application ratio is linearly decreased, and the second application ratio is linearly increased.
  • the first application ratio may be non-linearly (for example, exponentially) decreased, and the second application ratio may be non-linearly (for example, exponentially) increased.
  • one of the first application ratio and the second application ratio may non-linearly change, and the other may linearly change. Even in this case, the first application ratio may be decreased, and the second application ratio may be increased.
  • the lens side shake correction mechanism 29 In a case where the focal length is within the predetermined range, only the lens side shake correction mechanism 29 is used in the registration control. In a case where the focal length is within a range exceeding “threshold value th ⁇ x”, only the imaging element side shake correction mechanism 45 is used in the registration control.
  • One of the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 having a lower application ratio is used for shake correction by the shake correction execution portion 37 F and can exhibit an effect of correcting the shake.
  • the lens side shake correction mechanism 29 is used for correcting the shake by the shake correction execution portion 37 F.
  • the imaging element side shake correction mechanism 45 is used for correcting the shake by the shake correction execution portion 37 F.
  • ratio derivation table 60 L is illustrated, the disclosed technology is not limited thereto.
  • a calculation expression for ratio calculation that takes the focal length as an independent variable and takes the application ratio as a dependent variable may be used instead of the ratio derivation table 60 L.
  • the registration control portion 37 G performs the registration control.
  • the registration control is a control including a registration control of setting the subject image position detected by the subject image detection portion 37 A to the specific position in the captured image by increasing the first application ratio in a case where the focal length acquired by the acquisition portion 37 P is greater than or equal to the threshold value, and increasing the second application ratio in a case where the focal length is less than the threshold value.
  • the application ratio corresponding to the shift amount calculated in the shift amount calculation portion 37 D is derived by the registration control portion 37 G from the ratio derivation table 60 L stored in the storage 60 B.
  • the subject image position is set to the specific position in the captured image by driving the lens side shake correction mechanism 29 and/or the imaging element side shake correction mechanism 45 by the registration control portion 37 G in accordance with the derived application ratio.
  • FIG. 10 A and FIG. 10 B illustrate an example of a flow of registration control processing executed by the CPU 60 A in accordance with the registration control processing program 36 A.
  • the flow of registration control processing illustrated in FIG. 10 A and FIG. 10 B is an example of an “imaging support method” according to the embodiment of the disclosed technology.
  • the acquisition portion 37 P determines whether or not a timing (hereinafter, referred to as a “focal length acquisition timing”) at which the focal length is acquired is reached.
  • the focal length acquisition timing include a timing at which the target subject is imaged at the predetermined frame rate in the imaging element 25 . More specifically, in a case where the frame rate is 60 frames/second, and the acquisition portion 37 P acquires the focal length for each frame, the focal length is acquired with a frequency of 60 times per second.
  • the acquisition portion 37 P may acquire the focal length for a plurality of frames. For example, the focal length may be acquired with a frequency of once for every two frames.
  • step ST 10 in a case where the focal length acquisition timing is not reached, a negative determination is made, and the registration control processing transitions to step ST 30 .
  • step ST 10 in a case where the focal length acquisition timing is reached, a positive determination is made, and the registration control processing transitions to step ST 12 .
  • step ST 12 the acquisition portion 37 P acquires the most recent focal length of the surveillance camera 10 .
  • the registration control processing transitions to step ST 14 .
  • step ST 14 the focal length change determination portion 37 Q determines whether or not the focal length is changed based on the most recent focal length acquired in step ST 12 and the previous focal length in the internal memory 37 M. In step ST 14 , in a case where the focal length is not changed, a negative determination is made, and the registration control processing transitions to step ST 32 illustrated in FIG. 10 B . In step ST 14 , in a case where the focal length is changed, a positive determination is made, and the registration control processing transitions to step ST 16 .
  • step ST 16 the focal length change determination portion 37 Q updates the value of the previous focal length stored in the internal memory 37 M to the value of the most recent focal length.
  • step ST 18 the subject image detection portion 37 A determines whether or not the target subject image is detected from the captured image.
  • step ST 18 in a case where the target subject image is not detected from the captured image, a negative determination is made, and the registration control processing transitions to step ST 30 .
  • step ST 18 in a case where the target subject image is detected from the captured image, a positive determination is made, and the registration control processing transitions to step ST 20 .
  • step ST 20 the image position determination portion 37 B detects the subject image position in the captured image.
  • the registration control processing transitions to step ST 21 .
  • step ST 21 the shift amount calculation portion 37 D calculates the shift amount.
  • the registration control processing transitions to step ST 22 .
  • step ST 22 the shift determination portion 37 E determines whether or not there is a shift between the subject image position and the center region based on the shift amount detected in step ST 21 .
  • step ST 22 in a case where there is no shift, a negative determination is made, and the registration control processing transitions to step ST 32 .
  • step ST 22 in a case where there is a shift, a positive determination is made, and the registration control processing transitions to step ST 24 .
  • step ST 24 the shake correction execution portion 37 F derives the application ratio corresponding to the most recent focal length acquired in step ST 12 from the ratio derivation table 60 L.
  • step ST 25 the shake correction execution portion 37 F determines whether or not the shake correction is being performed. That is, a determination as to whether or not the correction of the shake by the shake correction portion 24 is being performed (hereinafter, referred to as “shake correction is being performed”).
  • step ST 25 in a case where the shake correction is not being performed, a negative determination is made, and the registration control processing transitions to step ST 28 .
  • step ST 25 in a case where the shake correction is being performed, a positive determination is made, and the registration control processing transitions to step ST 26 .
  • step ST 26 the shake correction execution portion 37 F causes the shake correction portion 24 to finish the correction of the shake.
  • the registration control processing transitions to step ST 28 .
  • step ST 28 the registration control portion 37 G executes the registration control in accordance with the shift amount calculated in step ST 21 and the application ratio acquired in step ST 12 .
  • the registration control portion 37 G determines whether or not a condition (hereinafter, referred to as a “finish condition”) under which the registration control processing is finished is satisfied.
  • the finish condition include a condition that an instruction to finish the registration control processing is received by the reception device 62 .
  • a negative determination is made, and the registration control processing transitions to step ST 10 .
  • a positive determination is made, and the registration control processing is finished.
  • step ST 32 the shake correction execution portion 37 F determines whether or not the shake correction is being performed. In step ST 32 , in a case where the shake correction is being performed, a positive determination is made, and the registration control processing transitions to step ST 30 . In step ST 32 , in a case where the shake correction is not being performed, a negative determination is made, and the registration control processing transitions to step ST 34 .
  • step ST 34 the shake correction execution portion 37 F starts the shake correction. A transition is made to step ST 30 .
  • the registration control of setting the subject image position to the center region in the captured image is performed. Since the subject image position is present in the center region in the captured image, a state where the subject image is easily visible can be implemented.
  • the application ratio of the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 is derived in accordance with the focal length of the surveillance camera 10 .
  • the registration control is performed by increasing the ratio of use of the lens side shake correction mechanism 29 compared to the imaging element side shake correction mechanism 45 in a case where the focal length is less than the threshold value th, and increasing the ratio of use of the imaging element side shake correction mechanism 45 compared to the lens side shake correction mechanism 29 in a case where the focal length is greater than or equal to the threshold value th.
  • FIG. 11 A and FIG. 11 B illustrate an example of a relationship between the imaging range in which the target subject is imaged, and the position of the target subject image in the captured image in a case where the focal length of the surveillance camera 10 is on the wide angle side out of the telephoto side and the wide angle side.
  • the target subject image is in the center region of the captured image.
  • the subject image position is shifted from the center region of the captured image as illustrated in FIG. 12 A and FIG. 12 B as an example.
  • the subject image position is within a range of the captured image.
  • the angle of view is narrowed.
  • the subject image position may be positioned outside the range of the captured image.
  • the registration control processing is performed, and the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 operate at the application ratio corresponding to the focal length in the present embodiment. Accordingly, as illustrated in FIG. 14 as an example, the subject image position can be positioned in the center region in the captured image.
  • the application ratio of the imaging element side shake correction mechanism 45 is set to be higher than the application ratio of the lens side shake correction mechanism 29 . Since the imaging element side shake correction mechanism 45 moves the light-receiving surface 25 A on which the image of the light from the subject is formed, the subject image position can be set to the center region independently of the focal length.
  • the ratio of application of the lens side shake correction mechanism 29 is set to be higher than the ratio of application of the imaging element side shake correction mechanism 45 .
  • the lens side shake correction mechanism 29 sets the position of the subject image position to the center region by inclining the optical axis OA compared to the optical axis OA before the operation of the lens side shake correction mechanism 29 . Accordingly, by moving the anti-vibration lens 15 B 1 in the direction perpendicular to the optical axis OA, the position of the subject image position can be set to the center region.
  • the surveillance camera 10 comprises an imaging support device 44 .
  • the registration control is performed by increasing the ratio of application of the imaging element side shake correction mechanism 45 in a case where the focal length of the surveillance camera 10 is greater than or equal to the threshold value th, and increasing the ratio of application of the lens side shake correction mechanism 29 in a case where the focal length is less than the threshold value th.
  • the subject image position can be set to the center region (an example of the specific position) in the captured image compared to a case of performing the registration control depending on only the imaging element side shake correction mechanism 45 or the lens side shake correction mechanism 29 regardless of the focal length.
  • the application ratio of the lens side shake correction mechanism 29 is lower than the application ratio of the imaging element side shake correction mechanism 45 in the registration control.
  • the lens side shake correction mechanism 29 executes a shake correction control. Accordingly, the shake can be corrected even while the registration control is being performed.
  • the lens side shake correction mechanism 29 is not used in the registration control.
  • the shake can be corrected using the lens side shake correction mechanism 29 for only the shake correction.
  • the application ratio of the imaging element side shake correction mechanism 45 is lower than the application ratio of the lens side shake correction mechanism 29 in the registration control. In this case, the shake can be corrected even while the registration control is being performed, by causing the imaging element side shake correction mechanism 45 to execute the shake correction control.
  • the imaging element side shake correction mechanism 45 is not used in the registration control.
  • the shake can be corrected using the imaging element side shake correction mechanism 45 for only the shake correction.
  • the surveillance camera 10 of the present embodiment includes the variable magnification mechanism.
  • the registration control is performed using the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 together.
  • the subject image position can be set to the center region of the captured image with high accuracy compared to a case of performing the registration control processing using only the lens side shake correction mechanism 29 or the imaging element side shake correction mechanism 45 .
  • the registration control processing is performed using the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 together in a case where the focal length is within the range of greater than or equal to “threshold value th ⁇ x” and less than or equal to “threshold value th+x”.
  • the subject image position can be set to the specific position in the captured image with high accuracy compared to a case of performing the registration control using only the imaging element side shake correction mechanism 45 or the lens side shake correction mechanism 29 at all times regardless of whether or not the focal length is within the range of greater than or equal to “threshold value th ⁇ x” and less than or equal to “threshold value th+x”.
  • the application ratio of the lens side shake correction mechanism 29 is linearly decreased, and the application ratio of the imaging element side shake correction mechanism 45 is linearly increased.
  • the focal length of the surveillance camera 10 continuously changes by the variable magnification mechanism 18 , rapid switching is not performed from the lens side shake correction mechanism 29 to the imaging element side shake correction mechanism 45 . Accordingly, visual recognition, from the captured image, of a timing of switching from one of the registration control processing using only the lens side shake correction mechanism 29 and the registration control using only the imaging element side shake correction mechanism 45 to the other can be suppressed.
  • the shake correction portion 24 causes one of the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 to execute the shake correction control. Accordingly, a captured image in which an effect of the shake is reduced can be obtained compared to a configuration in which the shake is not corrected in a case where the focal length is not changing.
  • the shake correction portion 24 performs the registration control by applying the lens side shake correction mechanism 29 .
  • the imaging element side shake correction mechanism 45 is not applied in the registration control.
  • the shake correction control can be performed using the imaging element side shake correction mechanism 45 . That is, room for correcting the shake can be secured.
  • the shake correction portion 24 performs the registration control by applying the imaging element side shake correction mechanism 45 .
  • the lens side shake correction mechanism 29 is not applied in the registration control.
  • the shake correction control can be performed using the lens side shake correction mechanism 29 . That is, room for correcting the shake can be secured.
  • the imaging support device in performing the registration control processing, causes the surveillance camera 10 to revolve by operating the revolution mechanism 16 in addition to the lens side shake correction mechanism 29 and/or the imaging element side shake correction mechanism 45 .
  • the subject image position may be able to be set in the center region by causing the surveillance camera 10 revolve by operating the revolution mechanism 16 . Furthermore, even in a case where the subject image position is not in the captured image, the subject image position may be able to be set in the captured image by causing the surveillance camera 10 to revolve by operating the revolution mechanism 16 .
  • the storage 60 B stores a revolution amount derivation table 60 M in addition to the ratio derivation table 60 L.
  • the revolution amount derivation table 60 M the shift amount and a revolution amount of the surveillance camera 10 are associated with each other.
  • the revolution amount is information used in a case of causing the subject image position to approach the center region in the captured image based on the shift amount between the subject image position and the center region.
  • the revolution amount is decided in accordance with each of a pair of axial directions. That is, the revolution amount is decided in accordance with each of the direction of the pitch axis PA and the direction of the yaw axis YA. Accordingly, in the captured image, the subject image position can be caused to approach the center region CE in both directions of the direction of the pitch axis PA and the direction of the yaw axis YA.
  • the revolution amount derivation table 60 M is illustrated, the disclosed technology is not limited thereto. For example, a revolution amount derivation calculation expression that takes the shift amount as an independent variable and takes the revolution amount as a dependent variable may be applied instead of the revolution amount derivation table 60 M.
  • the registration control portion 37 G acquires the revolution amount from the revolution amount derivation table 60 M.
  • the registration control portion 37 G causes the surveillance camera 10 to revolve in the pitch direction and/or the yaw direction by operating the revolution mechanism 16 with the derived revolution amount.
  • FIG. 16 A and FIG. 16 B illustrate an example of a flow of registration control processing executed by the CPU 60 A (refer to FIG. 5 ) in the second embodiment.
  • FIG. 16 A and FIG. 16 B are different from the flow of registration control processing illustrated in FIG. 10 A and FIG. 10 B , in that step ST 100 to step ST 106 are included between step ST 22 and step ST 24 .
  • the flow of registration control processing illustrated in FIG. 16 A and FIG. 16 B is an example of the “imaging support method” according to the embodiment of the disclosed technology.
  • step ST 100 the registration control portion 37 G derives the revolution amount corresponding to the shift amount from the revolution amount derivation table 60 M stored in the storage 60 B.
  • step ST 102 the registration control portion 37 G causes the surveillance camera 10 to revolve by operating the revolution mechanism 16 .
  • step ST 104 the registration control portion 37 G calculates the shift amount.
  • step ST 106 the registration control portion 37 G determines whether or not there is a shift. In step ST 106 , in a case where there is no shift, a negative determination is made, and the registration control processing transitions to step ST 32 (refer to FIG. 10 B ). In step ST 106 , in a case where there is a shift, a positive determination is made, and the registration control processing transitions to step ST 24 .
  • the surveillance camera 10 in a case where there is a shift between the subject image position and the center region, the surveillance camera 10 is caused to revolve by the revolution amount corresponding to the shift amount by operating the revolution mechanism 16 .
  • the surveillance camera 10 By causing the surveillance camera 10 to revolve, a state where the target subject image is positioned in the captured image can be implemented in a case where the target subject image is positioned outside the captured image.
  • Accuracy of the registration by the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 is high compared to the accuracy of the registration by the revolution of the surveillance camera 10 .
  • the registration by the revolution of the surveillance camera 10 enables registration within a wide range for a short time period compared to the registration by the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 . Accordingly, as illustrated in FIG. 17 as an example, coarse adjustment of the registration can be performed first by the revolution of the surveillance camera 10 , and then, fine adjustment of the registration can be performed by the lens side shake correction mechanism 29 and the imaging element side shake correction mechanism 45 with higher accuracy of the registration than the registration by the revolution of the surveillance camera 10 .
  • the registration control using the revolution mechanism 16 in a case where variations occur in the revolution angle based on the operation of the revolution mechanism 16 , it is difficult to position the subject image position in the center region in the registration control based on the revolution mechanism 16 .
  • coarse adjustment of the subject image position is performed by operating the revolution mechanism 16 , and then, fine adjustment of the subject image position is performed by operating the lens side shake correction mechanism 29 and/or the imaging element side shake correction mechanism 45 .
  • high accuracy of the registration control can be implemented.
  • an example of applying the imaging element side shake correction mechanism 45 in the registration control in a configuration in which the surveillance camera 10 includes the imaging element side shake correction mechanism 45 and the electronic shake correction portion 33 as the image region movement portion is illustrated.
  • the image region movement portion to be applied in the registration control may be the electronic shake correction portion 33 , or the imaging element side shake correction mechanism 45 and the electronic shake correction portion 33 may be used together.
  • a threshold value (hereinafter, referred to as a “shift threshold value”) may be set for the shift amount of the subject image position, and in a case where the shift amount is greater than or equal to the shift threshold value, the registration control may be performed by operating the shake correction portion 24 .
  • the shift amount may be calculated at a constant timing regardless of whether or not the focal length of the surveillance camera 10 changes. For example, even in a state where the focal length of the surveillance camera 10 does not change and is maintained on the wide angle side, the position of the target subject may be outside the center region while the position of the target subject is inside the captured image, due to movement of the target subject or the like. In a case where the shift amount in this state is large, and magnification is changed to the telephoto side by the variable magnification mechanism 18 , the subject image may be enlarged and move outside the captured image. Accordingly, the shift threshold value can be set based on whether or not the subject image position outside the center region moves outside the captured image due to the changing of magnification from the wide angle side to the telephoto side.
  • the subject image position can be set to the center region with high accuracy compared to a configuration of not performing the registration control.
  • the lens side shake correction mechanism 29 is illustrated as an example of the “lens moving mechanism” according to the embodiment of the disclosed technology.
  • the imaging element side shake correction mechanism 45 and the electronic shake correction portion 33 are illustrated as an example of the “image region movement portion (image region movement component)” according to the embodiment of the disclosed technology. All of the lens side shake correction mechanism 29 , the imaging element side shake correction mechanism 45 , and the electronic shake correction portion 33 are an example of the shake correction portion and are applied for correcting the shift of the subject image position.
  • the lens moving mechanism according to the embodiment of the disclosed technology is not limited to a mechanism for correcting the shift of the subject image position and may have a configuration in which a lens can be moved in a direction intersecting with the optical axis OA.
  • the image region movement portion according to the embodiment of the disclosed technology is also not limited to a configuration for correcting the shift of the subject image position as long as the region of the captured image in the imaging element 25 can be moved in the direction intersecting with the optical axis OA.
  • the center region described in the embodiments is an example of the “specific position” according to the embodiment of the disclosed technology.
  • the “specific position” is not limited to the center region in the captured image and may be a position shifted from the center of the captured image in the pitch direction and the yaw direction. By setting the specific position as the center region in the captured image, the subject image position can be set to the center in the captured image.
  • the shake is corrected by moving the anti-vibration lens 15 B 1 . Accordingly, it is possible to correct the shake within a movable range of the anti-vibration lens 15 B 1 and set the subject image position to the center region.
  • FIG. 9 an example in which in a case where the focal length is within the predetermined range, the ratio of application of the lens side shake correction mechanism 29 is linearly decreased, and the ratio of application of the imaging element side shake correction mechanism 45 is linearly increased in the registration control is illustrated.
  • FIG. 18 a control of setting the application ratio of the lens side shake correction mechanism 29 to 100% in a case where the focal length is less than the threshold value th, and setting the application ratio of the imaging element side shake correction mechanism 45 to 100% in a case where the focal length is greater than or equal to the threshold value th may be performed.
  • the registration control is facilitated.
  • the storage 60 B of the management apparatus 11 stores the registration control processing program 36 A, the display control processing program 36 B, and the variable magnification mechanism control processing program 36 C.
  • An example of a form of executing these programs in the memory 60 C of the management apparatus 11 is illustrated.
  • the disclosed technology is not limited thereto.
  • an example in which these programs are stored in the storage 36 of the surveillance camera 10 , and the CPU 37 of the surveillance camera 10 executes these programs in the memory 35 may be applied.
  • the registration control processing program 36 A, the display control processing program 36 B, and the variable magnification mechanism control processing program 36 C may be stored in a storage medium 100 that is a non-transitory storage medium.
  • the registration control processing program 36 A, the display control processing program 36 B, and the variable magnification mechanism control processing program 36 C stored in the storage medium 100 are installed on the control device 60 .
  • the CPU 60 A executes above each processing in accordance with the registration control processing program 36 A, the display control processing program 36 B, and the variable magnification mechanism control processing program 36 C.
  • the CPU 60 A is a single CPU.
  • the disclosed technology is not limited thereto, and a plurality of CPUs may be employed.
  • Examples of the storage medium 100 include any portable storage medium such as an SSD or a USB memory.
  • the registration control processing program 36 A may be stored in a storage portion of another computer, a server apparatus, or the like connected to the computer 19 through a communication network (not illustrated), and the registration control processing program 36 A may be downloaded to the computer 19 in response to a request of the surveillance camera 10 .
  • the downloaded registration control processing program 36 A is executed by the CPU 37 of the computer 19 .
  • the disclosed technology is not limited thereto.
  • the registration control processing program 36 A, the display control processing program 36 B, and the variable magnification mechanism control processing program 36 C may be installed on the computer 19 of the surveillance camera 10 .
  • the CPU 37 executes the registration control processing in accordance with the registration control processing program 36 A, executes display control processing described above in accordance with the display control processing program 36 B, and executes variable magnification mechanism control processing described above in accordance with the variable magnification mechanism control processing program 36 C.
  • the registration control processing, the display control processing, and the variable magnification mechanism control processing may be performed in a distributed manner between the surveillance camera 10 and the management apparatus 11 .
  • the registration control may be executed by the CPU 60 A of the management apparatus 11
  • the display control and the variable magnification mechanism control may be executed by the CPU 37 of the surveillance camera 10 .
  • variable magnification mechanism control portion 37 S the display control portion 37 T, the acquisition portion 37 P, the focal length change determination portion 37 Q, the subject image detection portion 37 A, the image position determination portion 37 B, the shift amount calculation portion 37 D, the shift determination portion 37 E, and the registration control portion 37 G by a software configuration using the control device 60 is illustratively described, the disclosed technology is not limited thereto.
  • variable magnification mechanism control portion 37 S, the display control portion 37 T, the acquisition portion 37 P, the focal length change determination portion 37 Q, the subject image detection portion 37 A, the image position determination portion 37 B, the shift amount calculation portion 37 D, the shift determination portion 37 E, and/or the registration control portion 37 G may be implemented by, for example, a device including an ASIC, an FPGA, and/or a PLD.
  • variable magnification mechanism control portion 37 S, the display control portion 37 T, the acquisition portion 37 P, the focal length change determination portion 37 Q, the subject image detection portion 37 A, the image position determination portion 37 B, the shift amount calculation portion 37 D, the shift determination portion 37 E, and/or the registration control portion 37 G may be implemented by a combination of a hardware configuration and a software configuration.
  • processors illustrated below can be used as a hardware resource for executing the registration control processing, the display control processing, and the variable magnification mechanism control processing (hereinafter, these processing will be collectively referred to as “control processing”).
  • the processors include a CPU that is a general-purpose processor functioning as the hardware resource for executing the control processing by executing software, that is, the programs.
  • examples of the processors include a dedicated electric circuit such as an FPGA, a PLD, or an ASIC that is a processor having a circuit configuration dedicatedly designed to execute specific processing. Any of the processors incorporates or is connected to a memory, and any of the processors executes the control processing using the memory.
  • the hardware resource for executing the control processing may be configured with one of those various processors or may be configured with a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA).
  • the hardware resource for executing the control processing may be one processor.
  • Examples of a configuration with one processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and the processor functions as the hardware resource for executing processing of each of the variable magnification mechanism control portion 37 S, the display control portion 37 T, the acquisition portion 37 P, the focal length change determination portion 37 Q, the subject image detection portion 37 A, the image position determination portion 37 B, the shift amount calculation portion 37 D, the shift determination portion 37 E, and the registration control portion 37 G Second, as represented by an SoC or the like, a form of using a processor that implements, by one IC chip, functions of the entire system including a plurality of hardware resources for executing the registration control processing is included.
  • variable magnification mechanism control portion 37 S the display control portion 37 T, the acquisition portion 37 P, the focal length change determination portion 37 Q, the subject image detection portion 37 A, the image position determination portion 37 B, the shift amount calculation portion 37 D, the shift determination portion 37 E, and the registration control portion 37 G is implemented using one or more of the various processors as the hardware resource.
  • circuit elements such as semiconductor elements are combined can be used as a hardware structure of those various processors.
  • the registration control processing is merely an example. Accordingly, unnecessary steps may be deleted, new steps may be added, or a processing order may be rearranged without departing from the gist of the disclosed technology.
  • the disclosed technology is not limited thereto.
  • the disclosed technology can also be applied to various electronic apparatuses such as a portable lens-interchangeable camera, a portable fixed lens camera, a personal computer, a smart device, or a wearable terminal apparatus instead of the surveillance camera 10 .
  • the same actions and effects as the surveillance camera 10 described in each of the embodiments are also obtained for these electronic apparatuses.
  • control processing is merely an example. Accordingly, unnecessary steps may be deleted, new steps may be added, or a processing order may be rearranged without departing from the gist of the disclosed technology.
  • a and/or B has the same meaning as “at least one of A or B”. This means that “A and/or B” may be only A, only B, or a combination of A and B. In addition, in the present specification, the same approach as “A and/or B” is applied to a case where three or more matters are represented by connecting the matters with “and/or”.
  • An information processing apparatus including a processor, and a memory that is incorporated in or connected to the processor, in which the processor is configured to acquire a focal length of an imaging apparatus which includes a lens and an imaging element and includes a lens moving mechanism which moves the lens in a direction intersecting with an optical axis of the lens, and an image region movement portion which moves a region of a captured image in the imaging element in the direction intersecting with the optical axis, detect a subject image position of a target subject image showing a target subject in the captured image obtained by capturing an imaging region including the target subject by the imaging apparatus, and perform a registration control of setting the detected subject image position to a specific position in the captured image by increasing a ratio of application of the image region movement portion compared to the lens moving mechanism in a case where the acquired focal length is greater than or equal to a threshold value, and increasing a ratio of application of the lens moving mechanism compared to the image region movement portion in a case where the focal length is less than the threshold value.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Studio Devices (AREA)
  • Adjustment Of Camera Lenses (AREA)
  • Accessories Of Cameras (AREA)
  • Automatic Focus Adjustment (AREA)
US17/728,993 2019-10-29 2022-04-26 Imaging support device that performs registration control for setting a detected subject image position, imaging support system, imaging system, imaging support method, and program Active US11895403B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/539,310 US20240114245A1 (en) 2019-10-29 2023-12-14 Imaging support device, imaging support system, imaging system, imaging support method, and program

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2019-196683 2019-10-29
JP2019196683 2019-10-29
PCT/JP2020/039427 WO2021085249A1 (ja) 2019-10-29 2020-10-20 撮像支援装置、撮像支援システム、撮像システム、撮像支援方法、及びプログラム

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/039427 Continuation WO2021085249A1 (ja) 2019-10-29 2020-10-20 撮像支援装置、撮像支援システム、撮像システム、撮像支援方法、及びプログラム

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/539,310 Continuation US20240114245A1 (en) 2019-10-29 2023-12-14 Imaging support device, imaging support system, imaging system, imaging support method, and program

Publications (2)

Publication Number Publication Date
US20220256086A1 US20220256086A1 (en) 2022-08-11
US11895403B2 true US11895403B2 (en) 2024-02-06

Family

ID=75715110

Family Applications (2)

Application Number Title Priority Date Filing Date
US17/728,993 Active US11895403B2 (en) 2019-10-29 2022-04-26 Imaging support device that performs registration control for setting a detected subject image position, imaging support system, imaging system, imaging support method, and program
US18/539,310 Pending US20240114245A1 (en) 2019-10-29 2023-12-14 Imaging support device, imaging support system, imaging system, imaging support method, and program

Family Applications After (1)

Application Number Title Priority Date Filing Date
US18/539,310 Pending US20240114245A1 (en) 2019-10-29 2023-12-14 Imaging support device, imaging support system, imaging system, imaging support method, and program

Country Status (4)

Country Link
US (2) US11895403B2 (ja)
JP (3) JP7265642B2 (ja)
CN (2) CN114631056B (ja)
WO (1) WO2021085249A1 (ja)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827447B (zh) * 2021-01-29 2024-02-09 北京小米移动软件有限公司 图像抖动校正方法和装置
JP2022155134A (ja) * 2021-03-30 2022-10-13 キヤノン株式会社 撮像装置
WO2023029046A1 (zh) * 2021-09-06 2023-03-09 深圳市大疆创新科技有限公司 拍摄设备及其对焦方法、拍摄系统、可读存储介质

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0514801A (ja) 1990-10-18 1993-01-22 Fuji Photo Film Co Ltd 手振れ補正装置
JPH0946575A (ja) 1995-07-26 1997-02-14 Sony Corp 撮像装置
US20010002843A1 (en) * 1999-12-03 2001-06-07 Kunio Yata Automatic following device
US6784927B1 (en) * 1997-12-22 2004-08-31 Canon Kabushiki Kaisha Image processing apparatus and image processing method, and storage medium
US20050057662A1 (en) * 2003-09-02 2005-03-17 Canon Kabushiki Kaisha Image-taking apparatus
JP2006071743A (ja) 2004-08-31 2006-03-16 Olympus Corp ブレ補正機能を有するカメラシステム及びその補正方法
US20070296821A1 (en) * 2006-06-22 2007-12-27 Nokia Corporation Method and system for image stabilization
JP2008199275A (ja) 2007-02-13 2008-08-28 Casio Comput Co Ltd 手ぶれ補正装置及びプログラム
JP2012247544A (ja) 2011-05-26 2012-12-13 Fujifilm Corp 撮像装置、プログラム及び撮像装置のブレ補正方法
US20160070115A1 (en) * 2014-09-08 2016-03-10 Apple Inc. Upright mounted camera actuator component with trapezoidal magnet for optical image stabilization
US20160134814A1 (en) 2014-11-11 2016-05-12 Olympus Corporation Imaging apparatus having camera shake correction device
CN106164730A (zh) 2014-04-10 2016-11-23 奥林巴斯株式会社 焦点调节装置、照相机系统以及摄像装置的焦点调节方法
US20170187960A1 (en) 2015-12-25 2017-06-29 Olympus Corporation Imaging apparatus
US20170214838A1 (en) 2016-01-26 2017-07-27 Canon Kabushiki Kaisha Image capturing apparatus and method for controlling the same, image stabilization apparatus and method for controlling the same, and lens unit
US10129464B1 (en) * 2016-02-18 2018-11-13 Gopro, Inc. User interface for creating composite images
US20190158745A1 (en) 2017-11-20 2019-05-23 Canon Kabushiki Kaisha Imaging apparatus and control method thereof
JP2019091063A (ja) 2019-01-17 2019-06-13 株式会社ニコン ブレ補正装置、電子機器及びカメラ
US20190199930A1 (en) 2017-12-26 2019-06-27 Canon Kabushiki Kaisha Image stabilization control apparatus, image capturing apparatus, image capturing system, control method, and storage medium
US20190222763A1 (en) * 2018-01-12 2019-07-18 Canon Kabushiki Kaisha Image stabilization control apparatus, image capturing system, control method, and recording medium

Patent Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0514801A (ja) 1990-10-18 1993-01-22 Fuji Photo Film Co Ltd 手振れ補正装置
US5282044A (en) * 1990-10-18 1994-01-25 Fuji Photo Film Co., Ltd. Camera shake correction system
JPH0946575A (ja) 1995-07-26 1997-02-14 Sony Corp 撮像装置
US6784927B1 (en) * 1997-12-22 2004-08-31 Canon Kabushiki Kaisha Image processing apparatus and image processing method, and storage medium
US20010002843A1 (en) * 1999-12-03 2001-06-07 Kunio Yata Automatic following device
US20050057662A1 (en) * 2003-09-02 2005-03-17 Canon Kabushiki Kaisha Image-taking apparatus
JP2006071743A (ja) 2004-08-31 2006-03-16 Olympus Corp ブレ補正機能を有するカメラシステム及びその補正方法
US20070296821A1 (en) * 2006-06-22 2007-12-27 Nokia Corporation Method and system for image stabilization
JP2008199275A (ja) 2007-02-13 2008-08-28 Casio Comput Co Ltd 手ぶれ補正装置及びプログラム
JP2012247544A (ja) 2011-05-26 2012-12-13 Fujifilm Corp 撮像装置、プログラム及び撮像装置のブレ補正方法
US20170013199A1 (en) 2014-04-10 2017-01-12 Olympus Corporation Focal point adjustment device, camera system, and focal point adjustment method for imaging device
CN106164730A (zh) 2014-04-10 2016-11-23 奥林巴斯株式会社 焦点调节装置、照相机系统以及摄像装置的焦点调节方法
US20160070115A1 (en) * 2014-09-08 2016-03-10 Apple Inc. Upright mounted camera actuator component with trapezoidal magnet for optical image stabilization
JP2016090973A (ja) 2014-11-11 2016-05-23 オリンパス株式会社 撮像装置
US20160134814A1 (en) 2014-11-11 2016-05-12 Olympus Corporation Imaging apparatus having camera shake correction device
US20170187960A1 (en) 2015-12-25 2017-06-29 Olympus Corporation Imaging apparatus
CN107026960A (zh) 2015-12-25 2017-08-08 奥林巴斯株式会社 摄像装置
US20170214838A1 (en) 2016-01-26 2017-07-27 Canon Kabushiki Kaisha Image capturing apparatus and method for controlling the same, image stabilization apparatus and method for controlling the same, and lens unit
US10129464B1 (en) * 2016-02-18 2018-11-13 Gopro, Inc. User interface for creating composite images
US20190158745A1 (en) 2017-11-20 2019-05-23 Canon Kabushiki Kaisha Imaging apparatus and control method thereof
JP2019095509A (ja) 2017-11-20 2019-06-20 キヤノン株式会社 撮像装置およびその制御方法
US20190199930A1 (en) 2017-12-26 2019-06-27 Canon Kabushiki Kaisha Image stabilization control apparatus, image capturing apparatus, image capturing system, control method, and storage medium
CN110012219A (zh) 2017-12-26 2019-07-12 佳能株式会社 图像稳定控制设备、摄像设备和系统及方法、存储介质
US20190222763A1 (en) * 2018-01-12 2019-07-18 Canon Kabushiki Kaisha Image stabilization control apparatus, image capturing system, control method, and recording medium
JP2019125864A (ja) 2018-01-12 2019-07-25 キヤノン株式会社 防振制御装置、撮像システム、制御方法及びプログラム
JP2019091063A (ja) 2019-01-17 2019-06-13 株式会社ニコン ブレ補正装置、電子機器及びカメラ

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
English language translation of the following: Office action dated Oct. 28, 2023 from the SIPO in a Chinese patent application No. 202080075914.8 corresponding to the instant patent application. This office action translation is submitted now in order to supplement the understanding of the cited references which are being disclosed in the instant Information Disclosure Statement.
International Search Report issued in International Application No. PCT/JP2020/039427 dated Dec. 28, 2020.
Written Opinion of the ISA issued in International Application No. PCT/JP2020/039427 dated Dec. 28, 2020.

Also Published As

Publication number Publication date
JP7265642B2 (ja) 2023-04-26
CN114631056A (zh) 2022-06-14
JP2023089098A (ja) 2023-06-27
JPWO2021085249A1 (ja) 2021-05-06
JP2024073503A (ja) 2024-05-29
US20240114245A1 (en) 2024-04-04
US20220256086A1 (en) 2022-08-11
CN118672030A (zh) 2024-09-20
WO2021085249A1 (ja) 2021-05-06
CN114631056B (zh) 2024-07-09
JP7456043B2 (ja) 2024-03-26

Similar Documents

Publication Publication Date Title
US11895403B2 (en) Imaging support device that performs registration control for setting a detected subject image position, imaging support system, imaging system, imaging support method, and program
US11979663B2 (en) Imaging support device, imaging support system, imaging system, imaging support method, and program
US20240187733A1 (en) Imaging support device, imaging apparatus, imaging system, imaging support system, imaging support method, and program
US20240129631A1 (en) Blur correction device, imaging apparatus, monitoring system, and program
US8553097B2 (en) Reducing blur based on a kernel estimation of an imaging device
US11706529B2 (en) Blur correction device, imaging apparatus, monitoring system, and non-transitory computer-readable storage medium
JP5393877B2 (ja) 撮像装置および集積回路
US20220286613A1 (en) Imaging support device, imaging apparatus, imaging system, imaging support method, and program
US11917297B2 (en) Imaging support device, imaging apparatus, imaging system, imaging support system, imaging support method, and program
US11930280B2 (en) Imaging support device, imaging system, imaging support method, and program
US20240323536A1 (en) Control device, control method, and control program

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: FUJIFILM CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FUJIKAWA, TETSUYA;FUJIKI, SHINICHIRO;TANAKA, JUNICHI;AND OTHERS;SIGNING DATES FROM 20220303 TO 20220304;REEL/FRAME:059970/0141

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE