JP2012253581A - Imaging device and imaging system having the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a stable pixel shift control for a long time with keeping high precision in an imaging device for imaging a low-resolution image used for super-resolution processing.SOLUTION: An imaging device 2 has an imaging element 31, a lens 51, a lens holder 52 for holding the lens and an optical shift mechanism 35 for moving the lens in a horizontal direction perpendicular to the optical axis O of the lens while the lens is kept to be contactless with the lens holder, thereby relatively displacing an optical image formed on a light-receiving surface of the imaging element and the imaging element. The lens holder is provided with horizontal direction magnets 53A to 53D and axial direction magnets 55A to 55D, and the optical shift mechanism is provided with horizontal direction position controllers 36A to 36D for controlling the position of the lens in the horizontal direction by applying magnetic force to the horizontal direction magnets and axial direction position controllers 38A to 38D for controlling the position of the lens in the optical axis direction by applying magnetic force to the axial direction magnets.

Description

  The present invention performs super-resolution processing from a plurality of low-resolution images acquired by so-called pixel shifting, which performs imaging while relatively displacing the optical image formed on the light receiving surface of the imaging element and the imaging element. The present invention relates to an imaging apparatus suitable for generating a resolution image and an imaging system including the imaging apparatus.

  An imaging apparatus in this type of imaging system employs a two-dimensional image sensor in which pixels are arranged in a matrix. In this two-dimensional image sensor, the resolution depends on the size of the pixel and the number of pixels. On the other hand, it is higher than the original resolution of the image sensor from a plurality of original images acquired by so-called pixel shifting, which is performed while relatively displacing the optical image formed on the light receiving surface of the image sensor and the image sensor. A technique for generating a resolution image is conventionally known.

  In such a technique for increasing the resolution by shifting pixels, a shift mechanism (hereinafter referred to as “optical shift mechanism”) that relatively displaces the optical image and the image sensor is referred to as “optical shift”. For example, a technique for minutely displacing the image sensor with an actuator made of a piezo element is known (see Patent Document 1).

  In addition, a parallel plate glass is disposed between the image pickup optical system and the image pickup device, and the parallel plate glass is driven by electrostatic force or electromagnetic force to be inclined with respect to the optical axis of the image pickup optical system. There is a technique for shifting the position of the optical image on the light receiving surface (see Patent Documents 2 to 4).

  Further, as an image processing method for generating a high-resolution image from a plurality of original images acquired by pixel shifting, an image shift process for mapping pixel values of a low-resolution image to pixels of a high-resolution image, or an ML (Maximum-likelihood) method A super-resolution technique using a MAP (Maximum A Posterior) method, a POCS (Projection On to Convex Sets) method, or the like is known (see Patent Document 1).

JP 2008-306492 A JP 2006-135467 A Japanese Patent No. 3684027 JP-A-8-070406

  By the way, when the imaging device is applied to a monitoring camera or the like, high accuracy is maintained in the position control (that is, pixel shift control) of the optical image on the light receiving surface of the imaging device performed for super-resolution processing. However, it is desirable to be performed stably for a long time.

  However, in the prior art described in Patent Documents 2 to 4, the parallel flat glass (or a member that holds the glass) is completely in relation to the surrounding members due to the structure of driving (pressing) when supporting and tilting. Since it is not in a non-contact state, when used for a long period of time, it is difficult to perform stable pixel shift control due to wear or damage of each member due to contact or sliding between the shift mechanism and surrounding members. There was a problem of becoming. Further, in the configuration in which the parallel flat glass is inclined and the optical image and the image pickup device are relatively minutely displaced as in the prior art described above, when the shift mechanism and the surrounding members are not in contact with each other, highly accurate pixel shifting is performed. There was also a problem that it was difficult to control.

  The present invention has been devised in view of the above-described problems of the prior art, and includes an imaging apparatus capable of performing pixel shift control stably for a long time while maintaining high accuracy, and the same. The main purpose is to provide an imaging system.

  An imaging device of the present invention includes an imaging device that photoelectrically converts light from a subject and outputs a pixel signal, a lens that forms an image of light from the subject on the imaging device, a lens holder that holds the lens, By moving the lens in a horizontal direction perpendicular to the optical axis in a non-contact state with the lens holder, the optical image formed on the light receiving surface of the image sensor and the image sensor are relatively displaced. An optical shift mechanism, and the lens holder is provided with a horizontal magnet for generating the horizontal magnetic field and an axial magnet for generating the magnetic field in the optical axis direction. Applies a magnetic force to the horizontal magnet so as to control the position of the lens in the horizontal direction and a magnetic force to the axial magnet. By use, characterized in that the axial position control section for controlling the position of the lens in the optical axis direction is provided.

  As described above, according to the present invention, it is possible to perform the pixel shift control stably for a long time while maintaining high accuracy.

1 is an overall configuration diagram of a network camera system 1 according to a first embodiment. 1 is a block diagram showing a schematic configuration of an imaging device 2 and an image processing device 3 according to a first embodiment. Schematic diagram showing the processing status in the imaging device 2 and the image processing device 3 according to the first embodiment The principal part perspective view of the optical shift mechanism 35 and the magnetic levitation | floating part 50 which concerns on 1st Embodiment. The principal part top view of the optical shift mechanism 35 and the magnetic floating part 50 which concern on 1st Embodiment. The principal part perspective view of the magnetic floating part 50 which concerns on 1st Embodiment. The schematic diagram which shows an example (when the lens 51 moves to the front-back direction) of the incident state of the light to the image pick-up element 31 which concerns on 1st Embodiment. The schematic diagram which shows an example (at the time of the optical axis direction movement of the lens 51) of the incident state of the light to the image pick-up element 31 which concerns on 1st Embodiment. The schematic diagram which shows an example (filling rate 25%) of the relative movement of the pixel with respect to the optical image which concerns on 1st Embodiment The schematic diagram which shows an example (filling rate 100%) of the relative movement of the pixel with respect to the optical image which concerns on 1st Embodiment. Explanatory drawing of the rotation suppression torque which acts on the magnetic floating part 50 which concerns on 1st Embodiment. The graph which shows the relationship between the rotation angle of the magnetic levitation | floating part 50 which concerns on 1st Embodiment, and rotation suppression torque. The principal part top view of the optical shift mechanism 35 and the magnetic floating part 50 which concern on 2nd Embodiment The principal part perspective view of the optical shift mechanism 35 and the magnetic levitation | floating part 50 which concerns on 3rd Embodiment. The principal part top view of the optical shift mechanism 35 and the magnetic floating part 50 which concern on 3rd Embodiment. Partially enlarged perspective view of the optical shift mechanism 35 and the magnetic levitation unit 50 according to the third embodiment.

  A first invention made to solve the above problems is an image sensor that photoelectrically converts light from a subject to output a pixel signal, a lens that forms an image of light from the subject on the image sensor, A lens holder for holding the lens, and a light image formed on the light receiving surface of the image sensor by moving the lens in a horizontal direction perpendicular to the optical axis in a non-contact state with the lens holder, and the imaging An optical shift mechanism for relatively displacing the element, and the lens holder is provided with a horizontal magnet for generating the horizontal magnetic field and an axial magnet for generating the magnetic field in the optical axis direction. The optical shift mechanism includes a horizontal position control unit that controls the position of the lens in the horizontal direction by applying a magnetic force to the horizontal magnet, By the action of magnetic force in the axial direction the magnet, a configuration that the axial position control section for controlling the position of the lens in the optical axis direction is provided.

  According to this, since the lens can be moved in the horizontal direction perpendicular to the optical axis in a non-contact state with the optical shift mechanism, the pixel shift control can be performed stably for a long time while maintaining high accuracy. It becomes possible.

  In the second aspect of the invention, a plurality of the horizontal magnets are arranged so as to surround the outer periphery of the lens, and the horizontal position control unit is opposed to each horizontal magnet on the outside of each horizontal magnet. It is set as the structure arrange | positioned so that it may carry out.

  According to this, the lens can be moved in the horizontal direction perpendicular to the optical axis in a non-contact state with the optical shift mechanism with a simple configuration.

  According to a third aspect of the present invention, one magnetic pole surface of the horizontal magnet and one magnetic pole surface of the horizontal position control unit facing the magnetic pole surface are convex when viewed from the optical axis direction. It is set as the structure which is a circular arc surface which makes | forms, and the other is a circular arc surface which makes concave shape.

  According to this, when the rotation of the lens about the optical axis (lens axis) occurs compared to the case where the magnetic pole surface of the horizontal magnet and the horizontal position control unit is flat, the rotation direction is opposite. Since a larger rotation suppression torque can be generated in the direction, more accurate pixel shift control can be performed.

  According to a fourth aspect of the present invention, the lens holder is further provided with a horizontal position detection magnet for generating the horizontal magnetic field, and the optical shift mechanism has a magnetic field generated by the horizontal position detection magnet. By detecting this, a horizontal magnetic sensor for detecting the horizontal position of the lens is provided.

  According to this, the horizontal position of the lens can be reliably detected with a simple configuration.

  Further, the fifth invention is configured such that the plurality of horizontal position control units include an electromagnet and a permanent magnet disposed at a position facing the electromagnet via the lens.

  According to this, by properly arranging and using permanent magnets, it is possible to reduce power consumption and reduce costs without affecting pixel shift control, and it is easy to control the position of the lens in the horizontal direction. Become.

  According to a sixth aspect of the present invention, the lens holder includes an outer peripheral holding portion that holds the outer peripheral portion of the lens, and a first and second protruding portion pair that protrudes outward from the outer peripheral holding portion in the horizontal direction. In the first and second projecting portion pairs, each projecting portion is disposed at a symmetrical position with the optical axis as the center of symmetry, and is projected in the opposite direction to the horizontal direction. The magnet is disposed at each projecting portion, and generates a magnetic field in a direction perpendicular to the projecting direction of the projecting portion and perpendicular to the optical axis. The horizontal position control unit includes the first position control unit. In each of the second protruding portion pairs, a magnetic force is applied to the corresponding horizontal magnet in the same direction.

  According to this, in each projecting portion pair (horizontal magnet), the magnetic force of the horizontal position control unit in which the magnetic pole directions are arranged in parallel to each other (on two axes) is applied to the horizontal magnet. Thus, the rotation of the lens around the optical axis can be effectively suppressed, and more accurate and stable pixel shift control is possible.

  According to a seventh aspect of the present invention, each projecting portion is further provided with a horizontal position detecting magnet for generating the horizontal magnetic field, and the optical shift mechanism includes the horizontal position detecting magnet. A horizontal magnetic sensor for detecting the horizontal position of the lens by detecting the magnetic field is provided.

  According to this, the horizontal position of the lens can be detected with high accuracy by a simple configuration.

  An eighth invention is an imaging system including the imaging device according to any of the first to seventh inventions and an image processing device that performs super-resolution processing on a captured image of the imaging device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the term “axial direction” indicates the optical axis direction (vertical direction in FIG. 4), and the term “horizontal direction” includes directions orthogonal to the optical axis (front-rear direction and horizontal direction in FIG. 4). ). Moreover, in each figure, the same code | symbol is attached | subjected about the component which has the same structure and function, and a subscript (AD) is attached | subjected to the code | symbol of a number as needed, and a several component is distinguished. And

First Embodiment [k1]
FIG. 1 is an overall configuration diagram of a network camera system (imaging system) 1 according to a first embodiment of the present invention. As shown in FIG. 1, the network camera system 1 includes at least one (here, three) imaging device 2 and an image processing device 3 that performs super-resolution processing on a video (captured image) from the imaging device 2. Is done. The imaging device 2 and the image processing device 3 are connected via the Internet 4. The imaging data generated by the imaging device 2 is transmitted to, for example, an image processing device 3 as a user terminal that exists in a remote place, and is subjected to image processing there and displayed as a video. Further, various command signals for controlling the imaging operation are transmitted from the image processing device 3 to the imaging device 2.

  The imaging data is transmitted from the imaging device 2 to the image processing device 3 using a so-called Internet protocol such as TCP (UDP) / IP. For example, the imaging data is encrypted and encapsulated, for example, VPN (Vertual As a network camera system called CCTV (Closed Circuit TV) in which the imaging device 2 and the image processing device 3 are connected in a one-to-one relationship by a dedicated line. Also good.

  FIG. 2 is a block diagram illustrating a schematic configuration of the imaging device 2 and the image processing device 3 illustrated in FIG. 1. As illustrated in FIG. 2, the imaging device 2 includes an imaging unit 11, an image processing unit 12, a data compression / transmission unit 13, and a shift control unit 14. The imaging unit 11 includes an imaging element 31 that photoelectrically converts light from a subject and outputs an analog pixel signal. A two-dimensional CMOS image sensor (for example, 15 × 15 mm size) can be used for the image sensor 31.

  The image pickup device 31 is mounted on a substrate of an image sensor module (not shown), and is arranged close to the lens 51 (see FIG. 4) (here, at an interval of 1 mm). The imaging device 31 has a so-called Bayer array in which R pixels that receive an R (Red) component of incident light, B pixels that receive a B (Blue) component, and G pixels that receive a G (Green) component. 1 is a so-called single-plate color image sensor. In this Bayer array, G pixels are arranged in a zigzag pattern (checker flag shape) with the number of pixels being 1/2 of the total number of pixels, and R pixels and B pixels are G with a number of pixels that is 1/4 of the total number of pixels. The pixels are dispersedly arranged at positions other than the pixel arrangement position. The analog signal output from the image sensor 31 is converted into a digital signal by the A / D converter 32, and this digital signal is input to the image processing unit 12, where color correction, demosaic processing, gradation correction (γ correction). , YC separation processing and the like are performed and converted into image data. This image data is stored in the data compression / transmission unit 13 by, for example, H.264. The image data is transmitted to the image processing apparatus 3 after being subjected to compression processing such as H.264 or MPEG4.

  As will be described in detail later, the imaging unit 11 includes an optical shift mechanism 35 that relatively displaces the light image formed on the light receiving surface of the imaging element 31 and the imaging element 31 relatively. The optical shift mechanism 35 includes a plurality of horizontal position controllers 36 that control the horizontal position of the lens 51 (see FIG. 4), and a plurality of horizontal magnetic sensors 37 that detect the horizontal position of the lens 51. The plurality of axial position control units 38 that control the axial position of the lens 51 and the multiple axial magnetic sensors 39 that detect the axial position of the lens 51 are mainly configured. Here, the horizontal position control unit 36 and the axial position control unit 38 not only hold the lens 51 in a stationary state at a predetermined position but also drive the lens 51 in the horizontal direction or the axial direction (horizontal direction). And axial movement).

  The optical shift mechanism 35 is controlled by the shift control unit 14. Both magnetic sensors 37 and 39 detect the position information of the lens 51 and output it to the shift control unit 14. The shift control unit 14 controls the horizontal position control unit 36 and the axial position control unit 38 based on this position information to move the lens 51 in the horizontal direction and the axial direction.

  The image processing device 3 includes a data receiving / decoding unit 21, a display unit 22, a storage unit 23, a super-resolution processing unit 24, a period setting unit 25, and an input unit 26. The image processing apparatus 3 is configured by installing necessary application software in an information processing apparatus such as a personal computer or a workstation, or may be a dedicated apparatus such as a CCTV recorder.

  In the image processing device 3, the compressed image data transmitted from the imaging device 2 is received and decoded by the data receiving / decoding unit 21, converted into RGB image data, and displayed in real time by a display or the like. 22 is displayed. Further, the RGB image data is sent to the storage unit 23 composed of a hard disk drive device or the like, temporarily accumulated therein, and can be read from the storage unit 23 and reproduced on the display unit 22 as necessary.

  For example, when a high-resolution image is necessary for verification of traffic accidents, etc., the image data is read from the storage unit 23 and super-resolution processing is performed by the super-resolution processing unit 24 so that the high-resolution image is obtained. (Still image) can be generated and the high resolution image can be displayed on the display unit 22.

  Further, the input unit 26 receives an input of an imaging cycle from the user and sends this to the cycle setting unit 25. The cycle setting unit 25 determines a horizontal moving cycle based on the imaging cycle sent from the input unit 26, and transmits a command signal related to this moving cycle to the imaging device 2. The shift control unit 14 of the imaging device 2 operates the optical shift mechanism 35 based on a command signal related to the horizontal movement cycle, thereby moving the lens 51 in the horizontal direction at a movement speed corresponding to the designated movement cycle. To drive.

  Further, the cycle setting unit 25 transmits information related to the imaging timing to the drive circuit 33, whereby a timing generator (not shown) of the drive circuit 33 generates a synchronization pulse for control at a predetermined interval. This synchronization pulse is output to the image sensor 31, thereby determining the start and end of the charge accumulation period (electronic shutter period) of the image sensor 31. Further, the synchronization pulse is also output to the shift control unit 14, whereby the shift control unit 14 can grasp the period during which the image sensor 31 is actually exposed.

  FIG. 3 is a schematic diagram illustrating processing states in the imaging device 2 and the image processing device 3. As shown in FIG. 3, the image sensor 31 is driven by a drive circuit 33, and imaging (sampling) is performed at a constant cycle (hereinafter, imaging cycle) according to a timing signal generated by the drive circuit 33. For example, when generating 30 frame images per second with a frame rate of 30 frames / sec, the imaging cycle is set to about 30 ms.

  The super-resolution processing unit 24 of the image processing apparatus 3 performs super-resolution processing for generating a high-resolution image from a plurality of temporally continuous frame images. In this super-resolution processing, first, the frame image stored in the storage unit 23 is displayed as a still image by frame advance. When a reference image (reference low-resolution image) is designated by the user from among them, a frame image serving as the reference image and a plurality of frame images before and after the frame image are read out from the storage unit 23 and a super-resolution processing unit 24 to perform super-resolution processing.

  As super-resolution processing, for example, ML (Maximum-likelihood) method, MAP (Maximum A Posterior) method, POCS (Projection On to Convex Sets) method, etc. are adopted and realized by executing application software on CPU. Is done. In general, since the super-resolution processing has a large amount of calculation, a part of the processing may be performed using a GPU (Graphics Processing Unit) or dedicated hardware.

  Here, the ML method uses a square error between a pixel value of a low-resolution image estimated from a high-resolution image and an actually observed pixel value as an evaluation function, and a high resolution that minimizes this evaluation function. This is a method of using an image as an estimated image. That is, the ML method is a super-resolution processing method based on the principle of maximum likelihood estimation. The MAP method is a method for estimating a high-resolution image that minimizes an evaluation function obtained by adding probability information of a high-resolution image to a square error. That is, the MAP method is a super-resolution processing method that estimates a high-resolution image as an optimization problem that maximizes the posterior probability by using some foresight information for the high-resolution image. The POCS method is a super-resolution processing method for obtaining a high-resolution image by creating simultaneous equations regarding pixel values of a high-resolution image and a low-resolution image and sequentially solving the equations.

  In these super-resolution processes, first, a high-resolution image is assumed, and based on the point spread function (PSF function) obtained from the assumed high-resolution image from the camera model, the pixel value of the pixel of the low-resolution image is determined. And a process of searching for a high-resolution image that reduces the difference between the estimated value and the observed pixel value (observed value). Therefore, these super-resolution processes are called reconfigurable super-resolution processes.

  Here, the process of searching for a high-resolution image is to search for a position corresponding to a pixel obtained as a low-resolution image in the high-resolution image, which is called “registration”. It is processing. In general, in super-resolution processing, even if the change in pixel position between a plurality of low-resolution images is unknown, it is possible to increase the resolution, so that the alignment processing is repeatedly performed over a wide range around the pixel of interest. For this reason, it is known that the calculation cost becomes extremely high. On the other hand, as described in detail later, in the present invention, the position of the pixel shifted by the optical shift mechanism 35 is known, and each frame image, that is, a low-resolution image is captured at the known position. For at least a stationary subject, much of the alignment processing can be omitted by optical shift, and the calculation cost can be greatly reduced.

  Note that super-resolution processing that uses image information that spans a plurality of temporally continuous frames may be particularly referred to as inter-frame reconstruction type super-resolution processing. On the other hand, when the reconstruction type super-resolution processing is performed within one frame, it is called intra-frame reconstruction type super-resolution. In this embodiment, inter-frame reconstruction type super-resolution is adopted.

  Here, the still image that has been increased in resolution by super-resolution processing is reproduced in the image processing device 3, but if the processing capability of the image processing device 3 is sufficiently high, it can be obtained by super-resolution processing. It is also possible to reproduce a moving image using a high-resolution image as a frame image.

  FIGS. 4 and 5 are a perspective view and a plan view of relevant parts of the optical shift mechanism 35 and the magnetic levitation unit 50, respectively, and FIG. Here, in FIG. 4 to FIG. 6, main components inside the housing of the imaging device 2 are shown.

  The imaging apparatus 2 is provided with a magnetic levitation unit 50 that can move in the horizontal direction and the axial direction in a non-contact state with the optical shift mechanism 35. The magnetic levitation unit 50 generates a horizontal magnetic field (magnetic lines of force) by generating a horizontal magnetic field (line of magnetic force) mainly by generating a lens 51 that forms an image of light from a subject on the image sensor 31, a lens holder 52 that holds the lens 51, and the like. Detection of the horizontal position of the lens 51 in the horizontal direction (i.e., the amount of displacement from the initial position) by generating four horizontal magnets 53 </ b> A to 53 </ b> D to be used for movement of the lens 51 and mainly a horizontal magnetic field. The four horizontal position detecting magnets 54A to 54D provided for the above and the axial magnetic field are generated to move the lens 51 in the axial direction and the axial position of the lens 51 (that is, the amount of displacement from the initial position). ) Is mainly composed of four axial magnets 55A to 55D.

  Here, the optical shift mechanism 35 and the magnetic levitation unit 50 have substantially symmetrical structures in the horizontal direction (here, the front-rear direction and the left-right direction in FIG. 4) about the optical axis (lens axis) O. Each component is provided in

  The lens 51 is a biconvex lens that has a substantially square shape when viewed from the axial direction, and is formed of an optical material (such as optical glass or transparent resin). The lens 51 is composed of one lens here, but may be configured as a lens group composed of a plurality of lenses. Further, in order to appropriately form an optical image on the light receiving surface of the image sensor 31, another optical system can be provided on the subject side separately from the magnetic levitation unit 50.

  The lens holder 52 has a substantially rectangular parallelepiped shape that protrudes outward in the horizontal direction from the bottom of the four corners (notches at the four corners) of the cylinder 61 that holds the outer periphery of the lens 51. It has four projecting portions 62A to 62D. The cylindrical portion 61 has a substantially square outer shape when viewed from the optical axis O direction, and has a rectangular through hole 61a into which the lens 51 is fitted.

  The horizontal magnets 53A to 53D are permanent magnets having a substantially rectangular parallelepiped shape. In the horizontal magnets 53 </ b> A to 53 </ b> D, the inner magnetic pole surface directed inward (on the optical axis O side) is fixed to the center in the width direction (front / rear or left / right direction) on each of the front / rear and left / right sides of the cylindrical portion 61. The outer magnetic pole surface (N pole in this case) that faces the inner magnetic pole surface is directed in the front-rear and left-right directions.

  Horizontal position control units 36A to 36D are respectively provided outside the horizontal magnets 53A to 53D so as to face each other. The horizontal position control units 36 </ b> A to 36 </ b> D are electromagnets in which a coil made of a conductive wire is wound around a magnetic core formed by laminating a plurality of electromagnetic steel strips, and is attached to a casing (not shown). The inner magnetic pole surfaces (here, N poles) of the horizontal position control units 36A to 36D are arranged to face the outer magnetic pole surfaces of the horizontal magnets 53A to 53D, and thereby the horizontal position control units 36A to 36A. When a predetermined current is supplied to the coil 36D, repulsive force is generated between the horizontal magnets 53A to 53D and the horizontal position control units 36A to 36D paired therewith.

  Horizontal position detecting magnets 54A to 54D are arranged above the horizontal magnets 53A to 53D at a predetermined interval, respectively. The horizontal position detecting magnets 54A to 54D are formed of a permanent magnet having a rectangular parallelepiped shape. In the horizontal position detection magnets 54A to 54D, as in the horizontal magnets 53A to 53D, the inner magnetic pole surface is fixed to the side surface, and the outer magnetic pole surface (here, N pole) is in the front-rear and left-right directions. Is directed to.

  Outside the horizontal position detection magnets 54A to 54D, horizontal direction magnetic sensors 37A to 37D made of Hall elements are respectively provided. The horizontal magnetic sensors 37A to 37D are fixed to a mounting member (not shown), and are arranged above the horizontal position control units 36A to 36D at predetermined intervals, respectively, and outside the horizontal position detecting magnets 54A to 54D. Opposite the center of the pole face. The horizontal magnetic sensors 37A to 37D convert the magnetic fields generated by the horizontal position detecting magnets 54A to 54D, respectively, into electrical signals and output them. Thereby, the horizontal position of the lens holder 52 (lens 51) can be detected with high accuracy.

  As shown in FIG. 6, axial magnets 55 </ b> A to 55 </ b> D are attached in the projecting portions 62 </ b> A to 62 </ b> D. The axial magnets 55 </ b> A to 55 </ b> D are permanent magnets having a rectangular parallelepiped shape. Each of the axial magnets 55A to 55D has an upper magnetic pole surface directed upward and a lower magnetic pole surface directed downward. The axial magnets 55A to 55D are respectively disposed at positions obtained by rotating the horizontal position control units 36A to 36D and the horizontal magnets 53A to 53D by 45 ° with the optical axis O as the center in the plan view of FIG.

  As shown in FIG. 4, the same number of axial position control units 38 </ b> A to 38 </ b> D are provided above each of the axial magnets 55 </ b> A to 55 </ b> D. The axial position control units 38A to 38D are electromagnets similar to the horizontal position control units 36A to 36D except for some differences in shape and the like, and are attached to a housing (not shown). The lower magnetic pole surfaces (here, N poles) of the axial position control units 38A to 38D are arranged so as to face the upper magnetic pole surfaces (here, N poles) of the axial magnets 55A to 55D. When a predetermined current is supplied to the coils of the axial position control units 38A to 38D, a repulsive force is generated between them.

  Further, the same number of axial position control magnets 40A to 40D are provided below the axial magnets 55A to 55D, respectively. The axial position control magnets 40A to 40D are formed of a rectangular parallelepiped permanent magnet, and a magnetic force is set in consideration of balance with the axial position control units 38A to 38D. The axial position control magnets 40A to 40D are arranged so as to be paired with the axial position control units 38A to 38D in the vertical direction via the axial magnets 55A to 55D, respectively. The upper magnetic pole surfaces (here, S poles) of the axial position control magnets 40A to 40D are arranged to face the inner region of the lower magnetic pole surfaces (here, S poles) of the axial magnets 55A to 55D. This creates a repulsive force between them.

  Above the axial magnets 55A to 55D, axial magnetic sensors 39A to 39D comprising Hall elements are provided, respectively. The axial magnetic sensors 39A to 39D are fixed to a mounting member (not shown), are arranged at predetermined intervals outside the axial position control units 38A to 38D, and are arranged on the upper magnetic pole surface of the axial magnets 55A to 55D. Opposite to the outer area. The axial magnetic sensors 39A to 39D can detect the axial position of the lens holder 52 (lens 51) with high accuracy by converting the magnetic fields generated by the axial magnets 55A to 55D into electric signals and outputting them. Is possible.

  With the above configuration, the imaging device 2 controls the currents that flow through the coils of the four horizontal position control units 36A to 36D based on the detection results of the horizontal magnetic sensors 37A to 37D, and changes the magnitude of the repulsive force of each unit. This makes it possible to control the movement (including stillness at a predetermined position) of the magnetic levitation unit 50 (that is, the lens 51) in the horizontal direction (the front-rear direction and the left-right direction in FIG. 4). Further, by controlling the currents flowing through the coils of the four axial position control units 38A to 38D based on the detection results of the axial magnetic sensors 39A to 39D and changing the magnitude of the repulsive force of each unit, the axial direction (see FIG. 4 (in the vertical direction in FIG. 4), the movement of the magnetic levitation unit 50 (including stillness at a predetermined position) can be controlled.

  The initial position (origin position) of the lens 51 is recognized by, for example, providing a plurality of optically identified markers at appropriate positions on the lens holder 52 and detecting the markers with a reflective photosensor or the like. Can do.

  As described above, in the imaging device 2, the lens 51 can be moved in the horizontal direction perpendicular to the optical axis O in a non-contact state with the optical shift mechanism 35 (that is, a configuration in which a sliding portion or the like does not exist). Thus, it is possible to perform pixel shift control stably for a long time while maintaining high accuracy. Therefore, the imaging device 2 is suitable for application to a surveillance camera or the like. A plurality of horizontal magnets 53 are arranged so as to surround the outer periphery of the lens 51, and a plurality of horizontal position control units 36 are arranged so as to face the horizontal magnets 53 on the outside of the horizontal magnets 53. Therefore, the pixel shift control can be performed with a simple configuration.

  FIG. 7 is a schematic diagram illustrating an example of light incident state on the image sensor 31 (when the lens 51 is moved in the front-rear direction), and FIG. 8 is an example of light incident state on the image sensor 31 (of the lens 51). It is a schematic diagram which shows (at the time of an optical axis direction movement).

  As shown by a solid line and a two-dot chain line in FIG. 7, the lens 51 is moved in the front-rear direction without changing the vertical position and the horizontal position of the lens 51. The optical image (see arrow) to be formed and the image sensor 31 can be relatively slightly displaced. Although not shown, even when the lens 51 is moved in the left-right direction, it is possible to relatively slightly displace the optical image formed on the light receiving surface 31a and the image sensor 31 in the same manner.

  Further, as shown by a solid line and a two-dot chain line in FIG. 8, multi-focusing is possible by moving the lens 51 in the vertical direction without displacing the front-rear direction position and the left-right direction position of the lens 51.

  9 and 10 are schematic diagrams illustrating an example of relative movement of pixels with respect to an optical image (a filling rate of 25% and a filling rate of 100%, respectively). The relative movement of the pixel with respect to the optical image is actually a slight displacement of the optical image with respect to the fixed pixels (here, four) of the image pickup device 31. The relative movement in the horizontal direction is illustrated so that the pixel (pixel center) of the image sensor 31 moves relative to a stationary light image (one pixel of the subject). The optical image is shown in the same shape as the pixel. Here, the pixel in the high resolution space in the super-resolution processing is defined as a sub-pixel having a 1/4 pixel pitch size.

  In the example shown in FIG. 9, based on the movement of the lens 51 in the horizontal direction as described above, the pixel center moves with a movement amount of 1/4 pixel pitch in the order of front, left, and rear as indicated by arrows. . Here, four sub-pixels (hereinafter referred to as “effective sub-pixels”) filled with diagonal lines among one pixel of the subject composed of 16 sub-pixels become the movement range of the pixel center, and the pixel filling rate Here, the ratio of effective subpixels in 16 subpixels is 4/16 (25%). In this case, the twelve sub-pixels that are not filled with diagonal lines lack pixel information in the high-resolution space in the super-resolution processing.

  In the example shown in FIG. 10, with respect to the pixel center, as indicated by the arrows, the 3/4 pixel pitch moves forward, the 1/4 pixel pitch moves left, and the 3/4 pixel pitch increases. Backward movement, 1/4 pixel pitch size leftward movement, 3/4 pixel pitch size forward movement, 1/4 pixel pitch size leftward movement, 3/4 pixel pitch size The rearward movement is sequentially performed. In this case, all 16 sub-pixels filled with diagonal lines in one pixel of the subject become the movement range of the pixel center, and the pixel filling rate is 16/16 (100%). Thus, by setting the filling rate to 100%, the effect of the super-resolution processing can be enhanced. In addition, the movement path | route of a pixel center is not limited to the case shown in FIG. 10, It is good also as employ | adopting another movement path | route and making a filling rate 100%.

  Imaging by the imaging device 2 is performed while continuously moving the pixels relative to the optical image as described above at a constant speed based on a predetermined moving period. That is, a plurality of frame images that are slightly shifted at positions corresponding to the sub-pixels are sequentially generated. At this time, charge accumulation is started at a predetermined imaging reference position in the subpixel region, and the operation of completing the charge accumulation and outputting the pixel signal before the next imaging reference position is repeated.

  In the present embodiment, the degree of refraction at the front and back surfaces of the lens 51, which is a biconvex lens, varies depending on the incident position of the light. Therefore, the amount of movement of the lens 51 (magnetic levitation portion 50) depends on the pixel relative to the optical image. There is no linear relationship with the relative movement. However, if the characteristics of the lens 51 are determined so that the movement amount of the lens 51 and the relative movement amount of the pixel with respect to the optical image have linearity, the configuration of the lens 51 is complicated, but the pixel shift control is easier. There is an advantage that

  Incidentally, the relative movement of the pixel with respect to the optical image as shown in FIG. 10 is caused by the movement in the horizontal direction (front-rear direction, left-right direction) of the magnetic levitation unit 50 in a non-contact state (floating state) with the optical shift mechanism 35. Although realized, in some cases, unintended rotation of the magnetic floating part 50 (lens 51) about the optical axis O may occur. When such rotation occurs, the control error of the movement amount of the lens 51 becomes large, and it becomes difficult to perform pixel shift control with high accuracy. However, in the imaging device 2, such rotation can be suppressed by applying a rotation suppression torque to the magnetic levitation portion 50 as follows.

  FIG. 11 is an explanatory diagram of the rotation suppression torque acting on the magnetic levitation portion 50, and FIG. 12 is a graph showing the relationship between the rotation angle of the magnetic levitation portion 50 and the rotation suppression torque. FIG. 11A shows a configuration according to the present invention, and FIG. 11B is a comparative example. Here, a state is shown in which the horizontal magnet 53 (lens holder 52) is slightly rotated clockwise about the optical axis O.

  As shown in FIG. 11A, in the imaging apparatus 2 according to the present invention, the outer magnetic pole surface 65 of the horizontal magnet 53 is formed as a concave arc surface in plan view (see FIG. 5), and this is supported. The inner magnetic pole surface 66 of the horizontal position control section 36 is formed as a convex arc surface having the same curvature as the outer magnetic pole surface 65. On the other hand, in the comparative example shown in FIG. 11B, the outer magnetic pole surface 65 of the horizontal magnet 53 and the inner magnetic pole surface 66 of the horizontal position control unit 36 are formed as flat surfaces. Here, an interval between the end portions of the outer magnetic pole surface 65 and the inner magnetic pole surface 66 in an initial state where the lens holder 52 is not rotated (see FIG. 5) (interval L0 when each magnetic pole surface is flat), The interval between the outer magnetic pole surface 65 and the inner magnetic pole surface 66 before the rotation in the comparative example is set to be the same.

  In the above configuration, in FIG. 11A, the rotation of the horizontal magnet 53 about the optical axis O in the clockwise direction causes the rotation around the optical axis O by the repulsive force F1 in a counterclockwise direction so as to cancel the rotation. The rotation suppression torque (F1 × L1) is automatically applied. Similarly, in FIG. 11B, the rotation suppression torque (F2 × L2) around the optical axis O due to the repulsive force F2 is applied by the rotation of the horizontal magnet 53 so as to cancel the rotation. In this case, since distance L1> distance L2, as shown in FIG. 12, it is possible to generate a large rotation suppression torque in (A) the present invention compared to (B) the comparative example. In the comparative example, the distance between the magnetic pole faces during rotation is smaller than in the present invention, and the repulsive force F2 can be larger than the repulsive force F1, but the effect of the difference in the distances L1 and L2 on the rotation suppression torque Therefore, a large rotation suppression torque can be obtained in the configuration of the present invention.

  Thus, in the imaging device 2, one magnetic pole surface of the horizontal magnet 53 and one magnetic pole surface of the horizontal position control unit 36 facing the magnetic pole surface are convex when viewed from the optical axis O direction. Since the other arc surface is a concave arc surface and the other is a concave arc surface, the rotation of the lens around the optical axis O can be performed more easily than when the magnetic pole surfaces of the horizontal magnet and the horizontal position control unit are flat. When this occurs, a large rotation suppression torque can be generated in the direction opposite to the rotation direction, and more accurate pixel shift control can be performed. The outer magnetic pole surface 65 of each horizontal magnet 53 and the corresponding inner magnetic pole surface 66 of the horizontal position control unit 36 may have opposite configurations.

Second Embodiment
FIG. 13 is a plan view of an essential part of the optical shift mechanism 35 according to the second embodiment of the present invention, and corresponds to FIG. 5 in the first embodiment described above. In FIG. 13, the same components as those in the first embodiment described above are denoted by the same reference numerals. In the second embodiment, the detailed description is omitted as in the case of the first embodiment, except for the matters specifically mentioned below.

  As shown in FIG. 13, the optical shift mechanism 35 according to the second embodiment is different from that of the first embodiment in that the horizontal position control units 36 </ b> B and 36 </ b> C are composed of permanent magnets. That is, one of the paired horizontal position control units 36A and 36B and the horizontal position control units 36C and 36D can be formed of a permanent magnet, thereby preventing the pixel shift control from being affected. There are advantages that the power consumption can be reduced and the cost can be reduced, and that the position control of the lens in the horizontal direction is easy.

<Third Embodiment>
FIGS. 14 to 16 are a perspective view, a plan view, and a partially enlarged perspective view of the main parts of an optical shift mechanism 35 and a magnetic levitation unit 50 according to a third embodiment of the present invention, respectively. 14 to 16, the same components as those in the first embodiment are denoted by the same reference numerals. In the third embodiment, the detailed description is omitted as it is the same as the case of the first or second embodiment except for the matters specifically mentioned below.

  In the magnetic levitation unit 50, the lens holder 52 has a substantially rectangular parallelepiped shape projecting outward in the horizontal direction from the center of the four sides in the outer shape (square) in a plan view (see FIG. 15) of the cylindrical part (outer peripheral holding part) 61. There are four projecting portions 62A to 62D. Here, the pair of projecting portions 62A, 62B (first projecting portion pair) are arranged in the front-rear direction, and the other pair of projecting portions 62C, 62D (second projecting portion pair) are arranged in the left-right direction. Has been placed. In addition, since the structure (including the surrounding structure) of each protrusion part 62A-62D is the same, it demonstrates paying attention to protrusion part 62A below.

  As shown in FIGS. 14 and 16, in the projecting portion 62A, a pair of horizontal magnets 53Aa and 53Ab arranged in the front-rear direction and a pair of axial magnets 55Aa and 55Ab arranged in the vertical direction are provided. Is provided.

  A horizontal position control unit 36A is disposed on the right side of the right horizontal magnet 53Aa. The left magnetic pole surface (here, N pole) of the horizontal position control unit 36A is disposed to face the right magnetic pole surface (here, N pole) of the right horizontal magnet 53Aa, and there is a repulsive force between them. Occurs.

  A horizontal position control magnet 71A is provided on the left side of the left horizontal magnet 53Ab. The horizontal position control magnet 71A is a permanent magnet having a substantially rectangular parallelepiped shape, and a magnetic force is set in consideration of the balance with the horizontal position control unit 36A. The horizontal position control magnet 71A is disposed so as to be paired with the horizontal position control unit 36A in the left-right direction via a pair of horizontal magnets 53Aa and 53Ab. The left magnetic pole surface (here, N pole) of the left horizontal magnet 53Ab is disposed to face the right magnetic pole surface (here, N pole) of the horizontal position control magnet 71A, and there is a repulsive force between them. Occurs.

  An axial position control unit 38A is disposed above the upper axial magnet 55Aa. The lower magnetic pole surface (here, N pole) of the axial position control unit 38A is disposed to face the upper magnetic pole surface (here, N pole) of the upper axial magnet 55Aa, and between them, Repulsive force is generated.

  A shaft position control magnet 40A is provided below the lower axial magnet 55Ab. The axial position control magnet 40A is composed of a permanent magnet having a substantially rectangular parallelepiped shape, and is disposed so as to be paired with the axial position control unit 38A in the vertical direction via a pair of axial magnets 55Aa and 55Ab. The lower magnetic pole surface (here, N pole) of the lower axial magnet 55Ab is disposed so as to face the upper magnetic pole surface (here, N pole) of the axial position control magnet 40A, and between the two. Produces repulsion.

  Further, in the projecting portion 62A, a horizontal position detecting magnet 54A and an axial position detecting magnet 72A, each of which is a permanent magnet having a rectangular parallelepiped shape, are provided inside the pair of horizontal magnets 53Aa and 53Ab. .

  A horizontal magnetic sensor 37A is provided to the right of the horizontal position detecting magnet 54A. The horizontal magnetic sensor 37A is disposed behind the horizontal position control unit 36A at a predetermined interval and faces the right magnetic pole surface of the horizontal position detecting magnet 54A. Thereby, the horizontal magnetic sensor 37A can accurately detect the horizontal position of the lens holder 52 (lens 51).

  An axial magnetic sensor 39A is provided below the shaft position detection magnet 72A (see FIG. 16). The axial magnetic sensor 39A is disposed behind the shaft position control magnet 40A at a predetermined interval, and faces the lower magnetic pole surface of the shaft position detection magnet 72A. As a result, the axial magnetic sensor 39A can accurately detect the axial position of the lens holder 52 (lens 51).

  As shown in FIG. 15, in the horizontal position control units 36A and 36B corresponding to the pair of projecting portions 62A and 62B, both magnetic poles are arranged in the same direction (left-right direction). Further, the horizontal position control units 36C and 36D respectively corresponding to the other pair of projecting portions 62C and 62D rotate the horizontal position control units 36A and 36B by 90 ° clockwise around the optical axis O. And both magnetic poles are arranged in the same direction (front-rear direction).

  That is, in the first and second embodiments described above, as shown in FIGS. 5 and 13, two horizontal position control units (horizontal position control units A and B with the magnetic pole directions arranged on one axis). Alternatively, the magnetic position 50 (the lens 51) is moved in one direction by the horizontal position control units C and D). In the third embodiment, the magnetic pole directions are parallel to each other as shown in FIG. Magnetic levitation in one direction by two horizontal position control units (horizontal position control units A and B or horizontal position control units C and D) arranged on two axes and separated by a lens It is the structure which performs 50 movements. Thereby, in the third embodiment, the rotation of the lens 51 around the optical axis O can be more effectively suppressed, and more accurate and stable pixel shift control can be performed. Here, although the four projecting portions are provided, a configuration in which the number of the projecting portions is further increased is possible.

  In the third embodiment, since the rotation amount of the lens 51 around the optical axis O can be reduced, the horizontal magnetic sensor 37A is arranged in the vicinity of the horizontal position control unit 36A as compared to the first and second embodiments. It is possible to increase the detection accuracy by securing the distance from the horizontal magnetic sensor 37A to the extent that the horizontal magnetic sensor 37A is not affected by the magnetic field of the horizontal position control unit 36A. Become.

  Although the present invention has been described based on specific embodiments, these embodiments are merely examples, and the present invention is not limited to these embodiments. For example, in the above embodiment, the optical shift mechanism and the magnetic levitation unit are configured to be held in a non-contact state using the repulsive force of the magnet. However, the attractive force of the magnet may be used. Further, the configuration of the permanent magnet or the electromagnet is not limited to the above-described embodiment, and any known configuration can be appropriately employed as long as at least the same function can be achieved. Note that all the components of the imaging device according to the present invention and the imaging system including the imaging device according to the present invention described in the above embodiments are not necessarily essential, and may be appropriately selected as long as they do not depart from the scope of the present invention. Is possible.

  An imaging apparatus and an imaging system including the same according to the present invention can perform pixel shift control stably for a long time while maintaining high accuracy, and perform super-resolution from a plurality of low-resolution images acquired by pixel shift. It is useful as an imaging apparatus suitable for generating a high-resolution image by processing and an imaging system provided with the imaging apparatus.

1 Network camera system (imaging system)
2 Imaging device 3 Image processing device 31 Imaging device 35 Optical shift mechanism 51 Lens 52 Lens holders 36A to 36D Horizontal position control units 37A to 37D Horizontal magnetic sensors 38A to 38D Axial position control units 39A to 39D Axial magnetic sensor 40A to 40D Axial position control magnets 53A to 53D Horizontal magnets 54A to 54D Horizontal position detection magnets 55A to 55D Axial magnet 65 Outer magnetic pole surface (concave arc surface)
66 Inner magnetic pole surface (convex circular arc surface)

Claims (8)

An image sensor that photoelectrically converts light from a subject and outputs a pixel signal;
A lens that focuses light from the subject on the image sensor;
A lens holder for holding the lens;
By moving the lens in a horizontal direction perpendicular to the optical axis in a non-contact state with the lens holder, the optical image formed on the light receiving surface of the image sensor and the image sensor are relatively displaced. An optical shift mechanism,
The lens holder is provided with a horizontal magnet that generates the horizontal magnetic field, and an axial magnet that generates the magnetic field in the optical axis direction,
In the optical shift mechanism, by applying a magnetic force to the horizontal magnet, a horizontal position control unit that controls the position of the lens in the horizontal direction, and by applying a magnetic force to the axial magnet, An imaging apparatus comprising: an axial position control unit that controls a position of the lens in the optical axis direction. A plurality of the horizontal magnets are arranged so as to surround the outer periphery of the lens,
The imaging apparatus according to claim 1, wherein a plurality of the horizontal position control units are arranged outside the horizontal magnets so as to face the horizontal magnets.   One magnetic pole surface of the horizontal magnet and one magnetic pole surface of the horizontal position control unit opposed to the magnetic pole surface are arcuate surfaces that are convex when viewed from the optical axis direction, The imaging device according to claim 2, wherein is an arcuate surface having a concave shape. The lens holder is further provided with a horizontal position detection magnet for generating the horizontal magnetic field,
The optical shift mechanism is provided with a horizontal magnetic sensor for detecting a horizontal position of the lens by detecting a magnetic field generated by the horizontal position detecting magnet. Item 4. The imaging device according to any one of Items 3.   The imaging apparatus according to claim 2, wherein the plurality of horizontal position control units include an electromagnet and a permanent magnet disposed at a position facing the electromagnet via the lens. The lens holder includes an outer peripheral holding portion that holds an outer peripheral portion of the lens, and first and second protruding portion pairs that protrude outward from the outer peripheral holding portion in the horizontal direction.
In the first and second protruding portion pairs, each protruding portion is disposed at a symmetrical position with the optical axis as the center of symmetry, and protrudes in the opposite direction to each other.
The horizontal magnet is disposed at each projecting portion, and generates a magnetic field in a direction perpendicular to the projecting direction of the projecting portion and perpendicular to the optical axis.
The said horizontal direction control part makes a magnetic force act on the said corresponding horizontal magnet in the same direction in each of the said 1st and 2nd protrusion part pair, respectively. Imaging device. Each protruding portion is further provided with a horizontal position detecting magnet for generating the horizontal magnetic field,
7. The optical shift mechanism is provided with a horizontal magnetic sensor that detects a horizontal position of the lens by detecting a magnetic field generated by the horizontal position detecting magnet. Imaging device.   An imaging system comprising: the imaging apparatus according to any one of claims 1 to 7; and an image processing apparatus that performs super-resolution processing on a captured image of the imaging apparatus.

Images