JP2007221711A - Sensor unit, and electronic device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily accomplish an apparatus capable of performing complicated and high-load arithmetic processing for restoring blurred photographed images. <P>SOLUTION: A photographing apparatus 100 is configured by connecting a gyro unit 3 to a camera 1, comprising a photographing section 15 for photographing an image, the gyro unit 3, integrally comprising gyro sensors 32, 33 for detecting the motion of the camera 1 during photographing; an orbit transform section 35 for determining an orbit, on the basis of the detected motion; and an inverse transform filter section 36 for correcting blurring in the photographed image, on the basis of the determined orbit. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a sensor unit having a motion sensor and an electronic device connected to the sensor unit.

Conventionally, various methods for correcting a photographed image in which a blur has occurred have been proposed. As an example, there has been an apparatus that corrects a shake by detecting a shake by an angular velocity sensor to obtain a shake transfer function, and performing arithmetic processing based on the shake transfer function (see, for example, Patent Document 1). According to this apparatus, image blurring can be corrected satisfactorily by performing advanced arithmetic processing, and an image without blurring can be restored.
JP-A-6-27512

However, in the case where image blurring is corrected by arithmetic processing, including the above-described conventional apparatus, it is necessary to use hardware capable of performing high-load arithmetic processing such as obtaining a locus. In particular, in order to restore moving image blurring, it is necessary to process frames constituting the moving image one after another in a short time, and it is necessary to use hardware with high capability. Therefore, if a camera for taking an image has a function of restoring an image by the above-described method, it is necessary to prepare hardware with significantly higher performance than that for controlling the shooting operation.
Furthermore, when a sensor is used as in the above-described conventional apparatus, processing such as correction of the sensor drift is required, and high technology is required for its realization. Therefore, there has been a demand for a method for easily realizing an apparatus that can quickly restore a photographed image that is blurred.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to make it possible to easily realize an apparatus capable of performing complicated and high-load arithmetic processing to restore a shot image that has been shaken. To do.

In order to achieve the above object, the present invention provides a sensor unit configured by housing a component in a housing, wherein the component includes a motion sensor that detects a motion of the housing during photographing, and the motion There is provided a sensor unit comprising trajectory calculating means for obtaining a trajectory of the housing based on movement detected by a sensor.
According to this configuration, it is possible to detect the movement of the housing of the sensor unit and perform complicated and high-load processing for obtaining a locus. Therefore, by using this sensor unit, it is possible to correct image blurring. The necessary trajectory can be easily obtained. Further, for example, by connecting this sensor unit integrally to the photographing apparatus, the movement of the photographing apparatus at the time of photographing can be detected and the locus can be easily obtained. Therefore, by combining this sensor unit with a device capable of low-load processing, a device that restores a blurred image can be easily realized.

  Here, a configuration may be provided in which sensor output correction means for correcting the output value of the motion sensor is further provided integrally. In this case, it becomes possible to correct the drift of the sensor in the sensor unit, and the locus of the housing can be obtained accurately and stably, so that the image can be restored with high accuracy. The motion sensor may be an angular velocity sensor that outputs a voltage value corresponding to the angular velocity of the casing, and the sensor output correction unit may correct an output voltage value of the angular velocity sensor. In this case, the locus can be obtained more accurately by detecting the angular velocity.

Furthermore, it may be configured to further include a shake correction unit that corrects a shake of a captured image based on the track obtained by the track calculation unit. In this case, since the blur of the captured image can be corrected in the sensor unit, an apparatus that restores the blurred image can be realized very easily. For example, the imaging apparatus that captures an image has a function of restoring the image. It is possible to make it.
Still further, the blur correction unit may obtain a matrix for image conversion from the trajectory obtained by the trajectory calculation unit, and correct the blur of the captured image by performing an operation using the matrix. In this case, since it is possible to correct a blurred image with high accuracy by performing a high-load matrix operation in the sensor unit, it is possible to provide a function to restore the image in the other device by combining with another device. Is possible.
Further, the casing may be configured to be connectable to a photographing apparatus that captures an image. In this case, there is an advantage that the above-described sensor unit can be used in combination with the photographing apparatus. The housing may also be configured to serve as a housing for an image capturing device that captures an image. In this case, it is possible to easily realize an image capturing device that has a function of detecting the movement of the housing at the time of capturing and obtaining a locus. .

In order to solve the above-described problem, the present invention is an electronic apparatus that includes an imaging unit that captures an image, based on a motion sensor that detects a motion at the time of capturing, and a motion detected by the motion sensor. A trajectory calculating means for obtaining a trajectory, and a blur correcting means for correcting blur of an image photographed by the photographing means based on the trajectory obtained by the trajectory calculating means, are connected to a sensor unit housed in a housing. An electronic apparatus is provided that outputs a photographed image photographed by the photographing means to the sensor unit, and acquires and outputs an image corrected by the sensor unit.
According to this configuration, in an electronic device that captures an image, the captured image is output to the sensor unit, and the image corrected by the sensor unit is output without having hardware capable of performing high-load processing. It is possible to output an image whose blur is corrected with high accuracy.

  As described above, according to the above-described configuration, it is possible to very easily realize a device that detects a movement at the time of shooting, obtains a trajectory, and restores a blurred image based on the trajectory.

Embodiments of the present invention will be described below with reference to the drawings. In the present embodiment, a case where the present invention is applied to the photographing apparatus 100 will be described. The photographing apparatus 100 functions as a so-called portable digital video camera, and has a function of photographing moving images and still images and recording data of the photographed images. The photographing apparatus 100 also has a function of restoring a blur-free image by detecting a blur at the time of photographing.
FIG. 1 is a block diagram showing the configuration of the photographing apparatus 100 according to the present embodiment. As shown in FIG. 1, the photographing apparatus 100 is configured by connecting a gyro unit 3 as a sensor unit to a camera 1 as an electronic apparatus. The camera 1 included in the photographing apparatus 100 is responsible for photographing and recording an image, and the gyro unit 3 is responsible for processing for detecting a camera shake during photographing of the camera 1 and restoring the image.
The camera 1 and the gyro unit 3 are integrally connected as will be described later with reference to FIG. 6, or are housed in the same housing as described later with reference to FIG.

  The camera 1 stores a control unit 11 that controls each part of the camera 1 according to a control program 1A stored in advance in the flash ROM 13, a RAM 12 that forms a work area for storing a program executed by the control unit 11, various programs, and the like. A flash ROM 13 and an operation unit 14 including various switches for operating the camera 1 are provided. In addition, the camera 1 is photographed by the photographing unit 15 as a photographing unit for photographing a moving image and a still image including continuous frames according to the control of the photographing control unit 16, the photographing control unit 16 that controls the photographing unit 15, and the photographing unit 15. The shooting unit RAM 17 for storing the moving image and still image data, the display unit 18 for displaying the moving image and the still image based on the data stored in the shooting unit RAM 17 according to the control of the control unit 11, and the camera 1. A removable medium 20 configured to be detachable from the storage medium. Further, the camera 1 includes an interface (I / F) 19 for connecting an external device such as the gyro unit 3 and the control unit 11.

The control unit 11 reads out and executes the control program 1 </ b> A to control each unit of the camera 1 according to the operation in the operation unit 14, cause the photographing unit 15 to perform photographing, and record the photographed image data on the removable medium 20. To do.
The flash ROM 13 is configured as a rewritable nonvolatile storage medium, and stores various programs and data related to these programs in addition to the control program 1A. Here, the control program 1A stored in the flash ROM 13 is recorded on a computer-readable recording medium (not shown) such as a portable memory using a CD-ROM, a DVD-ROM, a flexible disk, or a semiconductor storage element. Can be distributed. Further, the personal computer and the camera 1 are communicably connected by a cable or the like, and a control program 1A of a recording medium (not shown) read by the personal computer is output to the camera 1, whereby the control program 1A is stored in the flash ROM 13. It can also be stored.

The imaging unit 15 includes an image sensor in which photoelectric conversion elements such as a CCD and a CMOS are arranged in a matrix or a honeycomb, an optical lens system having a plurality of optical lenses, and a zoom lens that drives the optical lens system. A lens driving device for realizing a focus, a diaphragm, and the like, an A / D conversion circuit that converts an image of an analog signal acquired by an image sensor into a digital signal, and outputs image data are configured. The photographing unit 15 has a function of photographing a moving image and a still image as described above. In the following description, a case where a moving image composed of continuous frames is photographed will be described as an example. To do.
The image capturing unit 15 sequentially outputs image data of frames in which the subject is imaged (hereinafter simply referred to as “frame”) at a predetermined sampling rate. A moving image captured by the image capturing unit 15 is, for example, It is recorded as a set of a plurality of continuous still images of 30 frames (30 fps) per second, and image processing can be performed for each frame.
Under the control of the control unit 11, the shooting control unit 16 sets shooting conditions such as exposure time and zoom magnification for the shooting unit 15, causes the shooting unit 15 to perform shooting according to the shooting conditions, and from the shooting unit 15. The output frame image data is temporarily stored in the photographing unit RAM 17. The imaging unit RAM 17 functions as a buffer for temporarily storing frames, and the imaging unit RAM 17 stores image data for each frame. The frame stored in the photographing unit RAM 17 is read by the photographing control unit 16, output to the control unit 11, and recorded on the removable medium 20 under the control of the control unit 11.

  The removable medium 20 is a rewritable recording medium, and is configured using a magnetic / optical recording medium or a semiconductor storage device, and specifically includes a video tape, a writable optical disk, a removable hard disk, and a flash memory card. Etc. The removable media 20 stores moving image data shot by the shooting unit 15 under the control of the control unit 11. Here, the moving image data stored in the removable medium 20 may be frame image data itself, or may be data encoded in a predetermined format by the control unit 11.

The gyro unit 3 is connected to the camera 1 configured as described above via an interface 19. The control unit 11 outputs the frame stored in the photographing unit RAM 17 to the gyro unit 3 via the interface 19, and causes the gyro unit 3 to perform a process of restoring the image of this frame to obtain the processed frame. To do. The processed frame is displayed on the display unit 18 and stored in the removable medium 20.
As shown in FIG. 1, the gyro unit 3 includes an interface 39 connected to the interface 19 of the camera 1. The interface 19 and the interface 39 are connected in such a manner that data can be input and output with each other. As specific connection forms, for example, connectors according to various standards are provided. These connectors are connected to each other via a connection cable (not shown), etc., and the connectors of the interfaces 19 and 39 have a shape having a male-female correspondence, and these male-female connectors are directly connected. And a form in which each of the interfaces 19 and 39 includes a transmission / reception unit that transmits and receives a radio signal using radio waves of a predetermined frequency, infrared rays, or the like, and is connected via a wireless communication line.

The gyro unit 3 includes a control unit 31 that controls each unit, an X-axis gyro sensor 32 and a Y-axis gyro sensor 33 as motion sensors that detect the motion of the camera 1 and output an angular velocity detection signal corresponding to the angular velocity. The angular velocity detection signals output from the gyro sensors 32 and 33 are adjusted, the angular velocity at which the main body of the camera 1 moves is obtained based on the angular velocity detection signals, and the moving amount (moving angle) and moving direction of the camera 1 are obtained. Is provided with an angular velocity detector 34 as sensor output correcting means.
Further, the gyro unit 3 includes a trajectory conversion unit 35 serving as a trajectory calculation unit that obtains a trajectory in which the camera 1 has moved due to camera shake during shooting based on the angular velocity, the movement amount, and the movement direction obtained by the angular velocity detection unit 34. Inverse transformation as blur compensation means for obtaining an inverse matrix for image conversion based on the trajectory obtained by the conversion unit 35 and performing an arithmetic process using the inverse matrix for image transformation to restore the image of the frame in which the blur has occurred. A filter ROM 36, a flash ROM 37 that stores programs and data required for the arithmetic processing of the inverse transform filter unit 36, and a field memory 38 that stores image data of a frame related to the arithmetic processing of the inverse transform filter unit 36 for each frame. Prepare.
The gyro unit 3 is configured so that each component shown in FIG. 1 is housed in a single housing as described later, or mounted on a single substrate. In other words, the entire gyro unit 3 is a single unit. It is a unit.
The gyro unit 3 configured as described above is electrically connected to the camera 1 via the interfaces 19 and 39 and is also physically connected to the camera 1 integrally. Therefore, the angular velocity of the camera 1 is detected by the X-axis gyro sensor 32 and the Y-axis gyro sensor 33, and the locus of movement of the camera 1 can be obtained by the angular velocity detector 34 and the locus converter 35. Yes.

FIG. 2 is a diagram showing an outline of the operation of the gyro unit 3.
As shown in FIG. 2, when an angular velocity detection signal is output from the X-axis gyro sensor 32 and the Y-axis gyro sensor 33 (step ST1), the angular velocity detection signal is input to the angular velocity detection unit 34 and adjusted. (Step ST2). The adjustment by the angular velocity detection unit 34 is a process for correcting the drift of the angular velocity detection signals output from the X-axis gyro sensor 32 and the Y-axis gyro sensor 33, and specifically does not acquire an error when the power is turned on. As described above, after a certain time (for example, about 1 second) has passed since the power of the gyro unit 3 is turned on, an angular velocity detection below a predetermined voltage value is detected in order to remove the operation of starting the acquisition of the angular velocity detection signal and the influence of dark current. For example, the operation of ignoring the signal. Further, the angular velocity detection unit 34 has a function of adjusting the angular velocity detection signal output from each gyro sensor 32, 33 in response to the bias of the signal caused by the individual difference between each gyro sensor 32, 33. The function of the angular velocity detection unit 34 stabilizes the output voltage by correcting the drift of the X-axis gyro sensor 32 and the Y-axis gyro sensor 33, so that an accurate angular velocity can be detected.
Subsequently, the angular velocity of the camera 1 is obtained by the angular velocity detector 34, and the amount and direction of movement in a predetermined time (for example, exposure time) are obtained by integrating the angular velocity. The angular velocity, the movement amount, and the moving velocity are input to the trajectory conversion unit 35, and the trajectory conversion unit 35 obtains the trajectory of the camera 1 (step ST3). Next, the trajectory conversion unit 35 generates an inverse matrix for image conversion based on the trajectory of the camera 1 (step ST4), and an arithmetic process using the inverse matrix for image conversion is performed to blur the image of the frame. It is restored to the state without (step ST5).

FIGS. 3A and 3B are diagrams showing a trajectory and a state of frame shooting. FIG. 3A shows an example of a trajectory, and FIG. 3B shows a captured frame.
In FIG. 3A, the locus of the camera 1 moving in the X-axis direction and the Y-axis direction is represented by a curve. Symbols t1 to t7 indicate times when frames are captured, and points t1 to t7 on the curve indicate positions of the camera 1 at the time of frame capturing. The process of restoring the frame is performed based on the trajectory from the previous frame shooting. For example, as shown in FIG. 3B, when the frame taken at time t3 is restored, the restoration is performed based on the trajectory of the camera 1 from time t2 to time t3 when the previous frame was taken.

Next, the process of generating the inverse matrix for image conversion in step ST4 will be described.
The inverse transform filter unit 36 obtains the shake transfer function h from the trajectory obtained by the angular velocity detection unit 34 by various methods already known from the above-mentioned Patent Document 1 (Japanese Patent Laid-Open No. 6-27512). The inverse matrix h ⁻¹ is obtained.
That is, when an image without blurring (original image) is represented by a function f (x, y) and a blurred image is represented by a function g (x, y), these h, f, and g The relationship 1) holds.

上記式（１）の両辺をフーリエ変換すれば、下記式（２）に示す関係式が得られる。

If both sides of the above formula (1) are Fourier transformed, the relational expression shown in the following formula (2) is obtained.

上記式（２）の関係式を行列で表現すると、下記式（３）〜（６）のようになる。

When the relational expression of the above expression (2) is expressed by a matrix, the following expressions (3) to (6) are obtained.

但し、下記式（４）の関係式はｍｎ×ｍｎの行列であり、式（５）〜（６）の関係式は、それぞれｍｎ次の列ベクトルである。

However, the following relational expression (4) is an mn × mn matrix, and the relational expressions (5) to (6) are mn-order column vectors.

ぶれ伝達関数ｈは、図３（Ａ）に例示したようなカメラ１の動きの軌跡から求めることができる。従って、ぶれ伝達関数ｈを求め、この逆行列ｈ^-1を求めて、上記式（３）の行列式を解くことにより、ぶれ画像ｇからぶれのない画像ｆを得ることができる。

The shake transfer function h can be obtained from the movement trajectory of the camera 1 as illustrated in FIG. Accordingly, by obtaining the blur transfer function h, obtaining the inverse matrix h ⁻¹ and solving the determinant of the above equation (3), it is possible to obtain a blur-free image f from the blur image g.

  As described above, the gyro unit 3 restores the image of each frame of the moving image taken by the camera 1 based on the angular velocity detected by the X-axis gyro sensor 32 and the Y-axis gyro sensor 33, and restores the blurred frame. It has the function to do. Therefore, in the photographing apparatus 100, since the photographed frame can be restored by the gyro unit 3 while the moving image is photographed by the camera 1, a beautiful moving image without blurring can be obtained by using the camera 1 that does not have the function of restoring the blur. Obtainable.

FIG. 4 is a flowchart showing the operation of the camera 1 during shooting.
In performing the operation shown in FIG. 4, a gyro unit 3 is connected to the camera 1. The control unit 11 of the camera 1 outputs a control signal instructing the gyro unit 3 to start operation (step S11). Subsequently, the control unit 11 determines whether or not the gyro unit 3 is in a state where a trajectory can be calculated based on a control signal or the like returned from the gyro unit 3 (step S12). Here, if the gyro unit 3 cannot calculate the trajectory (step S12; No), the control unit 11 outputs a control signal instructing the gyro unit 3 to reset (step S13), and returns to step S11. .
If the gyro unit 3 is in a state in which the trajectory can be calculated (step S12; Yes), the control unit 11 causes the shooting unit 15 to capture a moving image and is necessary for restoring the image data of the captured frame and the frame. Various information (exposure time, resolution, frame image data capacity, etc.) is output to the gyro unit 3 (step S14).

Subsequently, when the frame is restored by the gyro unit 3 and the restored frame is input from the gyro unit 3, the control unit 11 acquires the frame (step S15). Here, the control unit 11 determines whether or not the frame restoration is normally performed by the gyro unit 3 (step S16). This determination may be made, for example, by determining the sharpness of the restored image and determining whether the sharpness is lower than a predetermined level, or, as described later, frame restoration If the control signal is input from the gyro unit 3 when an error sometimes occurs, it can be determined whether the restoration has been normally performed based on whether the control signal is input.
When normal restoration is not performed by the gyro unit 3 (step S16; No), the frame before restoration, that is, the frame photographed by the photographing unit 15 is output as a photographed image to the display unit 18 and recorded on the removable medium 20. (Step S17). On the other hand, when the restoration is normally performed by the gyro unit 3 (step S16; Yes), the control unit 11 outputs the restored frame as a captured image to the display unit 18 and records it on the removable medium 20 (step S18). ).

  After outputting the captured image in step S17 or S18, the control unit 11 determines whether or not to end the shooting (step S19). If the shooting is continued, the process returns to step S12, and if the shooting is to be ended. Then, a control signal for instructing the gyro unit 3 to stop the operation is output (step S20), and the operation of FIG.

FIG. 5 is a flowchart showing the operation of the gyro unit 3 during photographing.
In performing the operation shown in FIG. 5, the gyro unit 3 is connected to the camera 1.
The control unit 31 of the gyro unit 3 waits until receiving a control signal instructing the operation start transmitted from the camera 1 (step S31). When this control signal is input, the X-axis gyro sensor 32, the Y-axis gyro Detection by the sensor 33 is started (step S32). When each of the gyro sensors 32 and 33 starts outputting the angular velocity detection signal, the angular velocity detector 34 determines whether or not the output of each gyro sensor is stable. The controller 31 waits until the angular velocity detector 34 determines that the output is stable (step S33).

After the output of the gyro sensor is stabilized, the control unit 31 acquires the movement amount and movement direction of the camera 1 obtained based on the angular velocity detection signal by the angular velocity detection unit 34 (step S34), and moves by the trajectory conversion unit 35. The locus is calculated from the amount (step S35).
Subsequently, the control unit 31 acquires information about the frame input from the camera 1 (exposure time, resolution, capacity of image data of the frame, etc.) (step S36), and obtains the acquired information and the trajectory conversion unit 35. Based on the trajectory, the inverse transform filter unit 36 generates an inverse matrix for image conversion (step S37). Then, the control unit 31 acquires the image data of the frame input from the camera 1 (step S38), performs an arithmetic process using the inverse matrix for image conversion by the inverse transform filter unit 36, and converts the image of this frame into an image. Restoration is performed (step S39). Thereafter, the control unit 31 outputs the frame restored by the inverse transform filter unit 36 to the camera 1 (step S40).
In step S39, when an error occurs during frame restoration, the control unit 31 indicates in step S40 a control signal indicating that an error has occurred during restoration in place of the restored frame or together with the restored frame. Is output from the interface 39 to the camera 1.

  The control unit 31 determines whether or not a control signal instructing to stop the operation is input from the camera 1 (step S41). The operation is stopped to stop the operation (step S42). If the control signal is not inputted, the operation returns to step S34 and the operation is continued.

FIG. 6 is an external perspective view showing an example of a specific configuration of the photographing apparatus 100. As shown in FIG. 6, the photographing apparatus 100 can be realized as a form in which, for example, a camera-equipped mobile phone 2 and an external gyro unit 41 are connected to each other.
The camera-equipped mobile phone 2 shown in FIG. 6 is positioned on the side of the display 23, a main camera 21 exposed on the back side of the foldable body, a display 23 that faces the inside of the body, and displays various images. A sub camera 22 and a key operation unit 24 disposed on a surface facing the display 23 are provided. The main camera 21 and the sub camera 22 correspond to the photographing unit 15 of the camera 1 (FIG. 1), the display 23 corresponds to the display unit 18 of the camera 1, and the key operation unit 24 corresponds to the operation unit 14 of the camera 1. It has a function. Then, the camera-equipped mobile phone 2 performs shooting of a moving image and a still image by the main camera 21 and the sub camera 22 according to an operation in the key operation unit 24 under the control of a built-in control circuit (not shown). The captured moving image and still image are displayed on the display 23. In addition, although the camera-equipped mobile phone 2 has a communication function as a mobile phone, description thereof is omitted here.

The external gyro unit 41 is obtained by housing each part of the gyro unit 3 shown in FIG. 1 in a substantially box-shaped housing. The camera-equipped mobile phone 2 and the external gyro unit 41 are connected to each other via connectors (not shown) corresponding to the interfaces 19 and 30.
In the configuration shown in FIG. 6, when a moving image or a still image is captured by the main camera 21 or the sub camera 22 included in the camera-equipped mobile phone 2, the captured moving image frame or still image is displayed in the external gyro unit 41. The restored moving image or still image can be displayed on the display 23.
The camera-equipped mobile phone 2 that captures a moving image and a still image does not have to include a sensor such as the X-axis gyro sensor 32 and the Y-axis gyro sensor 33. It is not necessary to perform complicated and high-load computation processing such as computation processing for obtaining a matrix. For this reason, the control circuit (not shown) of the camera-equipped mobile phone 2 only needs to control the operation of each part such as the main camera 21 and the sub camera 22 in addition to the communication function described above. An image can be restored without increasing the processing load on the circuit, and a beautiful image without blurring can be obtained.

Further, in the example of FIG. 6, the external gyro unit 41 externally connected to the camera-equipped mobile phone 2 may be built in the housing of the camera-equipped mobile phone 2.
FIG. 7 shows an external perspective view of another specific configuration example of the imaging apparatus 100. In the configuration example shown in FIG. 7, a gyro unit substrate 42 on which each part of the external gyro unit 41 is mounted is built in the main body of the camera-equipped mobile phone 2. The gyro unit substrate 42 is connected to the control circuit of the above-described camera-equipped mobile phone 2 via an internal connection interface (not shown) corresponding to the interfaces 19 and 39.
The configuration example of FIG. 7 has an advantage that the entire apparatus can be reduced in size. Further, since the function of the photographing apparatus 100 described above can be realized by connecting the gyro unit substrate 42 to a control circuit (not shown) of the camera-equipped mobile phone 2, a function for photographing moving images and still images is provided. This can be realized simply by additionally incorporating the gyro unit substrate 42 in the camera-equipped mobile phone. Further, on the gyro unit substrate 42, angular velocity detection is performed for drift correction of the angular velocity detection signals output from the respective gyro sensors 32 and 33, corresponding to individual differences between the X-axis gyro sensor 32 and the Y-axis gyro sensor 33. Since the unit 34 is mounted, it is not necessary to adjust the outputs of the gyro sensors 32 and 33 in the control circuit of the camera-equipped mobile phone 2. For this reason, there is an advantage that the development burden of the camera-equipped mobile phone 2 having the same function as that of the photographing apparatus 100 can be greatly reduced.

As described above, according to the embodiment to which the present invention is applied, the gyro unit 3 can detect camera shake at the time of shooting to restore the frame of the shot image, and can easily obtain a moving image without blur. It is. In this photographing apparatus 100, the gyro unit 3 performs a series of high-load processes of obtaining the trajectory of the camera 1 at the time of photographing, generating an inverse matrix for image conversion, and restoring a frame using the inverse matrix for image conversion. Is done.
That is, the photographing apparatus 100 uses the gyro unit 3 that integrally includes a functional unit that detects the movement of the camera 1 and obtains a locus, and a functional unit that restores an image by calculation with a high load. Thus, it is possible to restore the image with high accuracy. The camera 1 combined with the gyro unit 3 may be any camera as long as it has a function of capturing an image and realizing a function of inputting / outputting data to / from the gyro unit 3, and uses a general-purpose camera. It is possible. Therefore, by using the gyro unit 3, it is possible to easily realize an apparatus that restores a blurred image.
Further, since the gyro unit 3 is integrally provided with the angular velocity detection unit 34 that corrects the drift of the X-axis gyro sensor 32 and the Y-axis gyro sensor 33, the locus can be obtained accurately and stably. Image restoration can be performed with high accuracy.

  In the above-described embodiment, the case where the camera 1 and the gyro unit 3 are used to capture and restore a moving image including a plurality of frames has been described. You can restore blurred still images. That is, in the camera 1, a still image is captured by the photographing unit 15 only once, and image data of the still image is output from the camera 1 to the gyro unit 3. In the gyro unit 3, the movement of the camera 1 due to camera shake during the exposure time is detected by the X-axis gyro sensor 32, the Y-axis gyro sensor 33, and the angular velocity detection unit 34, and the locus of the camera 1 is detected by the locus conversion unit 35. Based on this locus, the inverse transform filter unit 36 obtains an inverse matrix for image conversion, restores the blurred image, and outputs the restored image to the camera 1. That is, by processing a still image in the same manner as a frame constituting a moving image, it is possible to restore a still image with blurring to an image without blurring.

  In the above embodiment, the gyro unit 3 obtains the inverse matrix for image conversion by the inverse transform filter unit 36 and performs the arithmetic processing using the inverse matrix for image transform. However, the present invention is not limited to this. For example, it is possible to perform a process of enhancing the blurring frame or the outline of a still image. This case will be described as a modified example.

FIG. 8 is a flowchart showing the operation of the gyro unit 3 in the modification of the above embodiment.
The operation shown in FIG. 8 is executed in place of the operation of FIG. 5 described above. In performing the operation of FIG. 8, the gyro unit 3 is connected to the camera 1. The control unit 31 of the gyro unit 3 waits until receiving a control signal instructing the operation start from the camera 1 (step S51). When this control signal is input, the X-axis gyro sensor 32 and the Y-axis gyro sensor 33 are input. The detection by is started (step S52). When each of the gyro sensors 32 and 33 starts outputting the angular velocity detection signal, the angular velocity detector 34 determines whether the output of each gyro sensor is stable, and the controller 31 stabilizes the output by the angular velocity detector 34. It waits until it is determined that it has been done (step S53).

After the output of the gyro sensor is stabilized, the control unit 31 acquires the movement amount and movement direction of the camera 1 obtained based on the angular velocity detection signal by the angular velocity detection unit 34 (step S54), and the locus is converted by the locus conversion unit 35. Calculate (step S55). Here, the control unit 31 controls the trajectory conversion unit 35 to obtain the blur direction and blur amount in the frame from the trajectory already obtained (step S56).
And the control part 31 acquires the image data of the flame | frame input from the camera 1 via the interface 39 (step S57), and the blurring direction of the camera 1 and the function of the control part 31 itself or another functional part are obtained. Based on the amount of blurring, outline enhancement according to the amount of blurring is performed around the direction of blurring to sharpen the image and make blurring inconspicuous (step S58). Thereafter, the control unit 31 outputs the restored frame to the camera 1 (step S59).
If an error occurs during restoration of the frame image in step S58, the control unit 31 sends a control signal indicating that an error has occurred during restoration from the interface 39 to the camera instead of the restored frame in step S59. Output to 1.
Thereafter, the control unit 31 determines whether or not a control signal instructing to stop the operation is input from the camera 1 (step S60). If this control signal is input, the X-axis gyro sensor 32 and the Y-axis gyro sensor are detected. 33 is stopped to stop the operation (step S61). If the control signal is not input, the process returns to step S54.

  In the modification shown in FIG. 8, in the gyro unit 3, the frame contour (edge) is emphasized based on the trajectory of the camera 1 instead of the calculation using the inverse matrix for image conversion. Restore the frame. Also in this case, the frame can be restored with high accuracy in the gyro unit 3 as in the above embodiment. Furthermore, it is possible to restore a frame at a higher speed than in the case of using an inverse matrix for image conversion.

It should be noted that the above-described embodiment and modification only show one aspect of the present invention, and can be arbitrarily modified and applied within the scope of the present invention.
For example, in the above-described embodiment, an example in which a frame shot by the shooting unit 15 is restored and recorded on the removable medium 20 has been described. For example, for a moving image including a plurality of frames stored in the removable medium 20 or the like, You may make it perform the said process at the time of the display (reproduction | regeneration). That is, if information such as the captured frame and the camera 1 trajectory obtained by the gyro unit 3 during exposure at the time of capturing each frame is recorded in the removable medium 20 or the like, the frame can be restored in other devices. Is possible.
Furthermore, in the above-described embodiment and modification, the frame is restored in the gyro unit 3, but the present invention is not limited to this, and the trajectory obtained by the trajectory conversion unit 35 is the gyro unit 3 May be output to another device, and the image may be restored by performing contour enhancement processing based on the trajectory in the other device (for example, the camera 1). Also in this case, the high load processing for obtaining the trajectory is executed in the gyro unit 3, so that the low load processing may be performed on the apparatus for restoring the image, and the same effect as the above embodiment can be obtained.
In the embodiment and the modification described above, the gyro unit 3 does not correct the frame and corrects the camera 1 when the movement amount of the camera 1 obtained by the angular velocity detection unit 34 is smaller than a predetermined threshold value. It is also possible to output the captured image input from the camera 1 to the camera 1 as it is. In this case, it is possible not to restore a frame that is not shaken or small enough to be visually invisible, so that the processing efficiency can be improved.

In the above-described embodiment and modification, the configuration is such that the angular velocity is detected by the X-axis gyro sensor 32 and the Y-axis gyro sensor 33 in order to detect the movement of the camera 1, but the present invention is not limited to this. For example, an acceleration sensor can be used as long as the amount of movement of the camera 1 per unit time can be detected.
Furthermore, in the camera 1 according to the above-described embodiment and modification, an external interface that is communicably connected to a personal computer via a cable or the like, or a video for outputting a video signal of a moving image to an external display device such as a television or a projector An output terminal, an audio circuit for capturing and recording / reproducing an audio signal, an audio output terminal for outputting the audio signal to an external speaker, an external amplifier, or the like may be provided.
Further, the present invention can also be applied to a photographing apparatus having a moving image photographing function other than the photographing apparatus 100 described in the above-described embodiment and modifications. To give a specific example, a moving image photographing function is provided. The present invention can also be applied to a digital still camera and various electronic devices such as a mobile phone, a PDA, and a notebook personal computer equipped with such a digital still camera.

It is a block diagram which shows the structure of the imaging device which concerns on embodiment. It is a figure which shows the outline | summary of the process which decompress | restores an image. It is a figure which shows the relationship between a locus | trajectory and the imaging | photography timing of the flame | frame to restore | restore. It is a flowchart which shows operation | movement of a camera. It is a flowchart which shows operation | movement of a gyro unit. It is a figure which shows the specific structural example of an imaging device. It is a figure which shows another specific structural example of an imaging device. It is a flowchart which shows operation | movement of the gyro unit in a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera (electronic device), 2 ... Camera-equipped mobile phone (electronic device), 3 ... Gyro unit (sensor unit), 11 ... Control part, 15 ... Shooting part (shooting means), 17 ... Shooting part RAM, 18 ... Display unit, 19 ... interface, 21 ... main camera (photographing means), 22 ... sub camera (photographing means), 23 ... display, 31 ... control unit, 32 ... X-axis gyro sensor (motion sensor), 33 ... Y-axis gyro Sensor (motion sensor) 34... Angular velocity detection unit (sensor output correction unit) 35. Trajectory conversion unit (trajectory calculation unit) 36. Inverse conversion filter unit (blur correction unit) 39 39 interface 41 external gyro Unit (sensor unit), 42... Gyro unit substrate (sensor unit), 100.

Claims (8)

A sensor unit configured by housing a component in a housing,
As the component, a motion sensor that detects the movement of the casing at the time of shooting, and a trajectory calculation unit that obtains a trajectory of the casing based on the movement of the casing detected by the motion sensor;
A sensor unit comprising:   The sensor unit according to claim 1, further comprising sensor output correction means for correcting an output value of the motion sensor. The motion sensor is an angular velocity sensor that outputs a voltage value corresponding to the angular velocity of the housing,
The sensor unit according to claim 2, wherein the sensor output correction unit corrects an output voltage value of the angular velocity sensor.   The sensor unit according to any one of claims 1 to 3, further comprising a shake correction unit that corrects a shake of a captured image based on the track obtained by the track calculation unit.   The blur correction unit obtains a matrix for image conversion from the trajectory obtained by the trajectory calculation unit, and corrects the blur of the captured image by performing an operation using the matrix. 4. The sensor unit according to 4.   5. The sensor unit according to claim 1, wherein the housing is configured to be connectable to a photographing apparatus that captures an image.   5. The sensor unit according to claim 1, wherein the housing also serves as a housing of an imaging apparatus that captures an image. An electronic device having a photographing means for photographing an image,
A motion sensor for detecting movement at the time of shooting, a trajectory calculating means for obtaining a trajectory based on the motion detected by the motion sensor, and an image photographed by the photographing means based on the trajectory obtained by the trajectory calculating means Connected to a sensor unit in which a shake correction means for correcting shake is housed in a housing,
Outputting a photographed image photographed by the photographing means to the sensor unit, and obtaining and outputting an image corrected by the sensor unit;
Electronic equipment characterized by

Images