JP2015049446A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of accurately correcting blur of a subject.SOLUTION: An imaging device 10 and a terminal device 20 are communicable with each other. A subject carries the terminal device 20. The imaging device 10 acquires two or more pieces of position information on the terminal device 20 from the terminal device 20 before accepting a photographing instruction, calculates a direction and speed of movement of the terminal device 20 by using the two or more pieces of position information, and calculates a direction and quantity of correction by using the direction and speed of movement. When accepting the photographing instruction, the imaging device 10 corrects blur of the subject in an imaged image by using the calculated direction and quantity of correction.

Description

  The present invention relates to an imaging apparatus for imaging a subject.

  There are many opportunities to image a moving subject with an imaging device such as a camera. Conventionally, an imaging apparatus that is suitable for imaging the subject has been desired. For example, an imaging system that acquires position information of a moving subject and captures an image is known. Specifically, the following imaging system is disclosed.

  For example, Japanese Patent Laid-Open No. 2006-317848 (Patent Document 1) discusses correction of “subject blurring” caused by movement of a subject during still image shooting. More specifically, in Patent Document 1, a plurality of images are captured in advance before starting still image capturing, and a motion vector of a subject is calculated based on an analysis of the captured images. The vector is converted into the movement amount of the optical axis shift lens. Then, when shooting of a still image is started, the optical axis shift lens is moved by the previously calculated movement amount of the optical axis shift lens.

JP 2006-317848 A JP 2010-193324 A JP 2012-189827 A

  However, the conventional subject blur correction includes at least two problems. The first problem is that when there is no subject within the photographing range of the photographing apparatus until just before the still image photographing (for example, when photographing a subject moving at high speed from outside the angle of view), It is assumed that the subject blur correction cannot be handled. The second problem particularly relates to a situation assumed when the frame rate is low due to shooting in a dark place or the like. More specifically, when the frame rate is low, the shooting interval is too wide in the shooting of a plurality of images in advance (for calculating the amount of movement of the subject). For this reason, it takes a relatively long time to calculate the speed of movement of the subject. For this reason, in a situation where the speed of the subject changes, it is assumed that it is difficult to measure the speed of the subject immediately before the still image shooting.

  The present disclosure has been conceived in view of such circumstances, and an object thereof is to provide an imaging apparatus that can reliably correct subject shake.

  An imaging apparatus according to an aspect is an imaging apparatus capable of communicating with a terminal device configured to be able to acquire position information of the own apparatus, and an imaging unit for generating a captured image by photographing a subject; First position acquisition means for acquiring device position information indicating the position of the imaging device, direction acquisition means for acquiring orientation information indicating the orientation of the imaging device, and a terminal device attached to the subject. On the basis of the second position acquisition means for acquiring the terminal position information indicating the position of the subject, the apparatus position information, the terminal position information, and the orientation information. The captured image is calculated based on the calculation means for calculating the direction and distance of the subject within the unit time on the captured image and the direction and distance of the subject within the unit time on the captured image. And a correcting means for correcting the blur of that object.

  Preferably, the calculation means calculates a distance between the imaging surface of the imaging means and the subject based on the device position information and the terminal position information, and calculates a movement distance of the subject within the unit time based on the terminal position information. The distance of movement of the subject within the unit time is calculated based on the focal length of the imaging means, the distance between the imaging surface and the subject, and the movement distance of the subject within the unit time.

  Preferably, the imaging apparatus further includes information storage means for sequentially storing terminal position information for each predetermined time acquired by the second position acquisition means, and the distance of movement of the subject within the unit time is stored as information. It is calculated based on the terminal location information stored in the means.

  Preferably, the terminal position information includes information on a first position and information on a second position, which are positions of a subject before the imaging unit starts generating a captured image, and the calculation unit determines the imaging direction. On the second direction starting from the first position in the plane defined by the first direction shown and the second direction orthogonal to the first direction, and in the second direction in the plane And the virtual position of the subject indicating the position in the direction connecting the focal position of the imaging means in the plane and the distance of movement of the subject within the unit time are calculated by comparing the focal length of the imaging means and the first position. This is calculated based on the virtual position of the subject.

  Preferably, the correction unit calculates a distance at which the subject is expected to move during a period to be corrected for subject blur as an amount of subject blur correction.

  According to the present disclosure, it is possible to reliably correct subject shake in the imaging apparatus.

1 is a diagram illustrating a schematic configuration of an imaging system. It is a figure which shows typically an example of the specific condition where an imaging system is applied. It is a block diagram which shows the hardware constitutions of an imaging device. It is a block diagram which shows the hardware constitutions of a terminal device. It is a functional block diagram of an imaging device. It is a figure for demonstrating the 1st calculation method of the moving distance of the to-be-photographed object on the captured image in an imaging device. It is a figure for demonstrating the 2nd calculation method of the moving distance of the to-be-photographed object on the captured image in an imaging device. It is a figure which shows an example of the process sequence which an imaging device and a terminal device perform.

  Hereinafter, an embodiment of an information processing apparatus according to the present disclosure will be described with reference to the drawings. In the following description, parts having the same function and action are denoted by the same reference numerals throughout the drawings, and redundant description will not be repeated.

<System configuration>
An imaging system to which an embodiment of the imaging apparatus is applied will be described below. FIG. 1 is a diagram illustrating a schematic configuration of an imaging system.

  As illustrated in FIG. 1, in the imaging system, the imaging device 10 captures a subject that exists within a range FV that the imaging device 10 can capture. The range FV is also referred to as a range included in the angle of view of the imaging device 10. In FIG. 1, a subject A11 and a subject A12 represent states of the same subject (hereinafter referred to as “subject A” as appropriate) at different times. Subject A moves from the position indicated by subject A11 to the position indicated by subject A12 within the exposure time.

  The terminal device 20 is attached to the subject. The imaging device 10 acquires the position information of the terminal device 20 and the shooting direction of the imaging device 10 to obtain the direction and amount of movement of the subject within the exposure time, that is, the direction and amount of “subject blur”. calculate. Then, based on the direction and amount of the subject blur, the subject blur in the captured image is corrected. The imaging device 10 detects a shooting direction (for example, a direction in which an objective lens of a camera 122 described later is directed) by, for example, a geomagnetic sensor.

  Each of the imaging device 10 and the terminal device 20 includes an interface for communicating with each other. Communication between the imaging device 10 and the terminal device 20 may be direct such as “peer to peer”, or may be indirect such as communication via a server (not shown). In addition, each of the imaging device 10 and the terminal device 20 can acquire position information of its own device. For example, the imaging device 10 and the terminal device 20 receive radio waves for positioning from other satellites such as GPS (Global Positioning System) satellites, and each presents their current position information (position coordinates including longitude and latitude). get.

  The imaging device 10 is realized by a device having a shooting function, for example. The imaging device 10 can also be realized as a digital camera, a video camera, a PDA (Personal Digital Assistance), a tablet terminal device, or the like. The imaging device 10 may also be realized by a smartphone with an imaging function.

  The subject carries the terminal device 20. The terminal device 20 can be realized by any device as long as it can acquire position information. For example, the terminal device 20 may be a mobile phone with a GPS function, or may be built in a watch or the like worn by the subject A.

In the imaging system of FIG. 1, generally, the following controls [1] to [3] are executed.
[1] The terminal device 20 is attached to the subject in advance. Then, the terminal device 20 intermittently transmits the GPS position information of the terminal device 20 to the imaging device 10.

  [2] The imaging apparatus 10 determines the direction of movement of the subject on the imaging element (image sensor) of the imaging apparatus 10 based on the imaging direction, the GPS position information of the terminal apparatus 20, and the position information of the imaging apparatus 10. The speed is sequentially calculated and stored.

  [3] The imaging device 10 has a blur correction function. When shooting is instructed in the imaging apparatus 10 such as when a shutter button is operated, the imaging apparatus 10 corrects the captured image using the shake correction function. Specifically, the imaging apparatus 10 corrects the shake of the subject based on the movement direction and speed of the subject stored in advance according to [2].

  This embodiment does not limit the blur correction method that can be applied. For example, examples of the optical blur correction method include a method of moving an image sensor (image sensor) and a method of moving a lens. In the present embodiment, any of these methods can be employed. In the present specification, as an example, a case will be described in which a method of correcting blur by moving an image sensor is employed.

<Example of specific situation>
FIG. 2 is a diagram schematically illustrating an example of a specific situation where the imaging system is applied. The subject A01, the subject A02, the subject A11, and the subject A12 in FIG. 2 sequentially indicate the positions of the same subject at different timings. A subject A01 and a subject A02 indicate the positions of the subject A before the imaging instruction is accepted by the imaging device 10.

  In the situation shown in FIG. 2, obstacles 901 to 905 exist around the terminal device 20. The imaging apparatus 10 can only photograph the subject from the gap between the obstacles 901 to 905. That is, the range that can be imaged by the imaging apparatus 10 is narrowed by the obstacles 901 to 905. A range that can be captured by the imaging apparatus 10 is indicated by a range FV. The subject A01 and the subject A02 are located outside the range FV. For this reason, in the situation shown in FIG. 2, when shooting of the subject A is started from the position of the subject A11, it is difficult for the imaging device 10 to acquire an image of the subject A before the start of shooting.

  In the situation shown in FIG. 2, the terminal (terminal device 20) possessed by the subject A transmits the position information of the terminal device 20 to the imaging device 10 as needed. That is, even when the subject A is at the position of the subject A01 and the subsequent subject A02, the terminal device 20 transmits the position information of the terminal device 20 to the imaging device 10. “As needed” may be a given time interval, may be when the position of the terminal device 20 has changed by a predetermined distance or more, or when a request from the imaging device 10 or the server is received. It may be. The imaging device 10 calculates the direction and speed of movement of the terminal device 20 based on two or more pieces of position information transmitted from the terminal device 20.

  The imaging device 10 starts shooting when the subject reaches a position where the user wants to take a photo (subject A11 in FIG. 2). Specifically, when the user confirms that the subject A has entered the finder of the imaging device 10, the user presses the shutter button. In response to this, the imaging apparatus 10 starts exposure. When the given exposure period expires, the imaging device 10 ends the exposure. The pressing of the shutter button is an example of an imaging instruction to the imaging device 10.

  The imaging device 10 performs optical shake correction based on the movement direction and speed of the terminal device 20 calculated in advance as described above in order to correct the movement of the subject (in-screen shake). . The direction and speed of the movement are calculated based on at least two pieces of position information transmitted from the terminal device 20 at the positions of the subject A01 and the subject A02.

<Hardware configuration>
FIG. 3 is a block diagram showing a hardware configuration of imaging apparatus 10 according to the present embodiment. With reference to FIG. 3, the imaging apparatus 10 includes a central processing unit (CPU) 100, a memory 102, a touch panel 104, a display 106, a speaker 108, a button 110, and a memory interface (I / F) 112. A communication interface (I / F) 114, an electronic compass 116, an acceleration sensor 118, a GPS controller 120, and a camera 122.

  The CPU 100 controls the operation of each unit of the imaging apparatus 10 by reading and executing a program stored in the memory 102. More specifically, the CPU 100 implements each process (step) of the imaging apparatus 10 to be described later by executing the program. The CPU 100 is, for example, a microprocessor. Instead of the CPU 100, an FPGA (Field Programmable Gate Array), an ASIC (Application Spec I / Fic Integrated Circuit), and other circuits having arithmetic functions may be employed.

  The memory 102 is realized by a RAM (Random Access Memory), a ROM (Read-Only Memory), a hard disk, or the like. The memory 102 stores programs executed by the CPU 100 and / or data used by the CPU 100. The memory 102 can sequentially store subject position information acquired from the terminal device 20.

  The touch panel 104 is provided on a display 106 having a function as a display unit, and may be any type such as a resistance film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, and a capacitance method. . The touch panel 104 may include an optical sensor liquid crystal. The touch panel 104 detects a touch operation on the touch panel 104 by an external object every predetermined time, and inputs touch coordinates (touch position) to the CPU 100.

  The speaker 108 outputs sound based on a command from the CPU 100. For example, the CPU 100 causes the speaker 108 to output various notification sounds based on the audio data.

  The button 110 is arranged on the surface of the imaging device 10, receives a command from the user, outputs a signal corresponding to the command to the CPU 100, and includes, for example, a button for transitioning a display, a confirm button, a cancel button, a shutter button, , Includes camera activation button.

  The memory I / F 112 reads data from the external storage medium 113. That is, the CPU 100 reads data stored in the external storage medium 113 via the memory I / F 112 and stores the data in the memory 102. The CPU 100 reads data from the memory 102 and stores the data in the external storage medium 113 via the memory I / F 112. The storage medium 113 is, for example, a CD (Compact Disc), a DVD (Digital Versatile Disk), a BD (Blu-ray (registered trademark) Disc), a USB (Universal Serial Bus) memory, a memory card, an FD (Flexible Disk), or a hard disk. , Magnetic tape, cassette tape, MO (Magnetic Optical Disc), MD (Mini Disc), IC (Integrated Circuit) card (except memory card), optical card, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), etc. A medium for storing a program in a nonvolatile manner.

  Communication I / F 114 is realized by an antenna or a connector. The communication I / F 114 transmits and receives data to and from the terminal device 20 by wireless communication. The CPU 100 receives the current position information of the terminal device 20 from the terminal device 20 via the communication I / F 114.

  The electronic compass 116 includes a magnetic sensor, and detects the orientation of the imaging device 10 by sensing the slight magnetism of the earth. The electronic compass 116 inputs the acquired orientation (orientation) information of the imaging device 10 to the CPU 100.

  The acceleration sensor 118 is realized by, for example, a three-axis acceleration sensor of MEMS (Micro-Electro-Mechanical Systems), and detects the movement of the imaging device 10. The acceleration sensor 118 detects the moving direction and acceleration of the imaging device 10 itself. For example, the CPU 100 detects the orientation of the imaging device 10 based on the sensor signal from the acceleration sensor 118, and thereby determines the display direction of the screen displayed on the display 106. Specifically, when the imaging device 10 is in the portrait orientation (when the user is holding the imaging device 10 in the portrait orientation), the CPU 100 displays the display screen on the display 106 so as to display the same portrait orientation. Determine the direction. When the imaging device 10 is in the landscape orientation, the CPU 100 determines the display direction so that the display screen is displayed on the display 106 in the same landscape orientation. Further, the orientation of the imaging device 10 detected by the electronic compass 116 can be corrected based on the movement acquired by the acceleration sensor 118.

  The GPS controller 120 receives a GPS signal or a position signal (positioning signal) from the base station, and acquires position information of the imaging device 10. The GPS controller 120 inputs the acquired position information of the imaging device 10 to the CPU 100.

  The camera 122 includes, for example, an imaging sensor such as a CCD (Charge Coupled Device) system, a CMOS (Complementary Mental Oxide Semiconductor) sensor, an imaging lens, a lens driving unit for driving the lens, a shutter, and the like. The camera 122 generates a captured image by an imaging sensor that photoelectrically converts an image of a subject captured via a lens on a light receiving surface (imaging surface). The camera 122 has a shake correction function. The blur correction function is realized, for example, by shifting a sensor such as a CCD sensor or a CMOS sensor according to a known technique. The camera 122 may have a zoom function for changing the zoom magnification and a focus function for adjusting the focal length.

  FIG. 4 is a block diagram showing a hardware configuration of terminal device 20 according to the present embodiment. As illustrated in FIG. 4, the terminal device 20 includes a CPU 200, a memory 202, a communication I / F 204, a GPS controller 206, and a timer 208.

  The CPU 200 controls the operation of each unit of the terminal device 20 by reading and executing the program stored in the memory 202. The memory 202 is realized by a RAM, a ROM, or the like. The memory 202 stores a program executed by the CPU 200, data used by the CPU 200, and the like. The memory 202 stores, for example, position information of the terminal device 20 and terminal identification information for identifying the terminal device 20.

  The communication I / F 204 is realized by an antenna or a connector, and transmits / receives data to / from the imaging device 10 by wireless communication. The CPU 200 transmits the current position information of the terminal device 20 to the imaging device 10 as needed via the communication I / F 204. The CPU 200 may transmit terminal identification information to the imaging apparatus 10 via the communication I / F 204.

  The GPS controller 206 receives the GPS signal or the position signal from the base station and acquires the position information of the terminal device 20. The GPS controller 206 inputs the acquired position information of the terminal device 20 to the CPU 200.

  The timer 208 is a part that measures time, and sends a signal corresponding to the measured time to the CPU 200. CPU 200 can perform a predetermined process based on the signal, for example, at regular time intervals.

<Functional configuration>
FIG. 5 is a functional block diagram of imaging apparatus 10 according to the present embodiment. The imaging device 10 has, as its main functional configuration, a position acquisition unit 302, a direction acquisition unit 304, a terminal position acquisition unit 306, a correction unit 310, an output control unit 312, a calculation unit 314, and an imaging unit 320. including. These functions are realized mainly by the CPU 100 of the imaging apparatus 10 executing a program stored in the memory 102 and giving a command to the components of the imaging apparatus 10. The CPU 100 has a function as a control unit that controls the entire operation of the imaging apparatus 10. Note that some or all of these functional configurations may be realized by hardware resources. The imaging device 10 further includes an information storage unit 308 realized by the memory 102.

  The position acquisition unit 302 acquires position information of the imaging device 10 via the GPS controller 120. The orientation acquisition unit 304 acquires orientation information of the imaging device 10, specifically, orientation information indicating the imaging orientation of the camera 122 via the electronic compass 116.

  The terminal location acquisition unit 306 sequentially acquires (receives) the location information (terminal location information) of the terminal device 20 transmitted from the terminal device 20 every predetermined time (for example, every second) via the communication I / F 114. To do. Note that the terminal location acquisition unit 306 may request the terminal device 20 to transmit the terminal location information via the communication I / F 114 every predetermined time.

  The information storage unit 308 sequentially stores the terminal position information for each predetermined time acquired by the terminal position acquisition unit 306 according to the passage of time. The information storage unit 308 stores the time when the terminal position information is acquired and the terminal position information in association with each other as the position history of the terminal device 20 (subject) (terminal position information history).

  The calculation unit 314 performs imaging based on the position information of the imaging device 10 acquired by the position acquisition unit 302, the terminal position information acquired by the terminal position acquisition unit 306, and the direction information acquired by the direction acquisition unit 304. The movement distance (speed of movement) of the subject within the unit time on the captured image captured by the unit 320 is calculated. Specifically, the calculation unit 314 calculates the distance from the imaging surface of the imaging unit 320 to the subject based on the position information of the imaging device 10 and the terminal position information, and the movement distance of the subject within the unit time Is calculated. Based on the focal length of the imaging unit 320, the distance between the imaging surface and the subject, and the movement distance of the subject within the unit time, the calculation unit 314 includes the subject within the unit time on the captured image. Calculate the travel distance.

  More specifically, the calculation unit 314 has a first position of the subject (subject A01 in a plane defined by a first direction indicating the imaging direction and a second direction orthogonal to the first direction. ) Of the subject indicating the position in the second direction starting from) and indicating the position in the direction connecting the second position (subject A02) of the subject in the plane and the focal position in the plane. Calculate the position. Then, the calculation unit 314 calculates the movement distance of the subject within the unit time on the captured image based on the focal length, the position of the subject, and the virtual position of the subject. Also, the calculation unit 314 calculates the direction of movement of the subject based on these pieces of information. A method for calculating the movement distance will be described later using specific examples.

  The imaging unit 320 captures a subject and generates a captured image. Note that the imaging unit 320 is a function realized mainly by the camera 122 and the CPU 100 working together.

  The imaging unit 320 realizes a shake correction function by the correction unit 310. More specifically, in the imaging unit 320, a shake correction function is realized by continuously operating sensors such as a CCD sensor and a CMOS sensor based on a control signal from the correction unit 310. . The imaging unit 320 outputs captured image data to the output control unit 312.

  The output control unit 312 displays captured images generated by sequentially capturing images by the imaging unit 320 on a display. The display may be, for example, the touch panel 104 or an external device display.

<Calculation method>
With reference to FIG. 6, an example (first calculation method) of the calculation method of the direction and speed of movement of the subject A on the captured image will be described. The calculation unit 314 (CPU 100) mainly performs the calculation.

  FIG. 6 is a diagram for explaining a first calculation method of the moving distance of the subject A on the captured image in the imaging apparatus 10. FIG. 6 illustrates a calculation method (first calculation method) when the subject A moves in parallel with the imaging surface of the imaging device 10. FIG. 6 further shows a calculation formula for calculating the moving distance of the subject A on the captured image in the first calculation method. For ease of explanation, it is assumed that the imaging device 10 images the subject A without moving.

  In FIG. 6, the x coordinate axis is defined in the direction along the imaging surface. A y coordinate axis is defined in the imaging direction. The center (principal point) of the imaging lens is defined as the origin (0, 0). The focal position is indicated by coordinates (0, f). The position (first position) of the subject A01 is indicated by coordinates (x1, y1). The position (second position) of the subject A02 is indicated by coordinates (x2, y2). The imaging device 10 can specify each coordinate of the subject A based on the terminal position information transmitted from the terminal device 20. Each terminal location information includes information that identifies the time at which the terminal location information is transmitted from the terminal device 20.

  The CPU 100 can calculate the actual moving distance Mr within the unit time of the subject A based on the coordinates of the first position and the second position. The ratio of the actual moving distance Mr to the moving distance Mi on the captured image is equal to the ratio of the value obtained by subtracting the focal distance f from the distance D to the focal distance f (D−f). Thereby, Formula (1) of FIG. 6 is materialized. Then, equation (2) in FIG. 6 is derived from equation (1) in FIG. The distance D indicates the distance from the imaging surface to the subject A. More specifically, the distance D is a distance from a plane parallel to the imaging plane where the subject A is located to the imaging plane. In FIG. 6, the distance is represented by y coordinates (y1, y2) of the first position and the second position.

  The CPU 100 uses the equation (2) in FIG. 6 to move the subject on the captured image within the unit time based on the distance D, the focal length f, and the actual movement distance Mr within the unit time. Mi can be calculated. The CPU 100 is assumed in a period in which blur correction processing (step S26 in FIG. 8 described later) is targeted based on the moving distance Mi (speed of movement) of the subject A on the captured image within the unit time. Then, the moving distance (correction amount) of the subject A on the captured image is calculated.

  Further, the CPU 100 calculates the direction of movement of the subject A by projecting the coordinates of the first position and the second position on the imaging surface.

  Next, another example (second calculation method) of the calculation method of the direction and speed of movement of the subject A on the captured image will be described with reference to FIG. The calculation unit 314 (CPU 100) mainly performs the calculation. FIG. 7 is a diagram for explaining a second calculation method of the moving distance of the subject on the captured image in the imaging apparatus 10. FIG. 7 shows a calculation method (second calculation method) in the case where the subject A moves in a direction (oblique direction) intersecting the imaging surface of the imaging device 10. FIG. 7 further shows a calculation formula for calculating the moving distance of the subject A on the captured image in the second calculation method.

  In FIG. 7, as in FIG. 6, the x coordinate axis is defined in the direction along the imaging surface. A y coordinate axis is defined in the imaging direction. The center (principal point) of the imaging lens is defined as the origin (0, 0). The focal position is indicated by coordinates (0, f). The position (first position) of the subject A01 is indicated by coordinates (x1, y1). The position (second position) of the subject A02 is indicated by coordinates (x2, y2). The imaging device 10 can specify each coordinate of the subject A based on the terminal position information transmitted from the terminal device 20. Each terminal location information includes information that identifies the time at which the terminal location information is transmitted from the terminal device 20.

  The CPU 100 calculates the coordinates (x3, y2) of the virtual position of the subject A. The coordinates (x3, y2) are a line (y = y2) parallel to the x axis passing through the coordinates (x2, y2) of the second position and the coordinates of the first position in the xy plane shown in FIG. This is an intersection of a line connecting (x1, y1) and the coordinate (0, f) of the focal position (the direction of the line is indicated by an arrow 400 in FIG. 7).

  Here, the ratio between the x-coordinate (x1) of the first position and the x-coordinate (x3) of the virtual position is a value obtained by subtracting the focal length f from the y-coordinate (y1) of the first position (y1− It is equal to the ratio of f) to the value obtained by subtracting the focal length f from the y-coordinate (y2) of the virtual position (y2-f). For this reason, Formula (1) of FIG. 7 is materialized. Further, the ratio between the virtual moving distance Mr1 and the moving distance Mi on the captured image is equal to the ratio of the value obtained by subtracting the focal distance f from the distance D (D−f) and the focal distance f. For this reason, Formula (2) of FIG. 7 is materialized.

  Then, the CPU 100 uses the equations (1) and (2) in FIG. 7 based on the distance D (= y2), the focal length f, and the virtual moving distance Mr1 within the unit time. The movement distance Mi on the captured image within the unit time of the subject is calculated. The virtual movement distance Mr1 within the unit time is indicated by the absolute value | x2-x3 | of the difference between the x-coordinates at the second position and the virtual position. Based on the moving distance Mi (speed of movement) of the subject A within the unit time of the subject A, the CPU 100 picks up the image for each period in which the blur correction process (step S26 in FIG. 8 described later) is executed. The moving distance (correction amount) of the subject A on the image is calculated.

  Further, the CPU 100 calculates the direction of movement of the subject A by projecting the coordinates of the first position and the second position on the imaging surface.

  In this specification, in order to deepen the understanding of the calculation method of the moving distance on the captured image of the subject, first, referring to FIG. 6, the subject is moving in parallel with the imaging surface of the imaging device 10. Next, the calculation method when the subject is moving in the direction intersecting the imaging surface of the imaging device 10 has been described with reference to FIG. However, it is obvious that the calculation method shown in FIG. 7 can be applied even when the subject is moving in parallel with the imaging surface of the imaging device 10. That is, in FIG. 7, the y coordinate (y2) of the second position may be handled as the y coordinate (y1) of the first position. At this time, the virtual position is the same position as the first position.

  When the x-axis direction corresponds to the horizontal direction on the captured image, the z-axis direction orthogonal to both the x-axis direction and the y-axis direction corresponds to the vertical direction on the captured image. In this case, the same calculation method as in the case of the xy plane described above can also be used for the zy plane.

<Processing procedure>
FIG. 8 is a diagram illustrating an example of a processing procedure executed by the imaging device 10 and the terminal device 20 according to the present embodiment. Basically, among the steps in FIG. 8, the steps on the imaging device 10 side are realized by the CPU 100 executing a program stored in the memory 102, and the steps on the terminal device 20 side are performed by the CPU 200 in the memory 202. This is realized by executing the program stored in the.

  First, a processing procedure executed by the terminal device 20 will be described. CPU200 acquires the positional information (terminal positional information) of the terminal device 20 via the GPS controller 206 (step S40). The CPU 200 may always acquire the position information of the terminal device 20.

  Next, the CPU 200 transmits the acquired terminal position information to the imaging device 10 via the communication I / F 204 (step S42). CPU 200 refers to timer 208 to determine whether or not a predetermined time has elapsed (step S44). Specifically, the CPU 200 determines whether or not a predetermined time has elapsed since the terminal position information was last transmitted to the imaging device 10. If it is determined that the predetermined time has not elapsed (NO in step S44), CPU 200 repeats the process of step S44. On the other hand, when it is determined that the predetermined time has elapsed (YES in step S44), CPU 200 executes the processing after step S42. That is, the CPU 200 transmits terminal position information to the imaging device 10 at predetermined intervals via the communication I / F 204.

  Next, a processing procedure executed by the imaging apparatus 10 will be described. CPU 100 determines whether an activation instruction for camera 122 has been received via touch panel 104 (or button 110) (step S10). When CPU 100 determines that the activation instruction has not been received (NO in step S10), CPU 100 repeats the process of step S10. On the other hand, when CPU 100 determines that the activation instruction has been received (YES in step S10), control proceeds to step S12.

  In step S <b> 12, the CPU 100 receives terminal position information from the terminal device 20. Then, the CPU 100 compares the terminal position information already stored in the memory 102 with the terminal position information received in step S12. If they are different, the CPU 100 stores new terminal position information in the memory 102 (step S14). At this time, the terminal location information stored in the memory 102 until then may be deleted.

  Next, the CPU 100 acquires position information of the imaging device 10 based on the detection output of the GPS controller 120, and acquires angle information of the imaging surface of the imaging device 10 based on the detection output of the electronic compass 116 (step). S16). The angle information may be corrected based on the detection output of the acceleration sensor 118. Next, as described with reference to FIG. 6 or FIG. 7, the CPU 100 calculates the direction and speed of movement of the terminal device 20 (step S18). Next, the CPU 100 calculates the correction direction and the correction amount by using the direction and the amount calculated in step S18 (step S20). The correction direction is, for example, the direction calculated in step S18. The direction is illustrated in FIG. 7 as the direction of the arrow indicating the movement distance Mi on the captured image. The amount of correction is, for example, a distance that the subject is expected to move in a period specified by a period (predetermined period) in which step S26 described later is executed.

  Next, the CPU 100 determines whether an imaging instruction has been received (step S22). For example, when the shutter button is operated, the CPU 100 receives an imaging instruction. If CPU 100 determines that an imaging instruction has been received (YES in step S22), control proceeds to step S24. On the other hand, when CPU 100 determines that it has not received an imaging instruction (NO in step S22), it returns control to step S12.

  In step S24, the CPU 100 starts photographing (exposure). In step S26, the CPU 100 executes processing for correcting blurring of the subject in the captured image based on the correction direction and the correction amount calculated in step S20. In step S28, the CPU 100 determines whether or not a given exposure time has elapsed since the exposure was started in step S24. If CPU 100 determines that the exposure time has not yet elapsed (NO in step S28), CPU 100 returns control to step S26. On the other hand, if CPU 100 determines that the exposure time has elapsed (YES in step S28), CPU 100 advances the control to step S30. In step S30, the CPU 100 ends the exposure (shooting), outputs the captured image, and ends the control.

  In the process described above with reference to FIG. 8, the imaging device 10 acquires the position information of the terminal device 20 transmitted from the terminal device 20 before receiving the shooting instruction, and uses the position information. The direction and speed of movement of the terminal device 20 are calculated, and the direction and amount of correction are calculated using the direction and speed of the movement. When the imaging instruction is received, the imaging apparatus 10 corrects blurring of the subject in the captured image using the correction direction and amount calculated in this way.

  In the present embodiment, the direction and amount of subject blur correction are calculated based on at least two pieces of position information of the subject before the imaging instruction is accepted. The correction direction and amount are calculated after the position information is converted in such a manner that one position information in the position information used for the calculation is moved to the position of the subject at the start of exposure. It may be executed. More specifically, the CPU 100 calculates a conversion formula for converting the coordinates of the subject A01 in FIG. 3 into the coordinates of the subject A11 at the start of exposure in step S24. Then, the CPU 100 converts the coordinates of the subject A01 into the coordinates of the subject A11, and further converts the coordinates of the subject A02 using the conversion formula. Then, the CPU 100 uses the coordinates of the converted subject A01 and the coordinates of the converted subject A02 to calculate the correction direction and amount according to the method described with reference to FIG. 6 or FIG.

<Other embodiments>
In the present embodiment described above, the case where the imaging device 10 captures the subject A without moving has been described. However, the imaging device 10 may capture the subject A while moving. In this case, the imaging device 10 calculates the relative movement distance of the subject A with respect to the imaging device 10 based on the position information of the imaging device 10 and the terminal device 20, and thereby performs an appropriate correction direction for the moving subject A. And quantity can be calculated.

  Note that it is also possible to provide a program for causing a computer to execute control as described in the above flowchart. Such a program may be recorded on a non-transitory computer-readable recording medium such as a memory card attached to the computer and provided as a program product. Alternatively, the program can be provided by being recorded on a recording medium such as a hard disk built in the computer, or downloaded via a network. The program may be a program module that is provided as a part of an operating system (OS) of a computer and that calls a required module in a predetermined arrangement at a predetermined timing to execute processing. In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module may also be included in the program of the present disclosure. In addition, the program of the present disclosure may be provided by being incorporated in a part of another program.

<Effect of Embodiment>
In the present embodiment, the imaging apparatus calculates the distance and direction of subject blur by obtaining the position of the subject from a terminal device attached to the subject. Thus, the imaging apparatus can correct subject shake even when there is no subject within the imaging range of the imaging apparatus until immediately before still image shooting. In addition, the imaging apparatus corrects subject blur without taking a plurality of images in advance. This makes it possible to correct subject blur even when the frame rate is low due to shooting in a dark place or the like.

  It should be thought that embodiment disclosed this time and its modification are illustrations in all the points, and are not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  10 imaging device, 20 terminal device, 116 electronic compass, 118 acceleration sensor, 120, 206 GPS controller, 122 camera.

Claims (5)

An imaging device capable of communicating with a terminal device configured to be able to acquire position information of the own device,
Imaging means for generating a captured image by photographing a subject;
First position acquisition means for acquiring apparatus position information indicating the position of the imaging apparatus;
Orientation acquisition means for acquiring orientation information indicating the orientation of the imaging device;
Second position acquisition means for acquiring terminal position information indicating the position of the subject based on information transmitted from the terminal device attached to the subject;
Based on the device position information, the terminal position information, and the orientation information, a direction and distance of movement of the subject within the unit time on the captured image captured by the imaging unit are calculated. A calculation means;
An imaging apparatus comprising: correction means for correcting blurring of a subject in the captured image based on a moving direction and a distance of the subject in the unit time on the captured image. The calculating means includes
Based on the device position information and the terminal position information, the distance between the imaging surface of the imaging means and the subject is calculated,
Based on the terminal position information, the moving distance of the subject within a unit time is calculated,
The distance of movement of the subject within a unit time is calculated based on a focal length of the imaging unit, a distance between the imaging surface and the subject, and a movement distance of the subject within the unit time. Item 2. The imaging device according to Item 1. Further comprising information storage means for sequentially storing the terminal position information for each predetermined time acquired by the second position acquisition means;
The imaging apparatus according to claim 1, wherein a distance of movement of the subject within a unit time is calculated based on the terminal position information stored in the information storage unit. The terminal position information includes information on a first position and information on a second position, which are positions of the subject before the imaging unit starts generating a captured image,
The calculating means is on the second direction starting from the first position in a plane defined by a first direction indicating an imaging direction and a second direction orthogonal to the first direction. And calculating the virtual position of the subject indicating the position in the direction connecting the second position in the plane and the focal position of the imaging means in the plane,
The imaging apparatus according to claim 3, wherein the distance of movement of the subject within a unit time is calculated based on a focal length of the imaging unit, the first position, and a virtual position of the subject.   The correction means is expected to move the subject during a period subject to blur correction of the subject based on a distance of movement of the subject within a unit time as a correction amount of the subject blur. The imaging device according to claim 1, wherein a distance to be calculated is calculated.

Images