JP5531726B2 - Camera and image processing method - Google Patents

Description

  The present invention relates to a photographing machine and an image processing method for acquiring 3D data including a plurality of images having binocular parallax.

  In order to display an object three-dimensionally, for example, a left-eye image and a right-eye image having a parallax comparable to a human binocular parallax are displayed on a display by switching each frame using a slit. Is called. Note that if the parallax between the left-eye image and the right-eye image deviates greatly from the human binocular parallax, the image may not be viewed stereoscopically or may appear unnatural.

  In order to acquire 3D data composed of a left-eye image and a right-eye image, a photographing machine (stereo camera) having two photographing units for photographing a subject may be used.

  FIG. 3 is a block diagram illustrating a configuration of the stereo camera.

  A stereo camera 100 illustrated in FIG. 3 includes imaging elements 101-1 and 101-2, lenses 102-1 and 102-2, and a storage unit 103.

  The imaging element 101-1 and the lens 102-1, the imaging element 101-2 and the lens 102-2 constitute an imaging unit, and two imaging units are provided on the left and right sides with an interval of about both human eyes. Yes. Images obtained by simultaneously photographing the subject by the two photographing units are used as a left-eye image and a right-eye image, and are stored in the storage unit 103 as 3D data.

  However, since the stereo camera has two photographing units, the cost is increased and the configuration is increased in size.

  Therefore, Patent Document 1 describes an image processing method for acquiring 3D data by a photographing machine having one photographing unit.

  The image processing method described in Patent Document 1 accumulates a large number of images continuously shot while performing panning by shaking a camera to the left and right, and is stably panned using an acceleration sensor. It is determined whether or not, and two images shot in a state where it is determined that panning is stably performed are extracted from the accumulated images, and the extracted images are set as 3D data.

JP 2009-103980 A

  However, the image processing method described in Patent Document 1 has a problem that the amount of memory increases in order to store a large number of images.

  The objective of this invention is providing the imaging device and image processing method which can solve the subject mentioned above.

In order to achieve the above object, the camera of the present invention
A camera that acquires 3D data composed of a plurality of images having binocular parallax,
A shooting section for acquiring an image of the subject;
A control unit for causing the photographing unit to perform photographing;
After the photographing by the photographing unit is performed, when the photographing machine is moved, a calculating unit that calculates a moving distance in the left-right direction from the position before the movement;
A determination unit that determines whether or not a condition that the movement distance in the left-right direction calculated by the calculation unit is within a predetermined range is satisfied,
The control unit causes the photographing unit to perform photographing of the subject in a state where the determination unit determines that the condition is satisfied, and sets the images photographed before and after movement of the photographing machine as the 3D data.

In order to achieve the above object, an image processing method of the present invention comprises:
An image processing method applied to a camera that acquires 3D data including a plurality of images having binocular parallax,
Get an image of the subject,
When the photographing machine is moved after the photographing is performed, the moving distance in the left-right direction from the point where the photographing is performed is calculated,
Determining whether or not the calculated lateral movement distance is within a predetermined range;
An image obtained by photographing the subject in a state where the movement distance in the left-right direction is determined to be within the predetermined range is acquired, and images acquired before and after the movement of the photographing apparatus are used as the 3D data.

  According to the present invention, after obtaining an image obtained by photographing a subject, the photographing device calculates a moving distance in the left-right direction of the photographing device, and images an image of the subject in a state where the moving distance is within a predetermined range. The acquired images are taken as 3D data before and after the movement.

  In this way, by taking a picture in a state in which the moving distance in the left-right direction is within a predetermined range, it is not necessary to store an image that is far from the parallax of human eyes and is not used as 3D data. Can be reduced.

It is a block diagram which shows the structure of the imaging device of one Embodiment of this invention. It is a figure for demonstrating the acquisition operation | movement of 3D data by the imaging device shown in FIG. It is a block diagram which shows the other structure of the imaging device of one Embodiment of this invention. It is a block diagram which shows the structure of a stereo camera.

  EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated with reference to drawings.

  FIG. 1 is a block diagram showing a configuration of a photographing apparatus according to an embodiment of the present invention.

  A photographing machine 10 illustrated in FIG. 1 includes a photographing element 11, a lens 12, a storage unit 13, a gyro sensor 14, an acceleration sensor 15, a calculation unit 16, a determination unit 17, and a control unit 18. .

  The imaging element 11 and the lens 12 are an example of an imaging unit.

  The image sensor 11 photoelectrically converts a light image of a subject input via the lens 12 to acquire an image.

  The accumulation unit 13 accumulates images acquired by the imaging element 11. A specific example of the storage unit 13 is an SD card.

  The gyro sensor 14 detects the angular velocity at every predetermined period when the photographing apparatus 10 moves, and outputs the result to the control unit 18.

  The acceleration sensor 15 detects acceleration at predetermined intervals when the photographing apparatus 10 moves, and outputs the result to the calculation unit 16.

  Based on the result output from the acceleration sensor 15, the calculation unit 16 calculates the movement distance in the left-right direction of the photographing machine 10 and outputs the calculated movement distance to the determination unit 17.

  The determination unit 17 determines whether or not the movement distance in the left-right direction of the photographing device 10 output from the calculation unit 16 satisfies a preset condition, and notifies the control unit 18 of the determination result.

  The control unit 18 causes the imaging element 11 to perform imaging, causes the gyro sensor 14 to detect angular velocity, and causes the acceleration sensor 15 to detect acceleration.

  Next, the acquisition operation of 3D data by the photographing machine 10 of the present embodiment will be described.

  FIG. 2 is a diagram for explaining the operation of acquiring 3D data by the photographing machine 10.

  When acquiring 3D data, the photographer sets the 3D shooting mode as an operation mode via an operation unit (not shown). In this 3D shooting mode, the photographer moves the camera 10 left and right while facing the subject after shooting is performed according to the shooting operation.

  When the 3D shooting mode is set and the photographer performs a photographing operation, the control unit 18 causes the photographing element 11 to perform photographing according to the manipulation, and the accumulation unit 13 accumulates the photographed image. Further, the control unit 18 causes the gyro sensor 14 to start detecting angular velocity and the acceleration sensor 15 to start detecting acceleration.

  The gyro sensor 14 detects the angular velocity of the photographing machine 10, and the control unit 18 obtains the inclination of the photographing machine 10 itself from the detected angular velocity.

  Further, the acceleration sensor 15 detects the acceleration in the left-right direction when the photographing machine 10 is moved, and the calculation unit 16 integrates the detected acceleration twice on the time axis and performs the left-right from the point A where the photographing was performed. The moving distance in the direction is calculated and output to the determination unit 17.

  The determination unit 17 determines whether or not the condition that the moving distance in the left-right direction of the photographing apparatus 10 output from the calculation unit 16 is within a predetermined range set in advance is satisfied, and the determination result is determined by the control unit 18. Notify The predetermined range is a range that does not deviate greatly from the distance between the human eyes.

  If there is a notification that the condition is satisfied from the determination unit 17 and the difference in tilt before and after the movement of the photographing apparatus 10 is within a predetermined range, the control unit 18 causes the photographing element 11 to perform photographing, and the accumulation unit 13. Stores 3D data of images acquired at the point A and the point B where the shooting after the movement was performed.

  Note that if the difference in the inclination of the photographing device 10 between the point A and the point B is large, the photographed image is greatly different and cannot be used as 3D data. Therefore, the control unit 18 determines that the detected inclination difference is a predetermined value. Only when it is within the range, the imaging element 11 performs imaging.

  As described above, according to the present embodiment, the photographing device 10 calculates the moving distance in the left-right direction of the photographing device 10 from the point where the subject is photographed, and photographs the subject in a state where the moving distance is within a predetermined range. The images taken before and after the movement are set as 3D data.

  Therefore, an image that is not used as 3D data greatly deviating from the parallax between human eyes is not captured, and it is not necessary to store unnecessary images, so that the amount of memory can be reduced.

  Here, the gyro sensor 14 and the acceleration sensor 15 are often already mounted in the photographing apparatus 10 such as a digital camera for camera shake correction. These sensors are installed to detect fine fluctuations by the photographer, but there are many that can detect large movements such as panning, so in this embodiment, if these sensors are used, There will be no new cost increase.

  In the present embodiment, the calculation unit 16 calculates the moving distance of the photographing apparatus 10 from the acceleration detected by the acceleration sensor 15, but the present invention is not limited to this, and for example, a high-accuracy GPS You may make it calculate the movement distance of the imaging device 10 using (Global Positioning System).

  In this case, for example, as shown in FIG. 3, instead of the acceleration sensor 15, the photographing device 10 includes a GPS receiver 21 that receives a GPS signal transmitted from a GPS satellite and a direction sensor that detects the orientation of the photographing device 10. 22 is added.

  The calculating unit 16 obtains the position of the photographing machine 10 before and after the movement based on the GPS signal received by the GPS receiving unit 21. Then, the calculation unit 16 calculates the movement distance in the left-right direction from the position before and after the movement of the photographing machine 10 with the direction orthogonal to the direction of the photographing machine 10 before the movement detected by the orientation sensor 22 as the left-right direction. .

  A specific example of the orientation sensor 22 is a geomagnetic sensor that detects a geographical orientation by detecting geomagnetism.

  In the present embodiment, the control unit 18 causes the imaging element 11 to perform imaging when the difference in inclination of the imaging device 10 before and after the movement is within a predetermined range. Therefore, this process may be omitted.

  Further, in the present embodiment, shooting was performed in a state where the moving distance in the left-right direction is within a predetermined range. However, the present invention is not limited to this, and the acceleration sensor 15 is used not only in the left-right direction but also in the front-back direction. It is also possible to detect the acceleration in the vertical direction, calculate the movement distance in the front-rear direction and the vertical direction, and perform shooting in a state where the movement distances in the front-rear direction and the vertical direction are also within a predetermined range.

  Images taken with a large distance in the front-rear and up-and-down directions may deviate from the human binocular parallax, which may cause problems such as invisibility. By taking a picture in a state within a predetermined range, such a problem can be prevented.

  Further, in the present embodiment, the control unit 18 causes the imaging element 11 to shoot the subject when notified that the condition is satisfied from the determination unit 17, but the present invention is not limited to this. . For example, when the determination unit 17 notifies that the condition is satisfied, the control unit 18 notifies the photographer by displaying on a display unit (not shown) or generating a buzzer sound by a buzzer (not shown). If it is notified that the condition is satisfied even when a photographing operation is performed by the photographer, the photographing element 11 may be caused to perform photographing.

  In addition to images captured at two points, multiple objects can be combined to display a three-dimensional display of the object with higher accuracy and view from all directions as 3D graphics. It can also be applied to systems that can

DESCRIPTION OF SYMBOLS 10 Camera 11, 101-1, 101-2 Image sensor 12, 102-1, 102-2 Lens 13, 103 Accumulation part 14 Gyro sensor 15 Acceleration sensor 16 Calculation part 17 Determination part 18 Control part 21 GPS receiving part 22 Direction Sensor 100 Stereo camera

Claims (7)

A camera that acquires 3D data composed of a plurality of images having binocular parallax,
A shooting section for acquiring an image of the subject;
An acceleration sensor for detecting acceleration during movement of the camera;
A control unit for causing the photographing unit to perform photographing;
When the photographing device is moved after photographing by the photographing unit, based on the acceleration detected by the acceleration sensor, the moving distance of the photographing device in the left-right direction from the position before the movement, and A calculation unit that calculates a vertical movement distance of the photographing device;
Whether or not the condition that the moving distance in the left-right direction calculated by the calculating unit is within a range that does not greatly deviate from the distance between both eyes of the person and that the moving distance in the up-down direction is not more than a predetermined value is satisfied. A determination unit for determining
The control unit causes the photographing unit to photograph the subject in a state where the determination unit determines that the condition is satisfied, and sets the images photographed before and after the photographing machine is moved as the 3D data. A characteristic camera. A camera that acquires 3D data composed of a plurality of images having binocular parallax,
A shooting section for acquiring an image of the subject;
An acceleration sensor for detecting acceleration during movement of the camera;
A control unit for causing the photographing unit to perform photographing;
When the photographing device is moved after photographing by the photographing unit, based on the acceleration detected by the acceleration sensor, the moving distance of the photographing device in the left-right direction from the position before the movement, and A calculation unit for calculating a moving distance in the front-rear direction of the photographing device;
Whether or not the condition that the moving distance in the left-right direction calculated by the calculating unit is within a range that does not greatly deviate from the distance between both eyes of the person and the moving distance in the front-rear direction is not more than a predetermined value is satisfied. A determination unit for determining
The control unit causes the photographing unit to photograph the subject in a state where the determination unit determines that the condition is satisfied, and sets the images photographed before and after the photographing machine is moved as the 3D data. A characteristic camera. The photographing machine according to claim 1 or 2,
The control unit causes the photographing unit to perform photographing when it is determined that the condition is satisfied by the determination unit when photographing after movement of the photographing device. The photographing machine according to any one of claims 1 to 3,
The control unit notifies the photographer at the time when the determination unit determines that the condition is satisfied, and the photographing operation is performed by the photographer after the determination when the determination unit determines that the condition is satisfied. In particular, if it is determined that the condition is satisfied, the photographing unit causes the photographing unit to perform photographing. In the imaging device according to any one of claims 1 to 4,
A gyro sensor for detecting an angular velocity during movement of the photographing machine;
The control unit detects an inclination before and after the movement of the photographing device based on an angular velocity detected by the gyro sensor, and determines that the condition satisfies the condition by the determination unit, and an inclination before and after the movement of the photographing device. When the difference is within a predetermined range, the photographing unit causes the photographing unit to perform photographing. An image processing method applied to a photographing machine that has an acceleration sensor and acquires 3D data including a plurality of images having binocular parallax,
Get an image of the subject,
When the photographing machine is moved after the photographing is performed, based on the acceleration detected by the acceleration sensor, the moving distance of the photographing machine from the point where the photographing is performed, and Calculate the vertical movement distance of the camera,
It is determined whether or not the calculated lateral movement distance is within a range that does not greatly deviate from the distance between both eyes of the person and the vertical movement distance is a predetermined value or less. ,
An image processing method characterized in that an image obtained by photographing the subject in a state in which it is determined that the condition is satisfied is acquired, and the images acquired before and after the movement of the photographing device are used as the 3D data. An image processing method applied to a photographing machine that has an acceleration sensor and acquires 3D data including a plurality of images having binocular parallax,
Get an image of the subject,
When the photographing machine is moved after the photographing is performed, the moving distance in the left-right direction of the photographing machine from the point where the photographing is performed based on the acceleration detected by the acceleration sensor; and Calculate the moving distance of the camera in the front-rear direction,
It is determined whether or not a condition in which the calculated lateral movement distance is within a range that does not greatly deviate from the distance between both eyes of the person and the longitudinal movement distance is a predetermined value or less is satisfied. ,
An image processing method characterized in that an image obtained by photographing the subject in a state in which it is determined that the condition is satisfied is acquired, and the images acquired before and after the movement of the photographing device are used as the 3D data.

Images