JP6483661B2 - Imaging control apparatus, imaging control method, and program - Google Patents

Description

  The present invention relates to an imaging control technique.

  Most of the imaging devices currently distributed are equipped with an autofocus (hereinafter referred to as “AF”) function. In AF, which is a mechanism for automatically focusing, there are methods such as contrast (light / dark difference) AF and phase difference AF. Contrast AF is known to have a demerit that it takes time to focus in order to adjust the distance between a lens and an imaging device (CCD (Charge Coupled Device), CMOS (Complementary Metal Oxide Semiconductor, etc.)). On the other hand, the phase difference AF is known to have a demerit that it is difficult to reduce the size and weight, and the phase difference AF is not currently used in an imaging device mounted on a portable terminal.

  Patent Document 1 reduces the time required for focus detection by dynamically setting the initial drive speed of the focus lens according to any of the focus lens drive direction, the focal length of the imaging optical system, and the AF scan range. An automatic focus detection apparatus and a control method thereof are proposed. Patent Document 2 proposes a method of generating a three-dimensional content to be superimposed on the real world with an image obtained by actually capturing a real object existing in the real world.

JP 2014-38291 A International Publication No. 2011/148595

  However, the method described in Patent Document 1 and Patent Document 2 and an imaging device mounted on an existing portable device have a problem that it takes time to shoot a plurality of images due to slow focusing of contrast AF. . Such a problem is likely to occur when the photographer wants to perform a large number of shootings while moving. Specifically, the following shooting scenes are exemplified.

  For example, it is a case where the subject is photographed from a plurality of different directions (front, side, rear, upper, etc.). Hereinafter, such a photographing method is referred to as multidirectional photographing. In multidirectional shooting, shooting is performed while the photographer moves around the subject with the imaging device facing the subject, so the distance between the subject and the imaging device changes as the photographer moves. To do. On the other hand, since the photographer wants to photograph a subject from a plurality of directions, the photographer desires to photograph the subject many times from a plurality of directions. As a result, a long time is required for the photographing time due to the slow focusing of the contrast AF in the imaging device of the portable device. Such multi-directional shooting is performed to three-dimensionally analyze the shape of the subject because the entire image of the subject is personally communicated to a third party. Further, as proposed in Patent Document 2, such multi-directional imaging is performed in order to generate three-dimensional content used in augmented reality (AR) (hereinafter referred to as “AR”) technology. There is a case. However, the number of shooting directions in multidirectional shooting is arbitrary.

  The present invention has been made in view of such circumstances. An object of the present invention is to provide an imaging control technique that improves the focusing speed of an imaging apparatus.

  Each aspect of the present invention employs the following configurations in order to solve the above-described problems.

  The first aspect relates to an imaging control device. An imaging control apparatus according to a first aspect includes an image acquisition unit that acquires a captured image from an imaging unit, a marker detection unit that detects a marker from a captured image acquired by the image acquisition unit, and information about the detected marker Based on the reference calculation means for calculating the shooting positional relationship indicating the positional relationship between the marker and the imaging unit, the calculated shooting positional relationship, and the object position indicating the positional relationship between the marker and the object shown in the shot image Distance calculating means for calculating the actual distance between the imaging unit and the object using the relationship, and parameter determining means for determining a focus parameter for focusing the imaging unit on the object using the calculated actual distance; Setting means for setting the focus parameter in the imaging unit.

  The second aspect relates to an imaging control method. The imaging control method according to the second aspect acquires a captured image from an imaging unit, detects a marker from the acquired captured image, and determines the positional relationship between the marker and the imaging unit based on information about the detected marker. And calculating the actual distance between the imaging unit and the object using the calculated shooting position relationship and the object position relationship indicating the positional relationship between the marker and the object shown in the shot image. Using the actual distance thus determined, determining a focus parameter for focusing the imaging unit on the object, and setting the focus parameter in the imaging unit.

  Another aspect of the present invention may be a program that causes a computer having an imaging unit to execute the method of the second aspect, or a computer-readable recording medium that records such a program. There may be. This recording medium includes a non-transitory tangible medium.

  According to each aspect described above, it is possible to provide an imaging control technique that improves the focusing speed of the imaging apparatus.

It is a figure which shows notionally the hardware constitutions of the imaging device in 1st embodiment of this invention. It is a figure which shows notionally the process structural example of the imaging device in 1st embodiment of this invention. It is a flowchart which shows the operation example of the imaging device in 1st embodiment of this invention. It is a figure which shows notionally the process structural example of the imaging device in 2nd embodiment of this invention. It is a figure which shows the example of the correspondence of a still image and direction information. It is a flowchart which shows the operation example of the imaging device in 2nd embodiment of this invention. It is a figure which shows notionally the process structural example of the imaging device in the 1st modification in 1st embodiment of this invention. It is a figure which shows notionally the process structural example of the imaging device in a 3rd modification. It is a figure which shows notionally the process structural example of the imaging device in the Example of this invention. It is a figure which shows the example of the real position relationship between a marker and an object. It is a figure which shows the example of the real position relationship between a marker and an object. It is a figure which shows the example of a display of a real-time picked-up image.

  Embodiments of the present invention will be described below. In addition, each embodiment given below is an illustration and this invention is not limited to the structure of each following embodiment. The imaging control device according to the present invention corresponds to all or part of the imaging device exemplified below.

[First embodiment]
[Hardware configuration]
FIG. 1 is a diagram conceptually illustrating a hardware configuration of the imaging apparatus according to the first embodiment.
As illustrated in FIG. 1, the imaging device 10 includes a CPU (Central Processing Unit) 1, a memory 2, a communication unit 4, a display unit 5, an input unit 6, an imaging unit 7, and the like. The CPU 1 is connected to other units via a communication line such as a bus. The memory 2 is a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, or the like. The communication unit 4 transmits and receives signals to and from other devices and devices. A portable recording medium or the like can be connected to the communication unit 4.

  The display unit 5 displays a screen corresponding to drawing data processed by a CPU 1 or a GPU (Graphics Processing Unit) (not shown) such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube) display. Is a unit. The input unit 6 is a unit that receives an input of a user operation, and is realized as a hardware button unit, a touch sensor, or the like, for example. The display unit 5 and the input unit 6 are integrated, for example, and can be realized as a touch panel. The imaging unit 7 is a camera formed from a lens, an imaging element, and the like, acquires a real-time video image frame, and captures a still image and a moving image according to an instruction.

  The imaging device 10 includes an imaging unit 7, displays an image acquired by the imaging unit 7 on the display unit 5, captures a still image and a moving image according to a user operation received by the input unit 6, and captures the captured image. Holds information representing moving images. The imaging device 10 uses focus parameters set by processing of each processing unit described later when capturing a still image and a moving image. The imaging device 10 is a device having an imaging function such as a digital camera, a mobile phone, a smart device, and the like, and its specific product form is not limited, and its hardware configuration is not limited.

[Processing configuration]
FIG. 2 is a diagram conceptually illustrating a processing configuration example of the imaging device 10 in the first embodiment. As illustrated in FIG. 2, the imaging device 10 includes an image acquisition unit 11, a marker detection unit 12, a reference calculation unit 13, a distance calculation unit 14, a parameter determination unit 15, a setting unit 16, and the like. Each of these processing units is realized, for example, as a program stored in the memory 2 and executed by the CPU 1. In addition, the program may be installed from a portable recording medium such as a CD (Compact Disc) or a memory card or another computer existing in the network via the communication unit 4 and stored in the memory 2. .

  The image acquisition unit 11 sequentially acquires real-time captured images from the imaging unit 7. For example, the image acquisition unit 11 acquires an image frame of a real-time video from the imaging unit 7 as the real-time captured image at a predetermined period.

  The marker detection unit 12 detects a marker from the real-time captured image acquired by the image acquisition unit 11. Here, the marker is an image or an object arranged in the real world and is called an AR marker or the like. However, in the present embodiment, if a certain reference point and three directions orthogonal to each other from the reference point can be constantly obtained based on this marker, regardless of the reference direction, the specific example of this marker can be obtained. The form is not limited. For example, the marker detection unit 12 holds in advance marker shape information, color information, and the like, and detects the marker from the background image based on the held information. A known image recognition method may be used for this marker detection.

The reference calculation unit 13 calculates a photographing positional relationship indicating the positional relationship between the marker and the imaging unit 7 based on the actual information and the image information regarding the marker detected by the marker detection unit 12.
The actual information regarding the marker means information in the real world of the marker, and indicates, for example, the actual shape and size of the marker. The image information related to the marker means information in the real-time captured image of the marker, and indicates the shape, size, etc. of the marker in the real-time captured image. Real information about the marker is held in advance by the reference calculation unit 13. Image information regarding the marker is acquired from the real-time captured image by the reference calculation unit 13.

  The reference calculation unit 13 can calculate the photographing positional relationship indicating the positional relationship between the marker and the imaging unit 7 by comparing the actual information regarding the marker and the image information. For example, the reference calculation unit 13 can calculate the relative virtual position of the imaging unit 7 with respect to the marker and the actual distance between the marker and the imaging unit 7 as the shooting position relationship. The method for calculating the photographing position relationship by the reference calculation unit 13 is not particularly limited as long as the actual information and the image information regarding the marker are used.

  In calculating the photographing position relationship, the reference calculation unit 13 can set a reference position corresponding to the marker position as follows. For example, the reference position is set at an arbitrary point in the marker. When the marker is composed of a plurality of partial shapes, a certain point derived from the plurality of partial shapes may be set as the reference position.

  The distance calculation unit 14 uses the imaging positional relationship calculated by the reference calculation unit 13 and the object positional relationship indicating the positional relationship between the marker and the object captured in the real-time captured image, and the reality between the imaging unit 7 and the object. Calculate the distance. Here, the object that appears in the real-time photographed image together with the marker is an object that the photographer wants to photograph and is an individual that is placed at a position close to the marker in the real world.

  The object positional relationship may indicate a relative virtual position of the object with respect to the marker, or may indicate a positional relationship in the real world between the marker and the object. The object position relationship may be stored in advance by the distance calculation unit 14, or may be calculated by the distance calculation unit 14 using other information. There may be a plurality of methods for calculating the actual distance by the distance calculation unit 14. An example of this calculation method will be described in an embodiment described later. The method for calculating the actual distance is not particularly limited.

  In calculating the actual distance between the imaging unit 7 and the object, the distance calculation unit 14 can set a reference position corresponding to the position of the object as follows. For example, the reference position is the center of gravity point, the highest point, the lowest point, the point closest to the marker, the point farthest from the marker, the point closest to the imaging unit 7, the point farthest from the imaging unit 7, etc. , It is set at an arbitrary position in the object.

The parameter determination unit 15 uses the actual distance calculated by the distance calculation unit 14 to determine a focus parameter for focusing the object on the imaging unit. Specifically, the parameter determination unit 15 determines the focus position as the focus parameter.
However, the parameter determination unit 15 can also determine an aperture value and the like for adjusting the depth of field in addition to the focus position as the focus parameter. The focus position can be represented by, for example, the distance between the lens and the image sensor, the focal length, and the like according to the configuration for focusing in the imaging unit 7. A known method can be used as the parameter determination method by the parameter determination unit 15.

  The setting unit 16 sets the focus parameter determined by the parameter determination unit 15 in the imaging unit 7.

[Operation example (imaging control method)]
Hereinafter, the imaging control method in the first embodiment will be described with reference to FIG. The imaging control method in the first embodiment includes a plurality of steps shown in steps S31 to S36 as shown in FIG.

  FIG. 3 is a flowchart illustrating an operation example of the imaging apparatus 10 according to the first embodiment. In the following description, the execution subject of each step is described as the imaging device 10, but each step is executed by the CPU 1 that is a part of the imaging device 10. Further, since each process is the same as the processing content of each processing unit described above included in the imaging apparatus 10, details of each process are omitted as appropriate.

  The image acquisition unit 11 of the imaging device 10 sequentially acquires real-time captured images from the imaging unit 7 (step S31). For example, the image acquisition unit 11 sequentially acquires real-time captured images at a predetermined cycle.

  The marker detection unit 12 of the imaging device 10 detects a marker from the real-time captured image acquired in step S31 (step S32). Step S32 may be executed every time a real-time captured image is acquired, or may be executed in a predetermined cycle asynchronously with the acquisition timing of the real-time captured image. In addition, the imaging device 10 tries to detect the marker, and does not execute step S33 and subsequent steps when the marker cannot be detected in the real-time captured image.

The reference calculation unit 13 of the imaging device 10 calculates a shooting positional relationship indicating the positional relationship between the marker and the imaging unit 7 based on the actual information and the image information regarding the marker detected in step S32 (step S33).
The distance calculation unit 14 of the imaging apparatus 10 uses the imaging positional relationship calculated in step S33 and the object positional relationship indicating the positional relationship between the marker and the object captured in the real-time captured image, and the imaging unit 7 and the object. Is calculated (step S34).

The parameter determination unit 15 of the imaging apparatus 10 determines a focus parameter for focusing the object on the imaging unit 7 using the actual distance calculated in step S34 (step S35).
The setting unit 16 of the imaging device 10 sets the focus parameter determined in step S35 in the imaging unit 7 (step S36). In accordance with the user operation received by the input unit 6, the imaging device 10 instructs the imaging unit 7 with the focus parameter set in this way to capture a still image or a moving image.

[Operation and Effect in First Embodiment]
As described above, in the first embodiment, a real-time captured image in which a marker and an object are captured is sequentially acquired, a marker is detected from the real-time captured image, and imaging is performed based on the positional relationship between the detected marker and the imaging unit 7 or the like. The actual distance between the part 7 and the object is calculated. Then, a focus parameter determined from the actual distance between the imaging unit 7 and the object is set in the imaging unit 7.

  Thus, according to the first embodiment, since the focus parameter is determined and set in the imaging unit 7 based on the marker appearing in the real-time captured image, the focus is adjusted more quickly than the existing contrast AF. It can be performed. This is because, while the existing contrast AF moves the lens, the lens position where the contrast of the image becomes the highest is found by trial and error, whereas in the first embodiment, the imaging unit 7 obtained from the marker information and the subject This is because an appropriate focus parameter can be immediately determined from the actual distance.

  Furthermore, according to the first embodiment, since a special device such as a distance sensor is not required to obtain the actual distance between the imaging unit 7 and the object (subject), the imaging apparatus 10 can be reduced in size and price. Can be realized.

[Second Embodiment]
Hereinafter, the imaging device and the imaging control method in the second embodiment will be described focusing on the contents different from those of the first embodiment described above, and the same contents will be appropriately omitted.
The second embodiment is a particularly effective form for the above-described multidirectional shooting.

[Processing configuration]
FIG. 4 is a diagram conceptually illustrating a processing configuration example of the imaging device 10a in the second embodiment. As illustrated in FIG. 4, the imaging device 10 a according to the second embodiment further includes an image storage unit 17, a direction calculation unit 18, and a capture instruction unit 19 in addition to the configuration of the first embodiment. The processing units 17, 18 and 19 are also realized by the CPU 1 executing the program in the hardware of the image apparatus 10 illustrated in FIG. The image storage unit 17 is realized in a portable recording medium connected to the memory 2 or the communication unit 4.

  The direction calculation unit 18 calculates the shooting direction of the imaging unit 7 with respect to the object using the shooting position relationship calculated by the reference calculation unit 13 and the object position relationship. The object positional relationship may be stored in advance by the direction calculation unit 18 or may be calculated by the direction calculation unit 18 using other information. For example, the direction calculation unit 18 can calculate the positional relationship between the object and the imaging unit 7 based on the shooting positional relationship and the object positional relationship, and can calculate the shooting direction using this positional relationship. . However, the method for calculating the shooting direction is not limited.

  The image storage unit 17 stores the still image captured by the imaging unit 7 in association with the direction information corresponding to the shooting direction of the imaging unit 7 calculated by the direction calculation unit 18 when capturing the still image. . For the direction information associated with the still image, the shooting direction of the imaging unit 7 calculated by the direction calculation unit 18 may be used as it is, or direction information obtained from the shooting direction may be used. For example, the direction information may be information indicating any one of the front side, the right side, the left side, the rear, and the upper side.

  FIG. 5 is a diagram illustrating an example of a correspondence relationship between a still image and direction information. In FIG. 5, the direction information is represented by being arranged in a two-axis coordinate space of a horizontal direction and a vertical direction. The horizontal direction in the direction information corresponds to one axial direction that forms a plane on which the object is placed. The vertical direction corresponds to one axial direction perpendicular to the plane.

  In the example of FIG. 5, the direction information associated with the still image includes the vertical direction line Lv and the horizontal direction line on the spherical surface of the hemisphere obtained by cutting a sphere centered on the reference point of the object along the reference plane on which the object is placed. It is shown as information for identifying the region R surrounded by Lh. The vertical direction line Lv is an intersection line of the spherical surface of the hemisphere and a plurality of planes provided at a predetermined angle interval perpendicular to the reference plane and passing through the center point, and the horizontal direction line Lh is the reference line of the hemisphere and the reference plane. These are intersection lines with a plurality of planes provided at predetermined intervals in a direction parallel to the plane and perpendicular to the reference plane. The predetermined angular interval for determining the vertical line Lv and the predetermined interval for determining the horizontal line Lh can be arbitrarily determined.

  The image acquisition unit 11 acquires the still image captured by the imaging unit 7 and stores the still image in the image storage unit 17 in association with the direction information corresponding to the shooting direction of the imaging unit 7 calculated at the time of capture. To do. The image acquisition unit 11 can directly use the shooting direction calculated by the direction calculation unit 18 as direction information, or can generate direction information based on the shooting direction. For example, when the direction information indicates any one of front, right side, left side, rear, and top, the image acquisition unit 11 determines that the shooting direction calculated by the direction calculation unit 18 is the front, right side, or left side. , It is determined which one belongs to the rear and the upper, and direction information indicating any one direction is generated. When the direction information is the identification information of the region R as described above, the image acquisition unit 11 identifies the region R to which the shooting direction belongs, and identifies the identification information of the identified region R as the direction information. And

  The capture instruction unit 19 determines whether to instruct capture of a still image according to the shooting direction of the imaging unit 7 calculated by the direction calculation unit 18, and based on the determination result, the capture instruction unit 19 Instruct to capture image. In other words, the capture instruction unit 19 instructs capture so that a still image from a shooting direction that is not yet stored in the image storage unit 17 is obtained, and the shooting direction calculated by the direction calculation unit 18 is already stationary. When the shooting direction in which the image is stored is used, capture is not instructed. For example, the capture instruction unit 19 is based on the relationship between the shooting direction of the imaging unit 7 newly calculated by the direction calculation unit 18 and the direction information associated with the still image already stored in the image storage unit 17. It is possible to determine whether or not to instruct capture. The imaging unit 7 receives an instruction from the capture instruction unit 19 and captures a still image using the focus parameter set at that time.

[Operation example (imaging control method)]
Hereinafter, the imaging control method in the second embodiment will be described with reference to FIG. The imaging control method in the second embodiment includes a plurality of steps shown in steps S51 to S58 as shown in FIG. The imaging control method in the second embodiment is executed synchronously or asynchronously with the imaging control method in the first embodiment shown in FIG.

  FIG. 6 is a flowchart illustrating an operation example of the imaging apparatus 10 according to the second embodiment. In the following description, the execution subject of each step is described as the imaging device 10, but each step is executed by the CPU 1 that is a part of the imaging device 10. In addition, since each process is the same as the processing content of each processing unit described above that the imaging apparatus 10 has, details of each process are omitted as appropriate.

  The imaging device 10a calculates the imaging direction of the imaging unit 7 with respect to the object using the imaging positional relationship calculated in step S33 shown in FIG. 3 and the object positional relationship indicating the positional relationship between the marker and the object ( Step S51).

  If there is no still image stored in the image storage unit 17 (step S52; NO), the imaging device 10a executes step S56. On the other hand, when there is a still image stored in the image storage unit 17 (S52; YES), the imaging device 10 stores the shooting direction of the imaging unit 7 calculated in step S51 and the image storage unit 17. It is determined whether or not to instruct capture based on the relationship between the still image and the direction information associated with the still image (step S53). In other words, in step S53, the imaging apparatus 10a determines whether or not the still image from the shooting direction calculated in step S51 is already stored in the image storage unit 17. Here, when determining whether or not to instruct capture, the imaging apparatus 10a newly acquires an already stored image, not whether or not a still image stored in the image storage unit 17 exists. The captured images may be compared, and if it is determined that the state is better, a capture instruction may be issued to overwrite.

If the imaging apparatus 10a determines not to instruct capture (step S55; NO), the imaging apparatus 10a ends the process without instructing capture. This is because the still image from the shooting direction calculated in step S51 is already stored in the image storage unit 17.
On the other hand, when the imaging device 10a determines to instruct capture (step S55; YES), the imaging device 10a instructs the imaging unit 7 to capture a still image (step S56). This is because the still image from the shooting direction calculated in step S51 is not yet stored in the image storage unit 17. In response to this instruction, the imaging unit 7 captures a still image captured using the focus parameter set in step S36 illustrated in FIG.

The imaging device 10a acquires the captured still image from the imaging unit 7 (step S57).
The imaging device 10a stores the still image acquired in step S57 in the image storage unit 17 in association with the shooting direction of the imaging unit 7 calculated in step S51 (step S58).

  Step S51 may be executed every time the photographing positional relationship is calculated in step S33, or may be executed asynchronously with the flow shown in FIG. When executed asynchronously, for example, the imaging apparatus 10a detects that a predetermined user operation (a pressing operation of a button made of hardware or software, a touch operation, a gesture operation, a voice operation, or the like) is detected by the input unit 6. In response to this, step S51 can be executed. Further, step S51 may be executed after step S33 in the flow shown in FIG. 3, and may be deleted from the flow shown in FIG. In this case, the imaging control method in the second embodiment is executed from step S52.

[Operation and Effect in Second Embodiment]
In the second embodiment, the shooting direction of the imaging unit 7 with respect to the object shown in the real-time shot image is calculated, and the relationship between the direction information associated with the still image stored in the image storage unit 17 and the calculated shooting direction. Based on this, it is determined whether or not an instruction to capture a still image is given.

  Thereby, according to the second embodiment, still images can be captured when still images from the current shooting direction have not yet been stored. As a result, duplicate still images from the same direction are acquired. Can be avoided. In addition, the image stored in advance is compared with the newly acquired image, and if it is determined to be in a better state, it can be captured by overwriting, and as a result, a still image with better quality can be obtained. . Therefore, according to the second embodiment, the user can efficiently perform multidirectional shooting.

[Modification of First Embodiment (First Modification)]
FIG. 7 is a diagram conceptually illustrating a processing configuration example of the imaging device 10b in the first modification. As illustrated in FIG. 7, the imaging apparatus 10 b in the first embodiment described above captures a capture image 19 that instructs the imaging unit 7 to capture a still image or a moving image using the focus parameter set by the setting unit 16. May further be included. In this case, the capture instruction unit 19 may instruct the imaging unit 7 to capture immediately after the setting unit 16 sets the focus parameter in the imaging unit 7. Further, the processing unit in which the setting unit 16 and the capture instruction unit 19 are unified designates the focus parameter determined by the parameter determination unit 15 and instructs the imaging unit 7 to capture a still image or a moving image. May be.

[Modification of Second Embodiment (Second Modification)]
In the second embodiment described above, whether or not to instruct capture of a still image is determined by comparing the direction information associated with the still image stored in the image storage unit 17 with the current shooting direction. However, for example, when a predetermined direction to be photographed is determined in a wide range such as front, side, and rear, the photographer may shoot still images from the same direction in duplicate. It is easy to prevent to some extent. Therefore, in such a case, the capture instruction unit 19 determines whether to instruct to capture a still image according to the shooting direction of the imaging unit 7 calculated by the direction calculation unit 18 without performing the comparison. You can also. For example, the capture instruction unit 19 holds a predetermined direction to be photographed in advance, and determines an instruction to capture a still image when the photographing direction calculated by the direction calculation unit 18 becomes the predetermined direction. In this case, the imaging device 10b may not include the image storage unit 17.

[Modification of each embodiment (third modification)]
FIG. 8 is a diagram conceptually illustrating a processing configuration example of the imaging device 10c in the third modification. The imaging device 10c in each of the above embodiments may further include a stop instruction unit 21 that instructs the imaging unit 7 to stop autofocus. FIG. 8 illustrates an example in which the imaging device 10c according to the first embodiment includes a stop instruction unit 21. The stop instruction unit 21 is realized in the same manner as other processing units.

  Most imaging units 7 have an autofocus function, and in most cases, the imaging unit 7 is set so that autofocusing operates. Therefore, the stop instructing unit 21 instructs the image capturing unit 7 to stop the autofocus, and the image capturing unit 7 uses a focus parameter separately set as described in each of the above-described embodiments, so that the image capturing unit 7 Image. Thereby, it is possible to prevent the set focus parameter from being changed by the autofocus function. In this case, the marker detection unit 12 may execute the processing after waiting for autofocus to stop. Similarly, the imaging control method in each of the above-described embodiments may be executed after autofocus is stopped.

(Example)
Examples will be given below to describe the above-described embodiment in more detail. There is a possibility that the marker is not properly focused on the real-time captured image acquired from the imaging unit 7 in the initial state where the focus parameter has never been set in the imaging unit 7 according to the above-described embodiments. However, the marker only needs to be shown in the real-time captured image to such an extent that a certain reference point and three directions orthogonal to each other from the reference point can be constantly obtained regardless of the reference direction. Therefore, the above-described embodiments can be appropriately realized even in the initial situation as described above. However, when the imaging unit 7 has an autofocus function, it is possible to prevent the autofocus from stopping until the marker is properly detected from the real-time captured image in the initial situation. In this case, the imaging apparatus 10 determines whether or not the marker is appropriately detected from the real-time captured image, and stops autofocus only when the marker is appropriately detected, and executes the above-described processing.

  On the other hand, if an appropriate focus parameter is set even once in the imaging unit 7, even if the shooting direction is changed in multi-directional shooting, the focus on the marker shown in the real-time shot image is hardly shifted. Therefore, even if the shooting direction is changed in multidirectional shooting, the marker is appropriately detected, and each of the above-described embodiments can operate appropriately. However, when the photographer moves so much that the marker cannot be detected properly, the autofocus can be restarted until the marker is properly detected.

  Therefore, in the following embodiment, a specific example of a method for calculating the actual distance between the imaging unit 7 and the object will be given. The following description will be focused on the content different from the above content, and the same content as above will be omitted as appropriate.

  FIG. 9 is a diagram conceptually illustrating a processing configuration example of the imaging apparatus 10d in the embodiment. As illustrated in FIG. 9, the imaging device 10 d in the embodiment further includes an information acquisition unit 31. The information acquisition unit 31 is realized in the same manner as other processing units. FIG. 9 shows an example in which this example is applied to the above-described first embodiment, but this example can be applied to any of the above-described embodiments and modifications.

  The information acquisition unit 31 acquires information indicating the actual positional relationship between the marker and the object. This information may be obtained by the user's input operation on the input screen displayed on the display unit 5 being detected by the input unit 6 or may be stored in advance in the memory 2 and acquired from the memory 2. However, it may be acquired from the portable recording medium, another computer, or the like via the communication unit 4.

  This real positional relationship means the positional relationship between the marker and the object in the real world. For example, the actual positional relationship is shown as two-dimensional information indicating the actual positional relationship between the marker and the object on the same plane where the marker and the object are placed. Further, this actual positional relationship can be further indicated as three-dimensional information that takes into account the actual height of the object.

FIG. 10 and FIG. 11 are diagrams showing examples of the actual position relationship between the AR marker 8 (hereinafter also referred to as a marker) and the object 9.
In the example shown in FIG. 10, from the center point of the upper side S _P output AR marker 8, a predetermined distance L _R extending in a direction perpendicular to the upper side S _P, the reference point P _B of the object 9 is located (deployed) so In addition, the object 9 is placed, and the information acquisition unit 31 acquires information indicating the actual position relationship between the marker 8 and the object 9 as actual position information.
FIG. 11 is a view of the object 9 as viewed from above. In the example of FIG. 11, the marker 8 is formed from six square partial marks M1 to M6, and the object 9 has a line L _R that connects the center point S _M1 of the mark _M1 and the center point S _{M2 of the} mark M2. the center point, the center S _C of the object 9 to be positioned, placed.

  The reference calculation unit 13 sets a three-dimensional virtual space based on real information and image information regarding the marker 8 detected by the marker detection unit 12, and the virtual positions of the marker 8 and the imaging unit 7 in the three-dimensional virtual space. And the actual distance between the marker 8 and the imaging unit 7 is calculated. The reference calculation unit 13 sets the three-dimensional virtual space at a position corresponding to the reference point recognized from the detected marker 8. Based on information on the shape and size of the real world regarding the marker 8 (actual information) and information on the shape and size of the marker 8 in the real-time captured image, the reference point and the reference point are orthogonal to each other. The three directions to be obtained can be obtained constantly. Based on the reference point thus obtained and the three directions, the three-dimensional virtual space is set, and the virtual position of the marker 8 in the three-dimensional virtual space can be determined. However, the setting position of the three-dimensional virtual space with respect to the reference point recognized from the marker 8 is not particularly limited. Furthermore, the virtual position of the imaging unit 7 in the three-dimensional virtual space and the actual distance between the imaging unit 7 and the marker 8 can be calculated from the information.

  The distance calculation unit 14 is based on the real position relationship between the marker 8 and the object 9 indicated by the information acquired by the information acquisition unit 31 and the virtual position of the marker 8 determined by the reference calculation unit 13. The virtual position of the object 9 in the original virtual space is calculated. When the information acquired by the information acquisition unit 31 is two-dimensional information as described above, the distance calculation unit 14 calculates the position of a certain point on the plane on which the object 9 is placed as the virtual position of the object 9 To do. When the information is three-dimensional information, the distance calculation unit 14 calculates a three-dimensional position taking into account the height direction of the object 9 as a virtual position of the object 9. For example, when the information acquisition unit 31 acquires information indicating the actual height of the object 9, the distance calculation unit 14 calculates a virtual position in the height direction using the actual height of the object 9, and as a result, The three-dimensional position of the object 9 can be determined. The highest point of the object 9 may be calculated as the virtual position of the object 9.

  The distance calculation unit 14 uses the actual distance between the marker 8 and the imaging unit 7 calculated by the reference calculation unit 13, the virtual position of the imaging unit 7, and the calculated virtual position of the object 9 to The actual distance from the object 9 is calculated.

The calculation of the actual distance between the imaging unit 7 and the object can be calculated by other methods. For example, the actual distance between the imaging unit 7 and the object may be calculated by further using the shooting direction of the imaging unit 7 with respect to the object calculated by the direction calculation unit 18. The distance calculation unit 14 can also recognize at least a part of the object from the real-time captured image and calculate the virtual position of the object in the three-dimensional virtual space based on the recognized location.
Recognition of at least a part of an object from a real-time captured image can be performed by a general image recognition technique using at least one of a predetermined color and a predetermined shape of the object. In this case, the imaging device 10 may not have the information acquisition unit 31, and information indicating the actual positional relationship between the marker and the object may not be used.

The imaging device 10 can also display the real-time captured image acquired from the imaging unit 7 on the display unit 5 as shown in FIG.
FIG. 12 is a diagram illustrating a display example of a real-time captured image. According to the example of FIG. 12, the display unit 5 displays a real-time captured image in which the object 9 and the AR marker 8 are captured, and an indicator I indicating a multi-directional shooting state is superimposed on a part of the captured image. . In this case, the imaging apparatus 10 determines the shooting direction DR that has been shot and the shooting direction DN that has not been shot from the direction information associated with the still image stored in the image storage unit 17, and is calculated by the direction calculation unit 18. The current shooting direction DW of the imaging unit 7 to be acquired is acquired. As illustrated in FIG. 12, the imaging apparatus 10 generates indicator screen elements that can distinguish these various shooting directions DR, DN, and DW. According to this indicator, the photographer can easily grasp the photographing situation of the multidirectional photographing, and this display enables the photographer to efficiently perform the multidirectional photographing.

In addition, in the some flowchart used by the above-mentioned description, although several process (process) is described in order, the execution order of the process performed by each embodiment is not restrict | limited to the described order. In each embodiment, the order of the illustrated steps can be changed within a range that does not hinder the contents. Moreover, each above-mentioned embodiment and each modification can be combined in the range with which the content does not conflict.
Although the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  A part or all of each of the above embodiments and modifications may be specified as in the following supplementary notes. However, each embodiment and each modification are not limited to the following description.

(Appendix 1) Image acquisition means for sequentially acquiring real-time captured images from the imaging unit;
Marker detection means for detecting a marker from a real-time captured image acquired by the image acquisition means;
Reference calculation means for calculating a photographing positional relationship indicating a positional relationship between the marker and the imaging unit based on the actual information and the image information regarding the detected marker;
Distance calculating means for calculating an actual distance between the imaging unit and the object by using the calculated imaging position relationship and an object positional relationship indicating a positional relationship between the marker and the object captured in the real-time captured image; ,
Parameter determining means for determining a focus parameter for focusing the imaging unit on the object using the calculated actual distance;
Setting means for setting the focus parameter in the imaging unit;
An imaging control apparatus comprising:
(Additional remark 2) The direction calculation means which calculates the imaging | photography direction of the said imaging part with respect to the said object using the said imaging | photography positional relationship and the said object positional relationship;
Whether to instruct to capture a still image is determined according to the shooting direction of the imaging unit calculated by the direction calculation unit, and based on the determination result, the imaging unit is instructed to capture a still image. Capture instruction means;
The imaging control apparatus according to appendix 1, further comprising:
(Supplementary Note 3) The image acquisition unit stores the still image captured by the imaging unit in an image storage unit in association with direction information corresponding to the shooting direction of the imaging unit that was calculated at the time of capture,
The capture instruction unit instructs the capture from a relationship between a shooting direction of the imaging unit calculated by the direction calculation unit and the direction information associated with the still image stored in the image storage unit. Whether or not
The imaging control device according to attachment 2.
(Supplementary Note 4) Capture instruction means for instructing the imaging unit to capture a still image or a moving image using the focus parameter set by the setting means,
The imaging control apparatus according to any one of appendices 1 to 3, further comprising:
(Supplementary Note 5) Stop instruction means for instructing the imaging unit to stop autofocus,
The imaging control device according to any one of appendices 1 to 4, further comprising:
(Supplementary Note 6) Information acquisition means for acquiring information indicating the actual positional relationship between the marker and the object;
Further comprising
The reference calculating unit sets a three-dimensional virtual space based on the real information and image information related to the marker, the marker and the virtual position of the imaging unit in the three-dimensional virtual space, and the marker and the imaging unit And calculate the actual distance from
The distance calculating means calculates a virtual position of the object in the three-dimensional virtual space based on a real positional relationship between the marker and the object and a virtual position of the marker, and the marker, the imaging unit, Using the real distance, the virtual position of the imaging unit, and the virtual position of the object, the real distance between the imaging unit and the object is calculated.
The imaging control device according to any one of appendices 1 to 5.
(Supplementary note 7) Information acquisition means for acquiring the actual height information of the object,
Further comprising
The distance calculating means further uses the actual height of the object to calculate the actual distance between the imaging unit and the object.
The imaging control device according to any one of appendices 1 to 6.
(Supplementary Note 8) Real-time captured images are sequentially acquired from the imaging unit,
A marker is detected from the acquired real-time captured image,
Based on the actual information and image information regarding the detected marker, calculate a shooting positional relationship indicating the positional relationship between the marker and the imaging unit,
Using the calculated positional relationship of the image and the object positional relationship indicating the positional relationship between the marker and the object captured in the real-time captured image, the actual distance between the imaging unit and the object is calculated,
Using the calculated actual distance, determine a focus parameter for focusing the imaging unit on the object,
Setting the focus parameter in the imaging unit;
An imaging control method.
(Additional remark 9) Using the said imaging | photography positional relationship and the said object positional relationship, the imaging | photography direction of the said imaging part with respect to the said object is calculated,
According to the calculated shooting direction of the imaging unit, determine whether to instruct to capture a still image,
Instructing the imaging unit to capture a still image based on the determination result,
The imaging control method according to appendix 8, further including:
(Supplementary Note 10) The still image captured by the imaging unit is stored in the image storage unit in association with the direction information corresponding to the shooting direction of the imaging unit calculated at the time of capture.
Further including
The determination is based on a relationship between the calculated shooting direction of the imaging unit and the direction information associated with the still image stored in the image storage unit, and determines whether to instruct the capture.
The imaging control method according to appendix 9.
(Supplementary Note 11) Instructing the imaging unit to capture a still image or a moving image using the set focus parameter.
The imaging control method according to any one of appendices 8 to 10, further including:
(Supplementary Note 12) Instructing the imaging unit to stop autofocus,
The imaging control method according to any one of appendices 8 to 11, further including:
(Supplementary Note 13) Obtaining information indicating the actual positional relationship between the marker and the object,
Based on the real information and image information regarding the marker, a three-dimensional virtual space is set.
Further including
The calculation of the shooting position relationship is to calculate the virtual position of the marker and the imaging unit in the three-dimensional virtual space, and the actual distance between the marker and the imaging unit,
The calculation of the actual distance between the imaging unit and the object is as follows:
Calculating a virtual position of the object in the three-dimensional virtual space based on a real position relationship between the marker and the object, and a virtual position of the marker;
Using the actual distance between the marker and the imaging unit, the virtual position of the imaging unit, and the virtual position of the object, the actual distance between the imaging unit and the object is calculated.
The imaging control method according to any one of appendices 8 to 12.
(Supplementary Note 14) Obtaining the actual height information of the object,
Further including
The calculation of the actual distance between the imaging unit and the object further uses the actual height of the object to calculate the actual distance between the imaging unit and the object.
The imaging control method according to any one of appendices 8 to 13.
(Supplementary note 15) A recording medium for storing a program that causes a computer having an imaging unit to execute the imaging control method according to any one of Supplementary notes 8 to 14.
The present invention has been described above using the above-described embodiment as an exemplary example. However, the present invention is not limited to the above-described embodiment. That is, the present invention can apply various modes that can be understood by those skilled in the art within the scope of the present invention.
This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2014-057774 for which it applied on March 20, 2014, and takes in those the indications of all here.

1 CPU
2 Memory 4 Communication unit 5 Display unit 6 Input unit 7 Imaging unit 10 Imaging device 11 Image acquisition unit 12 Marker detection unit 13 Reference calculation unit 14 Distance calculation unit 15 Parameter determination unit 16 Setting unit 17 Image storage unit 18 Direction calculation unit 19 Capture Instruction unit 21 Stop instruction unit 31 Information acquisition unit

Claims (9)

An image obtaining unit for obtaining the shadow image shooting from the imaging unit,
A marker detection means for detecting a marker from the shooting image acquired by the image acquisition unit,
Reference calculation means for calculating a photographing positional relationship indicating a positional relationship between the marker and the imaging unit based on the information on the detected marker;
Distance calculation means for calculating the actual distance between the imaging unit and the object using the calculated shooting position relationship and the object position relationship indicating the positional relationship between the marker and the object captured in the captured image;
Parameter determining means for determining a focus parameter for focusing the imaging unit on the object using the calculated actual distance;
Setting means for setting the determined focus parameter in the imaging unit and adjusting a focus;
An imaging control apparatus comprising:

Direction calculating means for calculating a shooting direction of the imaging unit with respect to the object using the shooting position relationship and the object position relationship;
Whether to instruct to capture a still image is determined according to the shooting direction of the imaging unit calculated by the direction calculation unit, and based on the determination result, the imaging unit is instructed to capture a still image. Capture instruction means;
The imaging control apparatus according to claim 1, further comprising: The image acquisition means stores the still image captured by the imaging unit in an image storage unit in association with direction information corresponding to the shooting direction of the imaging unit that was calculated at the time of capture,
The capture instruction unit instructs the capture from a relationship between a shooting direction of the imaging unit calculated by the direction calculation unit and the direction information associated with the still image stored in the image storage unit. Whether or not
The imaging control apparatus according to claim 2. Capture instruction means for instructing the imaging unit to capture a still image or a moving image using the focus parameter set by the setting means;
The imaging control apparatus according to any one of claims 1 to 3, further comprising: Stop instruction means for instructing the imaging unit to stop autofocus;
The imaging control apparatus according to any one of claims 1 to 4, further comprising: Information acquisition means for acquiring information indicating a real positional relationship between the marker and the object;
The reference calculation means sets a three-dimensional virtual space based on the information about the marker, the virtual position of the marker and the imaging unit in the three-dimensional virtual space, and the actual distance between the marker and the imaging unit To calculate
The distance calculating means calculates a virtual position of the object in the three-dimensional virtual space based on a real positional relationship between the marker and the object and a virtual position of the marker, and the marker, the imaging unit, Using the real distance, the virtual position of the imaging unit, and the virtual position of the object, the real distance between the imaging unit and the object is calculated.
The imaging control apparatus according to any one of claims 1 to 5. Information acquisition means for acquiring real height information of the object;
Further comprising
The distance calculating means further uses the actual height of the object to calculate the actual distance between the imaging unit and the object.
The imaging control apparatus according to any one of claims 1 to 6. Acquire a captured image from the imaging unit,
A marker is detected from the acquired captured image,
Based on the information related to the detected marker, a shooting positional relationship indicating a positional relationship between the marker and the imaging unit is calculated,
The calculated photographing position relationship, and, by using the positional relationship object positional relationship shown of an object caught on the shooting image and the marker, and calculate the actual distance between the imaging unit and the object,
Using the calculated actual distance, determine a focus parameter for focusing the imaging unit on the object,
Set the determined focus parameter in the imaging unit and adjust the focus ;
An imaging control method.
The program which makes a computer perform the imaging control method of Claim 8 .

Images