JP6841977B2 - Shooting support device, shooting support method, and shooting support program - Google Patents

Description

The technology of the present disclosure relates to a shooting support device, a shooting support method, and a shooting support program.

Conventionally, when a user shoots a shooting target using an imaging device, a technique for presenting information for supporting the shooting to the user is known. For example, Patent Document 1 describes a technique of presenting an example stored in a database and performing imaging using setting information indicating setting conditions for imaging as in the example.

Further, Patent Document 2 describes information indicating the size, position, and moving speed of an imaging target, and a zoom magnification obtained by another imaging apparatus when an imaging target is photographed using a plurality of imaging devices. A technique for an image pickup apparatus to take an image based on a composition determined based on metadata such as the information shown is described.

Republished Patent No. 2013/118535 Japanese Unexamined Patent Publication No. 2011-21598

By the way, when a user shoots a shooting target, a plurality of imaging devices are used to shoot a specific shooting target from different directions or different positions, and the imaging is performed by each of the plurality of imaging devices. Images are combined to generate one captured image.

In this case, in order to obtain a user's desired image for one captured image obtained by combining the captured images, the relationship between the captured images captured by each of the plurality of imaging devices, for example, is included in each captured image. It is desired that the relationship such as the orientation of the shooting target is set to the user's desired state. Therefore, the relative positional relationship between each of the plurality of imaging devices and the imaging target is important.

However, since shooting is performed using a plurality of imaging devices, it may be difficult for the user to determine, for example, an appropriate shooting direction or shooting position, and as a result, the captured images obtained by each imaging device are combined. In that case, the captured image desired by the user may not be obtained.

Although the techniques described in Patent Document 1 and Patent Document 2 can grasp an example or composition for photographing an imaged object, they cannot grasp each of a plurality of imaging devices and the relative positional relationship of the imaged object.

The present disclosure has been made in consideration of the above circumstances, and is a photographing support device capable of obtaining a desired captured image when a plurality of captured images are combined to obtain one captured image. , Shooting support methods, and shooting support programs.

In order to achieve the above object, the imaging support device of the first aspect of the present disclosure acquires an image captured by a predetermined imaging device among a plurality of imaging devices that image a specific imaging target. Based on the acquisition unit and the image of the image to be photographed included in the image captured by the acquisition unit, the relative positional relationship between each of the plurality of imaging devices and the object to be photographed is recommended for photographing the object to be photographed. It includes a generation unit that generates a recommended bird's-eye view, and a display control unit that controls the display to display the recommended bird's-eye view generated by the generation unit.

The imaging support device of the second aspect is the imaging support device of the first aspect, in which the generation unit is an imaging device different from the predetermined imaging device, which is recommended when the predetermined imaging device is used as a reference. Generate a recommended bird's-eye view showing the relative positional relationship of.

In the shooting support device of the third aspect, in the shooting support device of the first aspect or the second aspect, the acquisition unit acquires the captured image captured by each of the plurality of imaging devices, and the generation unit acquires the captured image. A bird's-eye view of the actual positions of each of the plurality of imaging devices and the shooting target, which represent the relative positional relationship in which the shooting target was shot, based on the images of the shooting target included in each of the captured images acquired by the unit. Generate more diagrams.

The shooting support device of the fourth aspect is the shooting support device of the third aspect, and the display control unit displays the recommended bird's-eye view and the bird's-eye view of the actual position on the display unit in a superposed state. Control to make it.

The shooting support device of the fifth aspect is the shooting support device of any one of the first to fourth aspects, and the generation unit analyzes the captured image to obtain information on the size of the shooting target. , The bird's-eye view is generated based on at least one of the information regarding the position of the shooting target and the information regarding the orientation of the shooting target.

The shooting support device of the sixth aspect is the shooting support device of any one of the first to fifth aspects, and the acquisition unit acquires the captured image captured by each of the plurality of imaging devices. The display control unit controls the display unit to display a bird's-eye view generated by the generation unit and at least one image captured by each of the plurality of image pickup devices acquired by the acquisition unit.

In the shooting support device of the seventh aspect, in the shooting support device of the sixth aspect, the captured image to be displayed on the display unit is combined with any of the captured images captured by each of the plurality of imaging devices. The display control unit further includes a reception unit that accepts the designation of whether or not to do so, and the display control unit controls the display unit to display the captured image for which the reception unit has received the designation.

The shooting support device of the eighth aspect is shown in the bird's-eye view with respect to the bird's-eye view displayed on the display unit by the display control unit in the shooting support device of any one aspect from the first aspect to the seventh aspect. For at least one of the positions and orientations of each of the plurality of imaging devices, a designated unit for designating the changed state is further provided, and the generating unit is imaged by each of the plurality of imaging devices in the changed state. The estimated estimated captured image is further generated, and the display control unit controls the display unit to display the estimated captured image generated by the generation unit.

The imaging support device of the ninth aspect is a storage that stores information representing the relative positional relationship in which the imaging target is photographed in the imaging support device of any one aspect from the first aspect to the eighth aspect. It also has a part.

The photographing support device of the tenth aspect is the photographing support device of any one aspect from the first aspect to the ninth aspect, and the predetermined imaging device is a main camera mainly used for photographing the imaged object. ..

The imaging support device of the eleventh aspect is the imaging support device of any one aspect from the first aspect to the tenth aspect, and the captured image is a moving image.

The shooting support method of the twelfth aspect acquires a captured image captured by a predetermined imaging device among a plurality of imaging devices that capture a specific imaging target, and captures the image to be included in the acquired image. Based on the image, a recommended bird's-eye view showing the relative positional relationship recommended for shooting the shooting target of each of the plurality of imaging devices and the shooting target is generated, and the generated bird's-eye view is displayed on the display unit. This is a shooting support method for a computer to execute a process that controls display.

The shooting support program of the thirteenth aspect acquires a captured image captured by a predetermined imaging device among a plurality of imaging devices that capture a specific shooting target, and the shooting target included in the acquired captured image. Based on the image, a recommended bird's-eye view showing the relative positional relationship recommended for shooting the shooting target of each of the plurality of imaging devices and the shooting target is generated, and the generated bird's-eye view is displayed on the display unit. Let the computer execute the process that controls the display.

Further, in order to achieve the above object, the shooting support device of the present disclosure is a shooting support device having a processor, and the processor performs a predetermined imaging among a plurality of imaging devices for shooting a specific shooting target. The image captured by the device is acquired, and based on the image of the image to be captured included in the acquired image, the relative image of each of the plurality of image devices and the object to be photographed is recommended for photographing the object to be photographed. It is a shooting support device that generates a recommended bird's-eye view showing the positional relationship and controls the displayed bird's-eye view to be displayed on the display unit.

According to the present disclosure, when a plurality of captured images are combined to obtain one captured image, a desired captured image can be obtained.

It is a top view which shows an example of the structure of the imaging system of embodiment. It is a block diagram which shows an example of the hardware composition of the photographing support apparatus of embodiment. It is a block diagram which shows an example of the functional structure of the photographing support apparatus of 1st and 2nd Embodiment. It is a flowchart which shows an example of the flow of the shooting support processing of 1st Embodiment. It is explanatory drawing for demonstrating an example of the recommended bird's-eye view generated by the photographing support apparatus of embodiment. It is explanatory drawing for demonstrating an example of the actual position bird's-eye view generated by the photographing support apparatus of embodiment. It is explanatory drawing for demonstrating an example of the display of the recommended bird's-eye view, the actual position bird's-eye view, the main camera image, and the sub-camera image displayed on the display part of the photographing support apparatus of embodiment. It is explanatory drawing for demonstrating another example of the recommended bird's-eye view generated by the photographing support apparatus of embodiment. It is explanatory drawing for demonstrating another example of the recommended bird's-eye view generated by the photographing support apparatus of embodiment. It is a flowchart which shows another example of the flow of the shooting support processing of 1st Embodiment. It is explanatory drawing for demonstrating another example of the display of the recommended bird's-eye view, the actual position bird's-eye view, the main camera image, and the sub-camera image displayed on the display part of the photographing support apparatus of embodiment. It is a flowchart which shows an example of the flow of the shooting support processing of 2nd Embodiment. It is a block diagram which shows an example of the functional structure of the photographing support device of 3rd Embodiment. It is a flowchart which shows an example of the flow of the shooting support processing of 3rd Embodiment. It is explanatory drawing for demonstrating an example of the display of the recommended bird's-eye view, the actual position bird's-eye view, the main camera image, and the estimated image taken image displayed on the display part of the photographing support apparatus of 3rd Embodiment.

Hereinafter, examples of embodiments for carrying out the technique of the present disclosure will be described in detail with reference to the drawings. In the following, it is referred to as shooting that the user captures an image to be photographed by an imaging device and obtains an captured image such as a moving image. Therefore, "shooting" when the user takes a picture using the image pickup device and "imaging" by the image pickup device may be interchangeable.

[First Embodiment]

First, the configuration of the imaging system 10 of the present embodiment will be described with reference to FIG. As shown in FIG. 1, the image pickup system 10 of the present embodiment includes two image pickup devices, a main camera 12 and a sub camera 14, and a shooting support device 16. The main camera 12 and the sub camera 14 of the present embodiment are examples of the plurality of imaging devices of the present disclosure.

In the present embodiment, as an example, the user takes a picture of a person S to be photographed by the main camera 12 mainly used and the sub camera 14 used as an auxiliary, and each of the main camera 12 and the sub camera 14 takes a picture. A form of combining the obtained moving images to generate one moving image, so-called video content, will be described. In the shooting of the shooting target S, the person who actually operates each of the main camera 12 and the sub camera 14 may be one person or a plurality of people, but in the present embodiment, they are collectively referred to. Simply called "user". Further, in the following, the moving image captured by the main camera 12 is referred to as a "main camera image", and the moving image captured by the sub camera 14 is referred to as a "sub camera image".

In this embodiment, as an example, a mode in which a user connects a main camera image and a sub camera image in chronological order to generate one moving image by combining the main camera image and the sub camera image will be described. To do. Specifically, for example, the main camera image from the start of shooting of the shooting target S to 3 minutes later, the sub camera image from 3 minutes to 4 minutes later, and the main camera image from 4 minutes to 7 minutes later. Etc., and a case where the main camera image and the sub camera image are combined in chronological order will be described. The method of combining the main camera image and the sub camera image is not limited to this embodiment. For example, one screen may be divided into two areas, and the main camera image and the sub camera image may be combined by arranging the main camera image in one of the divided areas and the sub camera image in the other area. Further, one of the main camera image and the sub camera image may be combined with the other image, and both images may be displayed at the same time.

Each of the main camera 12 and the sub camera 14 is connected to the photographing support device 16 by an HDMI (High-Definition Multimedia Interface) (registered trademark) cable. The connection method between the main camera 12 and the sub camera 14 and the shooting support device 16 is not limited to this embodiment, and other forms of wired connection or wireless connection may be used. May be good. Further, the main camera 12 and the sub camera 14 may be connected to the photographing support device 16 by different methods. The moving image data representing the moving images captured by each of the main camera 12 and the sub camera 14 is output to the photographing support device 16.

The shooting support device 16 has a function of assisting the user in shooting the shooting target S by presenting the imaging positions of the main camera 12 and the sub camera 14 that are appropriate for obtaining the video content desired by the user. Have.

The hardware configuration of the photographing support device 16 of the present embodiment will be described with reference to FIG. As shown in FIG. 2, the photographing support device 16 of the present embodiment includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. Further, the photographing support device 16 includes a display unit 23 such as a liquid crystal display and an operation unit 24 including various buttons and the like. The display unit 23 and a part of the operation unit 24 are integrated as a touch panel display. Further, the photographing support device 16 includes an external I / F (Interface) 26 to which the main camera 12 and the sub camera 14 are connected. The CPU 20, the memory 21, the storage unit 22, the display unit 23, the operation unit 24, and the external I / F 26 are connected to the bus 29. Examples of the photographing support device 16 include a personal computer, a server computer, and the like.

The storage unit 22 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. The photographing support program 30 is stored in the storage unit 22 as a storage medium. The CPU 20 reads the shooting support program 30 from the storage unit 22, expands it into the memory 21, and executes the expanded shooting support program 30.

Next, with reference to FIG. 3, the functional configuration of the photographing support device 16 of the present embodiment will be described. As shown in FIG. 3, the photographing support device 16 of the present embodiment includes an acquisition unit 40, a generation unit 42, a display control unit 44, and a reception unit 46. When the CPU 20 executes the shooting support program 30, it functions as an acquisition unit 40, a generation unit 42, a display control unit 44, and a reception unit 46.

The acquisition unit 40 acquires the main camera image captured by the main camera 12, and more specifically, the moving image data representing the main camera image from the main camera 12. Further, the acquisition unit 40 acquires the sub-camera image captured by the sub-camera 14, and more specifically, the moving image data representing the sub-camera image from the sub-camera 14.

The generation unit 42 is at least one of information on the size of the shooting target S, information on the position of the shooting target S, and information on the orientation of the shooting target S, which is obtained by analyzing the main camera image and the sub camera image. It has a function to generate a bird's-eye view based on the information. Specifically, the generation unit 42 of the present embodiment of the main camera 12, the sub camera 14, and the shooting target S, based on the image of the shooting target S included in the main camera image acquired by the acquisition unit 40. A recommended bird's-eye view (hereinafter referred to as "recommended bird's-eye view") showing a relative positional relationship recommended for the user to shoot the shooting target S is generated. Further, the generation unit 42 of the present embodiment has the main camera 12, the sub camera 14, and the shooting target S based on the images of the shooting target S included in each of the main camera image and the sub camera 14 acquired by the acquisition unit 40. Generates a bird's-eye view of the actual position (hereinafter, referred to as "actual position bird's-eye view") showing the relative positional relationship in which the image capture target S is actually photographed.

The display control unit 44 controls the display unit 23 to display the recommended bird's-eye view generated by the generation unit 42. Further, the display control unit 44 of the present embodiment controls the display unit 23 to display the actual position bird's-eye view generated by the generation unit 42. Further, the display control unit 44 of the present embodiment controls the display unit 23 to display at least one moving image of the main camera image and the sub camera image acquired by the acquisition unit 40.

The reception unit 46 receives from the user via the operation unit 24 the designation of which of the main camera image and the sub camera image is the moving image to be displayed on the display unit 23.

Next, the operation of the photographing support device 16 of the present embodiment will be described with reference to FIG. When the CPU 20 executes the shooting support program 30, the shooting support process shown in FIG. 4 is executed. The shooting support process shown in FIG. 4 is executed when, for example, the operation unit 24 instructs the user to execute the shooting support process.

In step S100 of FIG. 4, the display control unit 44 causes the display unit 23 to display guidance for urging the user to connect the main camera 12 to the shooting support device 16. The user connects the main camera 12 to the HDMI terminal (not shown) of the shooting support device 16 according to the guidance displayed on the display unit 23. The main camera 12 may be connected to the shooting support device 16 before the guidance is displayed by this step.

In the next step S102, the acquisition unit 40 determines whether or not the shooting support process has been cancelled. In the shooting support device 16 of the present embodiment, for example, when the user has instructed to execute the shooting support processing but determines that the shooting support does not need to be performed, the cancel button (not shown) included in the operation unit 24 is included. By operating, it is possible to instruct to cancel the shooting support process. Therefore, in this step S102, it is determined whether or not the user has given the cancellation instruction by operating the cancel button. When the cancellation instruction is given, the determination in step S102 becomes an affirmative determination, and the process proceeds to step S126. In step S126, the display control unit 44 ends the main shooting support process after displaying the information to the effect that the shooting support process is canceled on the display unit 23.

On the other hand, if the cancellation instruction is not given, the determination in step S102 becomes a negative determination, and the process proceeds to step S104. In step S104, the acquisition unit 40 determines whether or not the main camera image from the main camera 12 is input to the shooting support device 16 via the external I / F 26. Until the main camera image is input, the determination in step S104 becomes a negative determination, and the process returns to step S102. On the other hand, when the main camera image is input and acquired by the acquisition unit 40, the determination in step S104 becomes an affirmative determination, and the process proceeds to step S106.

In step S106, the generation unit 42 starts the analysis of the main camera image acquired by the acquisition unit 40. The generation unit 42 of the present embodiment estimates the orientation (face orientation) of the shooting target S with respect to the main camera 12 and the distance between the shooting target S and the main camera 12 by performing image analysis on the main camera image. These estimation methods and image analysis methods are not particularly limited. For example, the generation unit 42 may detect the face of the shooting target S included in the main camera image and estimate the distance between the shooting target S and the main camera 12 from the size of the face occupying the angle of view. .. Further, for example, the face, eyes, nose, mouth, etc. of the photographing target S included in the main camera image may be detected, and the orientation of the face with respect to the main camera 12 may be estimated from these positions, sizes, and the like. The focal length of the main camera 12 is used to estimate the distance between the shooting target S and the main camera 12, but in order to perform more accurate position analysis, the focal length of the lens actually used for imaging is used. It is preferable to obtain the information of. For example, communication may be performed with the main camera 12 via an external I / F 26, and the generation unit 42 may acquire information representing the focal length of the lens used for imaging from the main camera 12, or the operation unit 24 may be used by the user. May enter information representing the focal length. If information indicating the focal length of the lens actually used for imaging cannot be obtained, it may be used assuming a so-called 35 mm equivalent focal length of 50 mm, which is widely used.

In the next step S108, the display control unit 44 causes the display unit 23 to display guidance for urging the user to connect the sub camera 14 to the shooting support device 16. The user connects the sub camera 14 to the HDMI terminal (not shown) of the shooting support device 16 according to the guidance displayed on the display unit 23. The sub camera 14 may be connected to the photographing support device 16 before the guidance is displayed by this step.

In the next step S110, the acquisition unit 40 determines whether or not the shooting support process has been canceled, as in the step S102. When the cancellation instruction is given, the determination in step S110 becomes an affirmative determination, the process proceeds to step S126, and the display control unit 44 displays information on the display unit 23 to the effect that the shooting support process is cancelled. After that, the main shooting support process is completed.

On the other hand, if the cancellation instruction is not given, the determination in step S110 becomes a negative determination, and the process proceeds to step S112. In step S112, the acquisition unit 40 determines whether or not the sub camera image from the sub camera 14 is input to the shooting support device 16 via the external I / F 26. Until the sub camera image is input, the determination in step S112 becomes a negative determination, and the process returns to step S110. On the other hand, when the sub camera image is input and acquired by the acquisition unit 40, the determination in step S112 becomes an affirmative determination, and the process proceeds to step S114.

In step S114, the generation unit 42 starts the analysis of the sub camera image acquired by the acquisition unit 40. The generation unit 42 of the present embodiment estimates the orientation (face orientation) of the shooting target S with respect to the sub camera 14 and the distance between the shooting target S and the sub camera 14 by performing image analysis on the sub camera image. These estimation methods and image analysis methods are not particularly limited, and can be the same as in step S106.

In the next step S116, the generation unit 42 generates a recommended bird's-eye view based on the analysis result of the main camera image. As described above, by analyzing the main camera image, the orientation of the face of the shooting target S with respect to the main camera 12 and the distance between the shooting target S and the main camera 12 are estimated. As an example, in the present embodiment, the shooting position and shooting direction of the sub camera 14 recommended based on the shooting position and orientation (shooting direction) of the main camera 12 based on the orientation of the face and the distance to the shooting target S. Derived. The recommended method for deriving the shooting position and shooting direction of the sub camera 14 is not particularly limited. For example, the correspondence between at least one of the face orientation of the shooting target S, the position of the main camera 12, and the shooting direction of the main camera 12 and the position and shooting direction of the sub camera 14 (hereinafter referred to as "recommended positional relationship"). May be stored in the storage unit 22 in advance, and the generation unit 42 may derive the recommended shooting position and shooting direction of the sub camera 14 based on the recommended positional relationship stored in the storage unit 22.

Further, the method of generating the recommended bird's-eye view in the generation unit 42 is not particularly limited. As an example, in the present embodiment, the icons corresponding to the main camera 12, the sub camera 14, and the shooting target S are arranged in a bird's-eye view from the head side (upper side) of the shooting target S according to their respective directions and positions. .. FIG. 5A shows an example of the recommended bird's-eye view 50 generated by the generation unit 42 of the present embodiment. In the recommended bird's-eye view 50 shown in FIG. 5A, an icon 60 indicating the shooting target S is arranged together with information indicating the orientation of the face. Further, in the recommended bird's-eye view 50, an icon 62 indicating the main camera 12 according to the shooting position and shooting direction of the main camera 12 and a sub camera 14 according to the shooting position and shooting direction of the recommended sub camera 14 are provided. The icon 65 is arranged. As an example, in the present embodiment, since the shooting position of the main camera 12 is used as a reference, the icon 62 in the recommended bird's-eye view 50 is based on the shooting position and shooting direction of the main camera 12 in which shooting was actually performed. Have been placed.

In the next step S118, the generation unit 42 generates an actual position bird's-eye view based on the analysis result of the main camera image and the analysis result of the sub camera image. The method of generating the actual position bird's-eye view in the generation unit 42 is not particularly limited, and for example, the same method as the recommended bird's-eye view generation method can be used. FIG. 5B shows an example of an actual position bird's-eye view 52 generated by the generation unit 42 of the present embodiment. In the actual position bird's-eye view 52 shown in FIG. 5B, an icon 60 indicating the shooting target S is arranged together with information indicating the orientation of the face. Further, in the actual position bird's-eye view 52, an icon 62 indicating the main camera 12 according to the shooting position and shooting direction of the main camera 12 and an icon 64 indicating the sub camera 14 according to the shooting position and shooting direction of the sub camera 14 And are arranged. In the actual position bird's-eye view 52, the icon 64 indicating the sub camera 14 is arranged based on the shooting position and the shooting direction of the sub camera 14 in which the shooting was actually performed.

In the next step S120, the display control unit 44 causes the display unit 23 to display the recommended bird's-eye view 50, the actual position bird's-eye view 52, the main camera image 70, and the sub camera image 72, as in the example shown in FIG. Take control. In the example shown in FIG. 6, the recommended bird's-eye view 50 and the actual position bird's-eye view 52 are superimposed and displayed on the display unit 23. Further, in the example shown in FIG. 6, a state in which the real-time main camera image 70 of the main camera 12 and the real-time sub camera image 72 of the sub camera 14 are displayed is shown.

By displaying the main camera image 70 and the sub camera image 72 on the display unit 23 in this way, the user can use the relationship between the shooting positions and shooting directions of the main camera 12 and the sub camera 14 and the moving image actually captured. It becomes easier to understand the relationship with the image.

Not limited to this embodiment, the display control unit 44 may not display the main camera image 70 and the sub camera image 72 on the display unit 23, or may display only one of them. is there. In the present embodiment, when the reception unit 46 receives the designation of the moving image to be displayed from the user via the operation unit 24, the display control unit 44 causes the display unit 23 to display the designated moving image, and the reception unit When the 46 does not accept the designation, both the main camera image 70 and the sub camera image 72 are displayed on the display unit 23.

In the next step S122, the generation unit 42 determines whether or not at least one of the main camera 12 and the sub camera 14 has moved, more specifically, the shooting position of at least one of the main camera 12 and the sub camera 14. And it is determined whether or not the shooting direction has changed. The method by which the generation unit 42 determines whether or not at least one camera has moved is not particularly limited. For example, the generation unit 42 may perform image analysis of each of the main camera image 70 and the sub camera image 72, and determine whether or not the position or orientation of the shooting target S or another subject has changed. Further, for example, when the main camera image 70 and the sub camera image 72 are provided with position detection sensors, the generation unit 42 may acquire the detection results of these sensors and make a determination based on the acquired detection results. ..

When at least one of the main camera 12 and the sub camera 14 moves, the determination in step S122 becomes an affirmative determination, the process returns to step S116, and the processes of steps S118 and S120 are repeated. In this case, the recommended bird's-eye view 50 and the actual position bird's-eye view 52 displayed on the display unit 23 are updated according to the movement of at least one of the cameras, and each becomes a bird's-eye view according to the real-time state.

On the other hand, when both the cameras of the main camera 12 and the sub camera 14 are not moving, the determination in step S122 becomes a negative determination, and the process proceeds to step S124. In step S124, the generation unit 42 determines whether or not to end the main shooting support process. In the present embodiment, as an example, predetermined termination conditions such as when the cancel button is operated by the user or when at least one of the main camera 12 and the sub camera 14 is disconnected from the HDMI terminal. When the condition is satisfied, the main shooting support process is terminated. If the end condition is not satisfied, the determination in step S124 becomes a negative determination, the process returns to step S122, and the above process is repeated. On the other hand, when the end condition is satisfied, the determination in step S124 becomes an affirmative determination, and the main shooting support process ends. When the main shooting support process is terminated, the display control unit 44 ends the display of the recommended bird's-eye view 50, the actual position bird's-eye view 52, the main camera image 70, and the sub camera image 72 displayed on the display unit 23.

In the shooting support device 16 of the present embodiment, the shooting position and shooting of the sub camera 14 are recommended for at least one of the face orientation of the shooting target S, the position of the main camera 12, and the shooting direction of the main camera 12. The case where the direction is uniquely determined, so to speak, the case where the recommended bird's-eye view 50 is one type has been described, but the present invention is not limited to this embodiment, and there may be a plurality of recommended bird's-eye views for recommendation. For example, there may be a plurality of recommended shooting positions and shooting directions of the sub camera 14 associated with the shooting position and shooting direction of the main camera 12. In other words, it may have a plurality of the above recommended positional relationships. As a specific example, as shown in FIG. 7A, a recommended bird's-eye view 50A in which the orientation of the sub-camera 14 with respect to the photographing target S is different from the orientation of the sub-camera 14 recommended by the recommended bird's-eye view 50 shown in FIG. As shown in FIG. 7B, the distance between the photographing target S and the sub camera 14 has a recommended bird's-eye view 50B or the like different from the distance of the sub camera 14 recommended by the recommended bird's-eye view 50 shown in FIG. 5A. You may.

An example of a mode in which the user can instruct to change the recommended bird's-eye view by operating the operation unit 24 will be described with reference to FIG. FIG. 8 shows a flowchart showing an example of the flow of shooting support processing when a plurality of types of recommended bird's-eye views are provided.

The shooting support process shown in FIG. 8 is different from the above-mentioned shooting support process (see FIG. 4) in that the processing of steps S121A to S121C is included between steps S120 and S122. In the shooting support process shown in FIG. 8, it is determined whether or not the reception unit 46 has received the instruction to change the recommended bird's-eye view (50, 50A, and 50B). If the reception unit 46 has not received the change instruction, the determination in step S121A is a negative determination, and the process proceeds to step S122. On the other hand, when the reception unit 46 receives the change instruction, the determination in step S121A becomes an affirmative determination, and the process proceeds to step S121B. In step S121B, the display control unit 44 updates the recommended bird's-eye view (either 50, 50A, or 50B) displayed on the display unit 23. Specifically, the generation unit 42 acquires one recommended positional relationship different from the recommended bird's-eye view currently displayed on the display unit 23 from the plurality of recommended positional relationships stored in the storage unit 22. A recommended bird's-eye view is generated based on the acquired recommended positional relationship, and the display control unit 44 displays the generated recommended bird's-eye view on the display unit 23. When the user decides to use the recommended bird's-eye view displayed on the display unit 23 for shooting, the user instructs the determination of the recommended bird's-eye view by operating the decision button (not shown) included in the operation unit 24. To do.

In the next step S121C, the reception unit 46 determines whether or not the instruction for the above decision has been accepted. If the reception unit 46 of the present embodiment does not accept the instruction for the above determination even after the lapse of a predetermined time, the determination in step S121C becomes a negative determination, the process returns to step S121A, and the processing in step S121B is performed. repeat. On the other hand, when the reception unit 46 receives the instruction for the above determination within a predetermined time, the determination in step S121C becomes an affirmative determination, and the process proceeds to step S122. After this processing, when a positive judgment is made in step S122 and the process returns to step S116 to generate the recommended bird's-eye view again, the recommended bird's-eye view is based on the recommended positional relationship determined in step S121C. Generate a bird's-eye view.

Further, in the present embodiment, as an example, a mode for deriving the shooting position and shooting direction of the sub camera 14 recommended based on the shooting position and shooting direction of the main camera 12 has been described, but the present embodiment is not limited to this embodiment. .. For example, the shooting position and shooting direction of the main camera 12 and the shooting position and shooting direction of the sub camera 14, which are recommended based on the orientation or position of the shooting target S, may be derived. In this case, the recommended bird's-eye view 50C generated by the generation unit 42 and displayed on the display unit 23 is the main according to the recommended shooting position and shooting direction of the main camera 12, as in the example shown in FIG. An icon 63 indicating the camera 12 and an icon 65 indicating the sub camera 14 according to the recommended shooting position and shooting direction of the sub camera 14 are arranged.

As described above, in the photographing support device 16 of the present embodiment, the acquisition unit 40 acquires the main camera image in which the image target S is photographed by the user from the main camera 12, and the generation unit 42 analyzes the acquired main camera image. A recommended bird's-eye view is generated based on the analysis result obtained from the image of the shooting target S included in the main camera image, and the display control unit 44 causes the display unit 23 to display the recommended bird's-eye view.

As described above, according to the shooting support device 16 of the present embodiment, the user determines the relative positional relationship between each camera and the shooting target S recommended for shooting the shooting target S by the main camera 12 and the sub camera 14. You will be able to grasp. Therefore, according to the photographing support device 16 of the present embodiment, the user can obtain a desired moving image when the main camera image and the sub camera image are combined to obtain one moving image.

[Second Embodiment]

Hereinafter, the second embodiment will be described in detail. In this embodiment, the same configurations and actions as those described in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

Since the configurations of the imaging system 10 and the photographing support device 16 of the present embodiment are the same as the configurations of the imaging system 10 and the photographing support device 16 of the first embodiment (see FIGS. 1 to 3), the description thereof will be omitted.

On the other hand, the operation of the photographing support device 16 of the present embodiment is partially different from the photographing support process. FIG. 10 shows a flowchart showing an example of the flow of the photographing support process of the present embodiment. As shown in FIG. 10, the shooting support processing of the present embodiment is different in that the processing of step S125 is performed after step S124 of the shooting support processing (see FIG. 4) of the first embodiment.

In step S125, which is performed after the determination as to whether or not to end the main shooting support process is an affirmative determination, the generation unit 42 is displayed on the display unit 23 under the control of the display control unit 44. After the information representing the main camera 12 and the information representing the sub camera 14 are associated and stored in the storage unit 22, the main shooting support process is terminated. That is, the generation unit 42 stores the information representing the relative positional relationship in which the photographing target S is photographed in the storage unit 22. The information representing the main camera 12 and the information representing the sub camera 14 stored in the storage unit 22 by the generation unit 42 are not particularly limited, and for example, information such as an identifier that enables the camera used for photographing to be identified. It should be.

As described above, in the shooting support device 16 of the present embodiment, when the shooting support process is terminated, that is, when the user ends the shooting of the shooting target S by the main camera 12 and the sub camera 14, the main camera 12 and the shooting support device 16 at the end time are finished. The actual position bird's-eye view 52 showing the relative positional relationship between the arrangement of the sub-camera 14 and the orientation of the shooting target S is described in the storage unit 22 in association with the information representing the main camera 12 and the information representing the sub-camera 14. Let me.

In step S125, the generation unit 42 correlates with the information representing the main camera 12 and the information representing the sub camera 14, and replaces the actual position bird's-eye view 52 with the recommended bird's-eye view 50 together with the actual position bird's-eye view 52. It may be stored in the storage unit 22. Further, the user may be able to instruct via the operation unit 24 which of the recommended bird's-eye view 50 and the actual position bird's-eye view 52 is to be stored in the storage unit 22.

Further, as shown in FIG. 10, in the shooting support processing of the present embodiment, as compared with the shooting support processing of the first embodiment (see FIG. 4), steps S115A to S115D are performed between steps S114 and S116. The difference is that the processing is executed.

In step S115A, the generation unit 42 stores the camera information of both the information representing the main camera 12 and the information representing the sub camera 14 in the storage unit 22 for the main camera 12 and the sub camera 14 currently connected. Judge whether or not. When both camera information is stored in the storage unit 22, the determination in step S115A becomes an affirmative determination, and the process proceeds to step S115B. In step S115B, the generation unit 42 reads the actual position bird's-eye view 52 associated with the information representing the main camera 12 and the information representing the sub-camera 14 from the storage unit 22 and makes it a recommended bird's-eye view 50. The process proceeds to step S118.

On the other hand, when both camera information is not stored in the storage unit 22, the determination in step S115A becomes a negative determination, and the process proceeds to step S115C. In step S115C, the display control unit 44 indicates that the actual position bird's-eye view 52 corresponding to both camera information is not stored in the storage unit 22, that is, the main shooting support device 16 is connected to the display unit 23. Information indicating that the camera 12 and the sub camera 14 are a combination of the main camera 12 and the sub camera 14 that have not been supported for shooting until now is displayed on the display unit 23 as guidance. In the present embodiment, as an example, the user refers to the guidance displayed on the display unit 23, and if there is no change in the connected camera, the user operates the approval button (not shown) included in the operation unit 24. , Instruct to approve, or instruct to cancel the shooting support process by operating the cancel button (not shown) included in the operation unit 24.

Therefore, in the next step S115D, the generation unit 42 determines whether or not the approval button has been operated. When the cancel button is operated, the determination in step S115D becomes a negative determination, and the main shooting support process ends.

On the other hand, when the approval button is operated, the determination in step S115D becomes an affirmative determination, and the process proceeds to step S116. In this case, the generation unit 42 obtains the recommended bird's-eye view 50 and the actual position bird's-eye view 52 based on the main camera image and the sub camera image input by the combination of the newly connected main camera 12 and the sub camera 14. It will be generated.

In this embodiment, it is determined in step S115A whether or not both camera information is stored, but the present embodiment is not limited to this embodiment, and the camera information of one of the main camera 12 and the sub camera 14 is not limited to this embodiment. May be in the form of determining whether or not is stored. For example, when the shooting of the shooting target S is resumed after a break, the main camera 12 mainly used for shooting may use the same camera, but the sub camera 14 may use a different model of camera. is there. As a form corresponding to such a case, in step S115A, a form for determining whether or not the information representing the main camera 12 is stored in the storage unit 22 may be used.

As described above, in the shooting support processing performed by the shooting support device 16 of the present embodiment, after the shooting support processing is performed, the shooting support processing is performed again for the shooting using the same main camera 12 and sub camera 14. , The bird's-eye view displayed on the display unit 23 last time is adopted as the recommended bird's-eye view. As a result, according to the shooting support device 16 of the present embodiment, it becomes easy to resume shooting from the same state as the arrangement state of the main camera 12 and the sub camera 14 at the time when the shooting was completed last time.

[Third Embodiment]

Hereinafter, the third embodiment will be described in detail. In this embodiment, the same configurations and actions as those described in the first and second embodiments are designated by the same reference numerals, and detailed description thereof will be omitted.

Since the configuration of the imaging system 10 of the present embodiment is the same as the configuration of the imaging system 10 of the first embodiment (see FIG. 1), the description thereof will be omitted. On the other hand, the configuration of the shooting support device 16 is different from the configuration of the shooting support device 16 of the first embodiment (see FIGS. 2 and 3) in that the designation unit 47 is further provided as shown in FIG. There is.

In each of the above embodiments, a mode in which the display control unit 44 causes the display unit 23 to display the main camera image 70 and the sub camera image 72 has been described. On the other hand, in the present embodiment, when the positions and shooting directions of the main camera 12 and the sub camera 14 are specified with respect to the recommended bird's-eye view 50 or the actual position bird's-eye view 52, the main camera 12 and the sub camera 12 and the sub camera 12 and the sub camera 12 are designated in the designated position and shooting direction. A mode in which the estimated captured image presumed to have been captured by the camera 14 is displayed on the display unit 23 instead of the main camera image 70 or the sub camera image 72 will be described. Hereinafter, the operation of generating and displaying the estimated captured image presumed to have been captured by the main camera 12 or the sub camera 14 at the position and the shooting direction specified by the user will be referred to as a simulation.

The designation unit 47 has at least the position and shooting direction of the icons 62, 63, 64, and 65 shown in the bird's-eye view with respect to the recommended bird's-eye view 50 or the actual position bird's-eye view 52 displayed on the display unit 23. One has a function that the changed state is specified by the user.

The operation of performing the above simulation in the photographing support device 16 of the present embodiment will be described with reference to FIG. When the CPU 20 executes the shooting support program 30, the simulation process shown in FIG. 12 is executed. As an example, in the simulation process shown in FIG. 12, for example, when the user touches any of the icons 62, 63, 64, and 65 displayed on the display unit 23, or an instruction to perform the simulation is given to the operation unit 24. Executed when the user uses it.

In step S200 shown in FIG. 12, the designation unit 47 determines whether or not the user's operation on any of the icons 62, 63, 64, and 65 has been completed. In this embodiment, the position can be moved, rotated, enlarged, and reduced for each of the icons 62, 63, 64, and 65 by the operation of the user. The movement of the positions of the icons 62, 63, 64, and 65 corresponds to the movement of the position of the main camera 12 or the sub camera 14 corresponding to each icon. Further, the rotation of each of the icons 62, 63, 64, and 65 corresponds to the change (rotation) of the shooting direction of the main camera 12 or the sub camera 14 corresponding to each icon. Further, the enlargement and reduction of each of the icons 62, 63, 64, and 65 are in the height direction of the main camera 12 or the sub camera 14 corresponding to each icon (height direction in the real world, depth direction in the display of the display unit 23). ), When it is enlarged, it corresponds to the movement in the direction of increasing height, and when it is reduced, it corresponds to the movement in the direction of decreasing height.

The determination in step S200 becomes a negative determination until the operation is completed, and when the operation is completed, the determination in step S200 becomes an affirmative determination, and the process proceeds to step S202. In step S202, the generation unit 42 generates an estimated captured image according to the state after the operation of any of the icons 62, 63, 64, and 65 operated by the user. Specifically, the generation unit 42 derives the position and shooting direction of the main camera 12 or the sub camera 14 with respect to the shooting target S based on the position, orientation, and size of the above icon after the operation, and the derived position. And generate an estimated captured image that is presumed to be captured in the imaging direction.

The specific method by which the generation unit 42 generates the estimated captured image is not particularly limited. For example, the three-dimensional person composition data is stored in the storage unit 22, and the three-dimensional person composition data is rendered according to the position and shooting direction of the main camera 12 or the sub camera 14 with respect to the derived shooting target S. This may generate an estimated captured image.

In the next step S204, the display control unit 44 is estimated to be generated by the generation unit 42 in place of the main camera image 70 or the sub camera image 72 corresponding to any of the icons 62, 63, 64, and 65 operated by the user. The captured image is displayed on the display unit 23. In FIG. 13, when the icon 65 of the recommended bird's-eye view 50 corresponding to the sub camera 14 is operated, the estimated captured image 73 generated by the generation unit 42 is displayed on the display unit 23 instead of the sub camera image 72. An example of the displayed state is shown.

In the next step S206, the designation unit 47 determines whether or not to end the simulation process. The determination in step S206 becomes a negative determination until the end condition is satisfied, the process returns to step S200, and the processes in steps S202 and S204 are repeated. On the other hand, when the user gives an instruction to end the simulation using the operation unit 24, it is determined that the end condition is satisfied, the determination in step S206 becomes an affirmative determination, and the simulation process is terminated.

As described above, the photographing support device 16 of the present embodiment generates an estimated captured image presumed to have been taken by the main camera 12 or the sub camera 14 according to the user's designation, and displays it on the display unit 23.

Therefore, according to the shooting support device 16 of the present embodiment, what kind of main camera image or sub camera image can be obtained by the user without actually moving the position of the main camera 12 or the sub camera 14 by a simple operation. Whether it becomes an image can be grasped from the estimated captured image. Therefore, it is possible to facilitate shooting of the shooting target S, and it is possible to shorten the shooting time.

As described above, the shooting support device 16 of each of the above embodiments is an acquisition unit that acquires a main camera image captured by the main camera 12 of the main camera 12 and the sub camera 14 that captures a specific shooting target S. It is recommended to shoot the shooting target S of each of the main camera 12 and the sub camera 14 and the shooting target S based on the image of the shooting target S included in the main camera image acquired by the acquisition unit 40 and 40. It includes a generation unit 42 that generates a recommended bird's-eye view showing a relative positional relationship, and a display control unit 44 that controls the display unit 23 to display the bird's-eye view generated by the generation unit 42.

With such a configuration, according to the photographing support device 16 of the present embodiment, even a user who is unfamiliar with photographing can easily grasp the setting position of each imaging device. Therefore, according to the photographing support device 16 of each of the above embodiments, when a main camera image and a sub camera image are combined to obtain one moving image, a desired moving image can be obtained.

In each of the above embodiments, the main camera 12 and the sub camera 14 can be connected to the shooting support device 16 at the same time, in other words, the acquisition unit 40 can acquire the main camera image and the sub camera image at the same time. However, the present invention is not limited to each of the above embodiments. For example, only one of the main camera 12 and the sub camera 14 can be connected to the shooting support device 16, and by connecting the main camera 12 and the sub camera 14 alternately, the main camera image and the sub camera by the acquisition unit 40 can be connected. It may be in a form that enables acquisition with a camera image. In the case of this form, the main camera image and the sub camera image acquired by the acquisition unit 40 are not moving images captured in real time, so that the recommended bird's-eye view and the actual position bird's-eye view generated by the generation unit 42 are It may not be a bird's-eye view according to the real-time state.

In each of the above embodiments, two image pickup devices, a main camera 12 and a sub camera 14, used by the user for shooting the shooting target S have been described, but the number of image pickup devices used for shooting is not limited. Further, in each of the above embodiments, the mode in which the main camera image and the sub camera image are moving images has been described, but at least one of the main camera image and the sub camera image may be a still image.

In each of the above embodiments, the hardware structure of the processing unit that executes various processes such as the acquisition unit 40, the generation unit 42, the display control unit 44, the reception unit 46, and the designation unit 47 , The following various processors can be used. As described above, the various processors include CPUs, which are general-purpose processors that execute software (programs) and function as various processing units, as well as circuits after manufacturing FPGAs (Field Programmable Gate Arrays) and the like. Dedicated electricity, which is a processor having a circuit configuration specially designed to execute a specific process such as a programmable logic device (PLD), which is a processor whose configuration can be changed, and an ASIC (Application Specific Integrated Circuit). Circuits etc. are included.

One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA). It may be composed of a combination). Further, a plurality of processing units may be configured by one processor.

As an example of configuring a plurality of processing units with one processor, first, one processor is configured by a combination of one or more CPUs and software, as represented by a computer such as a client and a server. An example is a form in which a processor functions as a plurality of processing units. Secondly, as typified by System On Chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. Can be mentioned. As described above, the various processing units are configured by using one or more of the above-mentioned various processors as a hardware structure.

Further, as the hardware structure of these various processors, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined can be used.

Further, in the above embodiment, the embodiment in which the photographing support program 30 is stored (installed) in the storage unit 22 in advance has been described, but the present invention is not limited to this. The shooting support program 30 is provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a USB (Universal Serial Bus) memory. May be good. Further, the shooting support program 30 may be downloaded from an external device via a network.

All documents, patent applications and technical standards described herein are to the same extent as if the individual documents, patent applications and technical standards were specifically and individually stated to be incorporated by reference. Incorporated by reference in the book.

10 Imaging system

12 Main camera

14 Sub camera

16 Shooting support device

20 CPU

21 memory

22 Memory

23 Display

24 Operation unit

26 External I / F

29 bus

30 Shooting support program

40 Acquisition department

42 Generator

44 Display control unit

46 Reception

47 Designated part

50, 50A, 50B, 50C Recommended bird's-eye view

52 Real position bird's-eye view

60, 62, 63, 64, 65 icons

70 Main camera image

72 Sub camera image

73 Estimated captured image

S Shooting target

Claims (16)

An acquisition unit that acquires an image captured by a predetermined imaging device among a plurality of imaging devices that capture a specific image target, and an acquisition unit.

Based on the image of the shooting target included in the captured image acquired by the acquisition unit, the relative positional relationship recommended for shooting the shooting target of each of the plurality of imaging devices and the shooting target is determined. A generator that generates a recommended bird's-eye view to represent

A display control unit that controls the display to display the recommended bird's-eye view generated by the generation unit.

Shooting support device equipped with.

The generation unit generates the recommended bird's-eye view showing the relative positional relationship of the imaging device different from the predetermined imaging device, which is recommended when the predetermined imaging device is used as a reference. To do,

The shooting support device according to claim 1.

The acquisition unit acquires an image captured by each of the plurality of image pickup devices, and obtains an image captured.

Based on the image of the image to be photographed included in each of the captured images acquired by the acquisition unit, the generation unit is relative to each of the plurality of imaging devices and the image to be photographed. Further generate a bird's-eye view of the actual position that represents the typical positional relationship,

The photographing support device according to claim 1 or 2.

The display control unit controls the display unit to display the recommended bird's-eye view and the bird's-eye view of the actual position in a superposed state.

The shooting support device according to claim 3.

The generation unit is based on at least one of information about the size of the shooting target, information about the position of the shooting target, and information about the orientation of the shooting target, which is obtained by analyzing the captured image. , Generate a bird's-eye view for the recommendations,

The photographing support device according to any one of claims 1 to 4.

The generation unit is based on at least one of information about the size of the shooting target, information about the position of the shooting target, and information about the orientation of the shooting target, which is obtained by analyzing the captured image. Generates the recommended bird's-eye view and the bird's-eye view of the actual position.

The photographing support device according to claim 3 or 4.

The acquisition unit acquires an image captured by each of the plurality of image pickup devices, and obtains an image captured.

The display control unit captures the recommended bird's-eye view generated by the generation unit and at least one image captured by each of the plurality of image pickup devices acquired by the acquisition unit. Control the display on the display unit,

The photographing support device according to any one of claims 1 to 6.

The acquisition unit acquires an image captured by each of the plurality of image pickup devices, and obtains an image captured.

The display control unit is among the recommended bird's-eye view and the bird's-eye view of the actual position generated by the generation unit and the captured images captured by each of the plurality of imaging devices acquired by the acquisition unit. Control is performed to display at least one captured image on the display unit.

The photographing support device according to claim 3, claim 4, or claim 6.

A reception unit that accepts designation of which of the captured images captured by each of the plurality of imaging devices is to be the captured image to be displayed on the display unit is further provided.

The display control unit controls the display unit to display the captured image for which the reception unit has received the designation.

The photographing support device according to claim 7 or 8.

With respect to the recommended bird's-eye view displayed on the display by the display control unit, at least one of the positions and orientations of each of the plurality of imaging devices shown in the recommended bird's-eye view has been changed. It also has a designated part where the state is specified,

The generation unit further generates an estimated captured image that is estimated to be captured by each of the plurality of imaging devices in the changed state.

The display control unit controls the display unit to display the estimated captured image generated by the generation unit.

The photographing support device according to any one of claims 1 to 9.

The plurality of said bird's-eye views shown in the recommended bird's-eye view or the actual position bird's-eye view with respect to the recommended bird's-eye view or the bird's-eye view of the actual position displayed on the display unit by the display control unit. Further, a designation unit for designating the changed state for at least one of the position and the orientation in each of the image pickup devices is provided.

The generation unit further generates an estimated captured image that is estimated to be captured by each of the plurality of imaging devices in the changed state.

The display control unit controls the display unit to display the estimated captured image generated by the generation unit.

The photographing support device according to claim 3, claim 4, claim 6 or claim 8.

A storage unit that stores information representing the relative positional relationship in which the image to be photographed was taken is further provided.

The photographing support device according to any one of claims 1 to 11.

The predetermined imaging device is a main camera mainly used for photographing the imaged object.

The photographing support device according to any one of claims 1 to 12.

The captured image is a moving image.

The photographing support device according to any one of claims 1 to 13.

Acquires an captured image captured by a predetermined imaging device among a plurality of imaging devices that capture a specific shooting target, and obtains the captured image.

Based on the image of the image to be photographed included in the acquired image, the recommended relative positional relationship between each of the plurality of image pickup devices and the object to be photographed is recommended for photographing the object to be photographed. Generate a bird's-eye view and

Controls to display the generated bird's-eye view on the display unit.

A shooting support method in which a computer executes processing.

Acquires an captured image captured by a predetermined imaging device among a plurality of imaging devices that capture a specific shooting target, and obtains the captured image.

Based on the image of the image to be photographed included in the acquired image, the recommended relative positional relationship between each of the plurality of image pickup devices and the object to be photographed is recommended for photographing the object to be photographed. Generate a bird's-eye view and

Controls to display the generated bird's-eye view on the display unit.

A shooting support program that allows a computer to perform processing.