JP7081198B2 - Shooting system and shooting control device - Google Patents

Description

The present invention relates to a photographing system and a photographing control device using a drone.

At theme parks and the like, there is a service in which staff take pictures of visitors in order, paste the printed pictures on the original mount, and sell them to the visitors. Shooting by the staff tended to be uninteresting poses and compositions.

When taking a commemorative photo in a theme park, I sometimes asked someone who was there to take a photo. However, if there were no people around, I couldn't shoot. In addition, there are problems such as being nervous and having a hard facial expression because the photograph is taken by another person, and it is troublesome to explain how to use the camera. Sometimes I took a commemorative photo by attaching a photography device to the tip of a stick-shaped instrument such as a selfie stick without asking another person. However, the composition was the same each time, and the maximum number of people who could shoot at one time was about 10.

In recent years, flight-type cameras that perform aerial photography such as so-called drones are becoming widespread. There is a demand for a service to take a commemorative photo using a drone.

Japanese Unexamined Patent Publication No. 2018-10630

An object of the present invention is to provide a photography system and a photography control device for aerial photography of a user by dispatching a drone to the user in response to a request from the user.

The photographing system according to the present invention is a photographing system including a drone equipped with a camera and a photographing control device for controlling the drone based on operation information from a user terminal, and the photographing control device is the user terminal. Receives the position information of the user terminal and the face image of the user, moves the drone to a place corresponding to the position information, collates the photographed image of the camera with the face image, and obtains the photographed image. The camera is controlled so as to identify the user from the inside and take a picture of the specified user.

In one aspect of the present invention, the photographing control device determines whether or not the drone can be used based on the position information of the user terminal.

In one aspect of the present invention, the imaging control device has a storage unit that stores landmark information including the names and positions of a plurality of landmarks, and one of the plurality of landmarks is selected from the user terminal. Then, the position of the drone and the angle of the camera are controlled so that the selected landmark is photographed by the camera with reference to the landmark information.

In one aspect of the present invention, the drone receives radio waves including predetermined identification information from the user terminal, and the photographing control device identifies the user by using the intensity of the radio waves received by the drone. ..

In one aspect of the present invention, when the drone moves to a place corresponding to the position information, the photographing control device notifies the user terminal of the approach of the drone.

In one aspect of the present invention, the photographing control device transmits the image photographed by the camera to the user terminal after identifying the user.

In one aspect of the present invention, the operation information from the user terminal includes at least one of the instruction of the position of the drone and the instruction of the shooting angle of the camera.

The shooting control device of the present invention receives an application for shooting using a drone equipped with a camera from a user terminal, and receives an application receiving unit for receiving position information of the user terminal and a user's face image from the user terminal. The face image receiving unit to be received, the flight control unit for moving the drone to the location corresponding to the position information, the captured image of the camera and the face image are collated, and the user is selected from the captured images. It includes a specific image processing unit and a shooting control unit that controls the camera so as to shoot a specified user.

According to the present invention, it is possible to dispatch a drone to the user in response to a request from the user and take an aerial photograph of the user.

It is a schematic block diagram of the photographing system which concerns on embodiment of this invention. It is a figure which shows the example of the operation screen of a user terminal. It is a schematic block diagram of the imaging control device. It is a functional block diagram of a user terminal. It is a schematic block diagram of a drone. It is a flowchart explaining the photographing method which concerns on the same embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

The photographing system according to the embodiment of the present invention is introduced and installed in a leisure facility such as a theme park. As shown in FIG. 1, when the user (theme park visitor) U applies for a commemorative photo using the user terminal 2, the drone 3 (unmanned aerial vehicle) flies from the base to the user U. Then, the camera 31 mounted on the drone 3 takes a picture. The user terminal 2 is a smartphone, a tablet terminal, or the like owned by the user U. The leisure facility may rent out the user terminal 2.

The user terminal 2 applies for the use of the commemorative photography service to the server device 1 that functions as the photography control device, and transmits the position information (GPS (Global Positioning System) coordinates) of the user terminal 2. When the usage application is accepted, the user terminal 2 transmits the face image of the user U to the server device 1. The face image of the user U may be an image stored in the memory of the user terminal 2, or may be an image taken on the spot by activating the built-in camera of the user terminal 2.

The server device 1 notifies the drone 3 of the GPS coordinates of the user terminal 2. The drone 3 flies to the position indicated by the received GPS coordinates. Drone 3 is a quadcopter that uses, for example, four rotors for takeoff and propulsion. The drone 3 identifies its own position by radio waves from GPS, and autonomously flies to the location indicated by the GPS coordinates of the user terminal 2. Sensors 33 (FIG. 5) may be installed on the front, back, left and right of the drone 3 together with the radio waves from the GPS so that the drone avoids obstacles and flies. A signal from a quasi-zenith satellite system composed of geostationary satellites may be used in combination to improve positioning accuracy.

The drone 3 constantly transmits an image (video) captured by the camera 31 to the server device 1. When the drone 3 flies to the position indicated by the GPS coordinates of the user terminal 2, it stops in the sky and photographs the surroundings. The server device 1 performs face recognition processing using the face image of the user U received from the user terminal 2, and identifies the user U from the camera image of the drone 3.

After identifying the user U, the server device 1 transmits the image captured by the drone 3 to the user terminal 2. The user terminal 2 displays the drone operation screen on the touch panel of its own machine by using the image received from the server device 1. The operation information input by the user U via the operation screen is transmitted to the server device 1. The server device 1 controls the drone 3 and the camera 31 based on the operation information.

For example, the user terminal 2 displays an operation screen as shown in FIG. 2 on the touch panel T. On the operation screen, the image taken by the drone 3 is displayed in live view together with various operation buttons.

When the forward button 201 on the operation screen is pressed, the drone 3 moves forward (moves in a direction approaching the user U). When the retreat button 202 is pressed, the drone 3 retreats (moves in the direction away from the user U).

When the move buttons 203 and 204 are pressed, the drone 3 moves up and down. When the move buttons 205 and 206 are pressed, the drone 3 moves clockwise / counterclockwise around the user U.

The azimuth angle (left-right orientation) of the camera 31 can be adjusted by the camera angle adjustment slider 207. The camera angle adjustment slider 208 can adjust the elevation / depression angle (vertical orientation) of the camera 31.

The shooting mode can be switched by the mode switching button 209. For example, the shooting mode includes "portrait shooting mode", "landscape shooting mode", and "portrait + landscape shooting mode". In the "portrait shooting mode", a person (user) is photographed. In "landscape shooting mode", the surrounding landmarks are shot. In the "person + landscape shooting mode", a person (user) is photographed against the background of the landmark.

When "landscape shooting mode" or "person + landscape shooting mode" is selected, the surrounding landmarks are displayed in a list, and the user U can select which landmarks are included in the shot image.

With the tracking switching button 210, it is possible to switch whether or not to automatically track the user U.

The brightness of the captured image can be adjusted by the brightness adjustment slider 211. The contrast adjustment slider 212 can adjust the contrast of the captured image.

In the remaining time display area 220, the remaining time during which the user U can use the drone 3 is displayed.

When the shooting button 230 is pressed, the shutter operation of the camera 31 is performed and the shooting process is executed. After the shooting process, the drone 3 returns to the base. The captured image data is transmitted by the drone 3 to the server device 1. The photographed image data may be stored in the memory provided in the drone 3, and the photographed image data may be collected from the memory after returning to the base. The captured image data may be transmitted from the server device 1 to the user terminal 2. Alternatively, a printed matter of the photographed image may be sold at the entrance / exit gate.

As described above, according to the present embodiment, the user can take a commemorative photo without asking the surrounding people. Since the shooting is done by the drone 3, it is possible to shoot without being nervous. Since the position of the drone 3 and the shooting angle of the camera 31 can be adjusted, the composition can be freely changed. By shooting from the sky, it is possible to take a commemorative photo with a large number of people.

Next, the configuration of the server device 1 will be described. The server device 1 may be composed of one computer or a plurality of computers. As shown in FIG. 3, the server device 1 includes a control unit 10, a storage unit M, and a communication unit C. The control unit 10 is a central processing unit (CPU) that controls the entire server device 1. The control unit 10 realizes various functions by reading and executing the operating system (OS) and the application program stored in the storage unit M.

The storage unit M is a storage area for storing programs, data, and the like necessary for the control unit 10 to execute various processes, and a hard disk, a semiconductor memory, or the like is used. The storage unit M has a landmark information DB (database) and a captured image DB.

The landmark information DB is a storage area for storing landmark information in which the names and positions of features such as buildings, structures, monuments, and mountains in or visible from the theme park are recorded.

The communication unit C communicates with the user terminal 2 and the drone 3 via a network such as the Internet or a LAN.

The control unit 10 includes a shooting application receiving unit 11, a face image receiving unit 12, an approach notification unit 13, an image transmitting unit 14, an operation information receiving unit 15, an image receiving unit 16, an image processing unit 17, a flight control unit 18, and shooting. It has the function of the control unit 19.

The photography application reception unit 11 receives an application for using the commemorative photography service from the user terminal 2. The shooting application reception unit 11 calculates the distance between the drone base and the user U from the GPS coordinates transmitted from the user terminal 2 together with the usage application, and if it is determined that the user U is too far from the drone base, the usage application is made. You may cancel the reception. Further, depending on the weather, it may be determined whether or not to accept the application.

The face image receiving unit 12 receives the face image of the user U from the user terminal 2. The received face image is temporarily stored in the storage unit M and deleted after using the commemorative photography service.

When the drone 3 flies to the position of the GPS coordinates received from the user terminal 2, the approach notification unit 13 notifies the user terminal 2 of the approach of the drone 3.

The image transmission unit 14 transmits an image (video) taken by the camera 31 of the drone 3 to the user terminal 2. The video transmitted to the user terminal 2 is displayed on the user terminal 2 together with the operation screen. Further, when the "landscape shooting mode" or the "person + landscape shooting mode" is selected as the shooting mode in the user terminal 2, the image transmission unit 14 can shoot from the position of the user U by referring to the landmark information. The name of the landmark and the like are transmitted to the user terminal 2.

The operation information receiving unit 15 receives from the user terminal 2 the operation information input by the user terminal 2 from the user U via the operation screen.

The image receiving unit 16 receives a captured image (still image or video (moving image)) of the camera 31 from the drone 3. The image receiving unit 16 stores the still image received from the drone 3 in the captured image DB of the storage unit M.

The image processing unit 17 collates the camera image of the drone 3 with the face image of the user U, and identifies the user U from the camera images. Obstacles may be detected by analyzing the camera image of the drone 3.

The flight control unit 18 notifies the drone 3 of the GPS coordinates received from the user terminal 2 and instructs the drone 3 to fly to the position of the user U. When an obstacle is detected by the image processing unit 17, an avoidance instruction may be issued. The flight control unit 18 controls the position of the drone 3 based on the operation information from the user U. Further, when a landmark is selected from the user U in the "landscape shooting mode" or the "person + landscape shooting mode", the flight control unit 18 determines the position and attitude of the drone 3 so that the selected landmark can be shot. , The angle of the camera 31 and the like are controlled.

The shooting control unit 19 adjusts the angle (orientation) of the camera 31 so that the specified user U is within the shooting range. Further, the shooting control unit 19 controls the angle, brightness, contrast, etc. of the camera 31 based on the operation information from the user U, and causes the camera 31 to execute the shooting process. The shooting control unit 19 may automatically determine the shooting settings (brightness, contrast) suitable for the composition based on the operation information from the user U, and control the camera 31.

FIG. 4 is a functional block diagram of the user terminal 2. The user terminal 2 is, for example, a smartphone or tablet terminal equipped with a GPS module, and by executing an application program installed in the terminal in advance, a shooting application unit 21 and a position information transmission unit as shown in FIG. 4 are executed. 22, the face image transmission unit 23, the approach notification unit 24, the operation screen display unit 25, and the operation information transmission unit 26 are realized.

The photography application unit 21 applies to the server device 1 to use the commemorative photography service.

The position information transmission unit 22 transmits the position information (GPS coordinates) of the user terminal 2 to the server device 1.

When the server device 1 accepts the application for using the commemorative photo service, the face image transmission unit 23 transmits the face image of the user U to the server device 1. The face image to be transmitted may be an image stored in the memory of the user terminal 2 or may be an image taken by activating the built-in camera of the user terminal 2.

The approach notification unit 24 notifies the user U that the drone 3 is nearby by outputting an alarm or using the vibration function in response to the drone approach notification from the server device 1.

The operation screen display unit 25 displays the camera image of the drone 3 received from the server device 1 in a live view, and displays an operation screen (see FIG. 2) including various operation buttons on the touch panel T.

The operation information transmission unit 26 transmits the operation information input from the user U to the server device 1 via the operation screen.

FIG. 5 is a schematic configuration diagram of the drone 3. The drone 3 includes a drone control unit 30, a camera 31, a GPS unit 32, a sensor 33, a motor 34, a communication unit 35, a battery 36, and the like.

The GPS unit 32 receives a signal from the positioning satellite. The sensor 33 includes, for example, a flight state sensor configured by a 6-axis gyro sensor and a radar for measuring a downward distance.

The communication unit 35 communicates with the server device 1.

The drone control unit 30 has the functions of the positioning unit 302, the flight control unit 304, and the information transmission unit 306. The positioning unit 302 positions the position of the drone 3 based on the positioning result of the GPS unit 32.

The flight control unit 304 controls the rotation speed of the rotor motor 34 based on the instruction from the server device 1 and the detection result of the sensor 33, and the flight altitude, flight direction, flight speed, and flight attitude of the drone 3. Etc. are controlled.

The information transmission unit 306 transmits the shooting data by the camera 31, the position information of the drone 3, the posture information, and the like to the server device 1.

Next, a commemorative photography method using the photography system according to the present embodiment will be described.

The user U activates the application program of the user terminal 2 and applies for the use of the commemorative photo service (step S1). Along with the usage application, the location information of the user terminal 2 is transmitted to the server device 1.

The server device 1 determines whether or not to accept the application based on the weather and the position of the user U. When the application is accepted (step S2_Yes), the user terminal 2 transmits the face image of the user U to the server device 1 (step S3).

The server device 1 transmits the position information of the user terminal 2 to the drone 3, gives a flight instruction, and dispatches the drone 3 to the user U (step S4).

When the drone 3 reaches the position indicated by the position information of the user terminal 2, the server device 1 collates the face of the person shown in the camera image of the drone 3 with the face image of the user U. When the user U is specified (step S5_Yes), the server device 1 transmits the camera image of the drone 3 to the user terminal 2.

The user terminal 2 displays an operation screen including various operation buttons together with the camera image of the drone 3. The operation information input by the user U via the operation screen is given to the drone 3 via the server device 1. The user U adjusts the position of the drone 3 and the orientation of the camera 31 to specify the composition (step S6).

When the shooting process by the camera 31 is performed (step S7), the drone 3 autonomously flies to the base and returns (step S8).

The user can call the drone 3 using the user terminal 2 and take a commemorative photo. Since the position of the drone 3 and the angle of the camera 31 can be adjusted, the composition can be freely changed. The user does not have to bring in a drone, and can easily take a commemorative photo.

The user terminal 2 may transmit the identification information included in the radio wave transmitted by the own terminal, such as the SSID of wifi and the connection ID of Bluetooth (registered trademark), to the server device 1 together with the face image. The server device 1 notifies the drone 3 of this identification information. When the drone 3 flies to the position of the user U, the drone 3 receives the radio wave including the identification information notified from the server device 1 and transmits the radio wave strength to the server device 1. The server device 1 can narrow down the position of the user terminal 2 and identify the user from the change in the radio wave strength when the drone 3 is moved.

In the above embodiment, if the remaining time becomes zero before shooting, it may be confirmed whether or not to extend the time, or the drone 3 may automatically return.

The drone 3 may return to the base once after shooting, or may fly to the position of the next user if the battery level is sufficient.

It should be noted that the present invention is not limited to the above embodiment as it is, and at the implementation stage, the components can be modified and embodied within a range that does not deviate from the gist thereof. In addition, various inventions can be formed by an appropriate combination of the plurality of components disclosed in the above-described embodiment. For example, some components may be removed from all the components shown in the embodiments. Furthermore, components over different embodiments may be combined as appropriate.

1 Server device 2 User terminal 3 Drone

Claims (7)

With a drone equipped with a camera,
A shooting control device that controls the drone based on operation information from the user terminal,
It is a shooting system equipped with
The photographing control device receives the position information of the user terminal and the face image of the user from the user terminal, moves the drone to a place corresponding to the position information, and captures the image taken by the camera and the face image. The camera is controlled so as to identify the user from the captured images, capture the identified user, and store the landmark information including the names and positions of a plurality of landmarks. When any one of the plurality of landmarks is selected from the user terminal, the position of the drone is referred to with the landmark information so that the selected landmark is photographed by the camera. And a shooting system characterized by controlling the angle of the camera . The imaging system according to claim 1, wherein the imaging control device determines whether or not the drone can be used based on the position information of the user terminal. The drone receives radio waves including predetermined identification information from the user terminal and receives radio waves.
The imaging system according to claim 1 or 2 , wherein the imaging control device identifies the user by using the intensity of radio waves received by the drone. The imaging control device according to any one of claims 1 to 3 , wherein when the drone moves to a place corresponding to the position information, the user terminal is notified of the approach of the drone. Shooting system. The imaging system according to any one of claims 1 to 4 , wherein the imaging control device transmits an image captured by the camera to the user terminal after the user is specified. The photographing system according to any one of claims 1 to 5 , wherein the operation information from the user terminal includes at least one of the instruction of the position of the drone and the instruction of the photographing angle of the camera. .. An application reception unit that accepts applications for shooting using a drone equipped with a camera from the user terminal and receives the location information of the user terminal, and
A face image receiving unit that receives a user's face image from the user terminal,
A flight control unit that moves the drone to a location corresponding to the location information,
An image processing unit that collates the captured image of the camera with the face image and identifies the user from the captured image.
A shooting control unit that controls the camera to shoot a specified user,
A storage unit that stores landmark information including the names and positions of multiple landmarks,
Equipped with
When any of the plurality of landmarks is selected by the user terminal, the position of the drone and the angle of the camera are referred to so that the selected landmark is photographed by the camera. An imaging control device characterized by controlling the camera.

Images