WO2017000792A1

WO2017000792A1 - Automatic panoramic picture shooting system and method

Info

Publication number: WO2017000792A1
Application number: PCT/CN2016/086051
Authority: WO
Inventors: 周勇俊
Original assignee: 芋头科技（杭州）有限公司
Priority date: 2015-06-30
Filing date: 2016-06-16
Publication date: 2017-01-05
Also published as: TWI610570B; CN106331467B; CN106331467A; TW201701645A; HK1231653A1

Abstract

The present invention relates to the technical field of robots, and in particular to an automatic panoramic picture shooting system and method. The automatic panoramic picture shooting system, which is applied to a robot system, comprises: a voice recognition unit for recognizing instruction information in voice information, and the instruction information at least including a photographing instruction and photographing trajectory information; a control unit connected to the voice recognition unit and used to receive the instruction information and generate a control signal to control a corresponding device to act; a moving unit connected to the control unit and being capable of moving from a start position to an end position along a photographing trajectory under the function of the control unit; and an image capturing unit arranged on the moving unit to move along with the moving unit, and the image capturing unit being connected to the control unit, and capturing an image under the function of the control unit. The present invention can automatically shoot a panoramic picture according to a panoramic picture shooting instruction sent by a user, solves the technical problem of how to automatically shoot a panoramic picture, and makes panoramic picture shooting simpler and the effect of a mosaicked panoramic picture better.

Description

Panoramic photo automatic shooting system and method

Technical field

The present invention relates to the field of robot technology, and in particular, to a panoramic photo automatic shooting system and method.

Background technique

At present, the shooting technology of panoramic photos on the market is widely used in handheld devices. The so-called panoramic photo is to take multiple photos from left to right or right to left or from top to bottom or from bottom to top, and then stitch them into a panoramic photo like a jigsaw puzzle. The drawback of this panoramic photo shooting is that multiple photos need to be manually moved horizontally or vertically, and multiple photos taken during the movement may not be on a coincident line, and a large displacement phenomenon occurs, and the final panoramic image is spliced. The expected effect is not achieved or the stitching fails.

Summary of the invention

An object of the present invention is to provide a panoramic photo automatic shooting system that solves the above technical problems;

Another object of the present invention is to provide a method for automatically taking a panoramic photo to solve the above technical problems;

The technical problem solved by the present invention can be implemented by the following technical solutions:

Panoramic photo automatic shooting system for robot systems, including,

a voice recognition unit, configured to identify instruction information in the voice information, where the instruction information is at least Including camera instructions and shooting track information;

a control unit, coupled to the voice recognition unit, configured to receive the instruction information and generate a control signal to control a corresponding device action;

a moving unit, connected to the control unit, and moving from the starting position to the ending position from the starting position under the action of the control unit;

An image capturing unit is disposed on the mobile unit to move with the mobile unit, and the image capturing unit is connected to the control unit to capture an image under the action of the control unit.

In the panoramic photo automatic photographing system of the present invention, the speech recognition unit includes

a voice capture unit for capturing voice information;

And a signal processing unit coupled to the voice capture unit for extracting instruction information in the voice information and/or for positioning the sound source.

The panoramic photo automatic photographing system of the present invention further includes an image recognizing unit connected to the image capturing unit for classifying objects in the image and extracting picture feature information of objects in the image.

The panoramic photo automatic photographing system of the present invention further includes a matching unit connected to the image recognizing unit, and matching the object mark information at the mark start position and the object mark information at the mark end position with the picture feature information to Re-determine the start position, end position, and shooting track of the shot.

The panoramic photo automatic photographing system of the present invention further includes a storage unit for storing the instruction information.

The present invention also provides an automatic photographing method for panoramic photos, which is used in the above-mentioned automatic photographing system for panoramic photographs, including

Step s11, obtaining a starting position of the shooting object, an ending position of the shooting object, and a shooting track by using the voice information sent by the sound source;

Step s12, the moving unit drives the image capturing unit to move from the starting position to the ending position along the shooting track, and capture a group of photos during the moving process;

In step s13, a panoramic photo is generated according to the photo stitching.

The method for automatically taking a panoramic photo of the present invention, wherein the step s11 is specifically as follows:

Step s111a, receiving a voice information for taking a panoramic photo;

Step s112a, extracting instruction information included in the voice information, where the command information includes at least a photographing instruction, an initial starting position, an initial ending position, and an initial shooting trajectory;

Step s113a, generating a control signal under the action of the instruction information to drive the moving unit to move from the initial starting position to the initial shooting position along the initial shooting position, and the image capturing unit is driven by the mobile unit Taking a first set of reference photos along a 360-degree space of the shot track;

Step s114a, extracting picture feature information of each object in the first group of reference photos, and matching and re-determining the starting position and ending by matching the object tag information with the feature information of the object in the first group of reference photos. Location and shooting trajectory.

In the panoramic photo automatic shooting method of the present invention, the step s11 is specifically as follows:

Step s111b, receiving a voice message for taking a panoramic photo;

Step s112b, extracting instruction information included in the voice information, where the command information includes at least a photographing instruction and a shooting track;

Step s113b, positioning a sound source of the voice information, and controlling the image capturing unit to face and track the movement of the sound source;

In step s114b, the image capturing unit tracks the first position to which the sound source is moved as the starting position of the shooting by the moving unit, and tracks the second position to which the sound source is moved as the ending position of the shooting.

Step s111c, receiving a voice information for taking a panoramic photo;

Step s112c, extracting instruction information included in the voice information, where the command information includes at least a photographing instruction and a shooting track;

Step s113c, positioning the sound source of the voice information;

Step s114c, under the control of the voice command of the sound source, the image capturing unit moves to a first position according to the set direction as a starting position of shooting; the image capturing unit continues to move to a second position as a shooting The end position.

Step s111d, receiving a voice information for taking a panoramic photo;

Step s112d, extracting instruction information included in the voice information, where the instruction information includes at least a photographing instruction, a starting position object feature combination, an ending position object feature combination, and a shooting track;

Step s113d, rotating the image capturing unit to acquire a set of reference photos;

Step s114d, extracting the name and picture feature information of each object in the reference photo, and matching the name and feature information of the object in the reference photo by the start position object feature combination to find the starting position The end position is found by matching the end position object feature combination with the name and feature information of the object in the reference photo.

Advantageous Effects: According to the above technical solution, the present invention can automatically take a panoramic photo by issuing a command for taking a panoramic photo by a user, thereby solving the technical problem of automatically taking a panoramic photo, and letting Taking panoramic photos is even simpler, and stitched panoramas work better.

DRAWINGS

1 is a schematic structural view of a system of the present invention;

2 is a schematic flow chart of a method according to a specific embodiment of the present invention;

Figure 3 is a timing diagram of the steps of the method of the present invention;

4 is a schematic flow chart of a method according to another embodiment of the present invention;

FIG. 5 is a schematic flowchart of a method according to another embodiment of the present invention; FIG.

FIG. 6 is a schematic flow chart of a method according to still another embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

It should be noted that the embodiments in the present invention and the features in the embodiments may be combined with each other without conflict.

The invention is further illustrated by the following figures and specific examples, but is not to be construed as limiting.

Referring to Figure 1, a panoramic photo automatic shooting system for a robot system, including,

The voice recognition unit 11 is configured to identify instruction information in the voice information, where the command information includes at least a photographing instruction and a photographing track information;

The control unit 12 is connected to the voice recognition unit 11 for receiving command information and generating a control signal to control the corresponding device action;

The mobile unit 13 is connected to the control unit 12 and moves from the starting position to the end position from the starting position under the action of the control unit 12;

The image capturing unit 14 is disposed on the mobile unit 13 and moves with the mobile unit 13. The image capturing unit 14 is connected to the control unit 12 to capture an image under the action of the control unit 12.

The invention is applied to the field of robots, and uses a rotating device of the robot, that is, a moving unit 13, and an image capturing unit or a camera to take a panoramic photo, and combines object recognition and locking technology, camera taking photo technology, automatic voice recognition technology, The technology of turning the rotating function of the device can automatically take a panoramic photo by the user's instruction to take a panoramic photo, which solves the technical problem of automatically taking a panoramic photo, makes the panoramic photo easier, and the stitched panoramic photo is better.

In the panoramic photo automatic photographing system of the present invention, the speech recognition unit 11 includes

a voice capture unit for capturing voice information;

The signal processing unit is coupled to the voice capture unit for extracting instruction information in the voice information and/or for positioning the sound source.

The panoramic photo automatic photographing system of the present invention further includes an image recognizing unit 15 connected to the image capturing unit 14 for classifying objects in the image and extracting picture feature information of the objects in the image.

The panoramic photo automatic photographing system of the present invention further includes a matching unit 16 connected to the image recognizing unit 15 to match the object mark information of the mark start position and the object mark information of the mark end position and the picture feature information to be newly determined. The start position, end position, and shooting track of the shot.

The panoramic photo automatic photographing system of the present invention further includes a storage unit 17 for storing the command letter interest.

Referring to FIG. 2, FIG. 3, FIG. 4, FIG. 5, and FIG. 6, the present invention further provides a method for automatically capturing a panoramic photo, which is used in the above-mentioned panoramic photo automatic shooting system, including

Step s12, the moving unit drives the image capturing unit to move from the starting position along the shooting track to the ending position, and takes a group of photos during the moving process;

In step s13, a panoramic photo is generated according to the photo stitching.

Referring to FIG. 2, the method for automatically capturing a panoramic photo of the present invention may be as follows:

Step s111a, receiving a voice information for taking a panoramic photo;

Step s112a, extracting instruction information included in the voice information, the command information including at least a photographing instruction, an initial starting position, an initial ending position, and an initial shooting trajectory;

Step s113a, generating a control signal under the action of the command information to drive the moving unit to move from the initial starting position to the initial shooting position from the initial shooting position, and the image capturing unit takes the first group in the 360-degree space of the shooting track under the driving of the moving unit. Reference photo

Step s114a, extracting picture feature information of each object in the first group of reference photos, and matching and re-determining the start position and the end position and the shooting track by matching the object mark information with the feature information of the object in the first set of reference photos .

The following briefly describes the timing diagram in conjunction with FIG. 3: First, the user issues some voice commands for taking panoramic photos, and then extracts object information of the start position and the end position in the voice command by automatic voice recognition technology, and the shooting track (horizontal or vertical). )information. Then the robot turns the Camera A set of reference photos is taken in a horizontal or vertical 360-degree space, and the object is classified by the object recognition image technology, and the picture feature information of each object in the reference photo is extracted. The object mark information is matched with the object feature information in the reference picture, and finally the start position, the end position, and the shooting track (horizontal or vertical) of the panoramic picture are determined. Finally, the robot rotates the position of the object mark information from the Camera to the start position, and according to the shooting trajectory (horizontal or vertical), starts shooting a group of photos until the position of the object mark information at the end position. Finally, this group of photos is stitched together into a panoramic photo.

The method for automatically capturing a panoramic photo of the present invention, referring to FIG. 4, the step s11 can be specifically as follows:

Step s111b, receiving a voice message for taking a panoramic photo;

Step s113b, positioning the sound source of the voice information, and controlling the image capturing unit to face and track the movement of the sound source;

In step s114b, the image capturing unit tracks the first position to which the sound source is moved by the moving unit as the starting position of the shooting, and tracks the second position to which the sound source is moved as the ending position of the shooting. A specific implementation is as follows:

The user sends a panoramic photo command to the robot, such as: "Give me a panoramic photo in horizontal or vertical direction";

Extracting a shooting trajectory, such as horizontal or vertical direction information; the image capturing unit, such as a photographic camera, is positioned by the rotating device and the voiceprint recognition technology to the user who issued the instruction;

The user moves to the starting position of the panoramic photo, and the camera captures the user to the starting position;

The user issues a command to record the starting position of the panoramic photo, such as: "Start shooting panorama from here Photo", extracting the initial photographing position information;

The user moves to the end position where the panoramic photo is taken, and the camera captures the user to the end position;

The user issues an end position command for recording a panoramic photo: "Come here", extracts the end position information of the terminated photograph;

The robot passes the start position, the end position, and shoots the track information, and the mobile camera automatically takes a panoramic photo on the same coincidence line.

The method for automatically capturing a panoramic photo of the present invention, referring to FIG. 5, the step s11 can be specifically as follows:

Step s111c, receiving a voice information for taking a panoramic photo;

Step s113c, positioning the sound source of the voice information;

Step s114c, under the control of the voice command of the sound source, the image capturing unit moves to a first position according to the set direction as the starting position of the shooting; the image capturing unit continues to move to a second position as the ending position of the shooting. A specific embodiment is as follows:

Extracting the shooting track information, the shooting track is horizontal or vertical information, and the camera is positioned by the rotating device and the voiceprint recognition technology to the position of the user who issued the command;

The user observes a preview photo of the robot display system and issues a move command: "Move left or right or up or down." By default, the length of one frame of the photo is moved each time. The length of the movement can be customized by the voice command. When moving to the starting position of the panoramic photo, a voice command is issued: "here as the starting position", and the initial photographing position information is extracted. Same as when moving to the end of taking a panoramic photo Position, voice command: "here as the end position", extract the end position information; the robot passes the start position, the end position, shoots the track information, and the mobile camera automatically takes a panoramic photo on the same coincidence line.

The method for automatically capturing a panoramic photo of the present invention, referring to FIG. 6, the step s11 can be specifically as follows:

Step s111d, receiving a voice information for taking a panoramic photo;

Step s112d, extracting instruction information included in the voice information, the command information includes at least a photographing instruction, a starting position object feature combination, an ending position object feature combination, and a shooting track;

Step s114d, extracting the name and picture feature information of each object in the reference photo, and matching the name of the object in the reference position with the name and feature information of the object in the reference photo to find the starting position; The combination matches the name and feature information of the object in the reference photo to find the end position.

In a specific embodiment, the user issues a panoramic photo command to the robot, such as: "From a white chair with yellow clothes left/right/upper/lower to a brown teapot on the right/left/down/top table, take a panoramic photo. ";

Automatic speech recognition technology parses the starting position object feature combination: "left/right/upper/lower side yellow clothes", "left/right/upper/lower side white chairs", "yellow clothes on white chairs";

Automatic speech recognition technology analyzes the end position object feature combination: "right/left/down/top table", "right/left/down/top teapot", "teapot on the table";

Automatic speech recognition technology analyzes the shooting track information: horizontal/vertical direction;

Take a photo by shooting with the camera and collect a 360-degree reference photo in the horizontal or vertical direction;

Using the object recognition image technology to classify the set of reference photos and extract the reference photos Name and picture feature information of each object in the medium;

Matching the starting position object feature combination and the name of each object extracted by the reference photo and the picture feature information to find the initial photographing position information;

Matching the end position object feature combination and the name of each object extracted by the reference photo and the picture feature information to find the end position information of terminating the photograph;

The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the embodiments and the scope of the present invention, and those skilled in the art should be able to Alternatives and obvious variations are intended to be included within the scope of the invention.

Claims

A panoramic photo automatic shooting system, characterized in that it is used in a robot system, including,

a voice recognition unit, configured to identify command information in the voice information, where the command information includes at least a photographing instruction and a photographing track information;

a control unit, coupled to the voice recognition unit, configured to receive the instruction information and generate a control signal to control a corresponding device action;

a moving unit, connected to the control unit, and moving from the starting position to the ending position from the starting position under the action of the control unit;

An image capturing unit is disposed on the mobile unit to move with the mobile unit, and the image capturing unit is connected to the control unit to capture an image under the action of the control unit.
The panoramic photo automatic photographing system according to claim 1, wherein the speech recognition unit comprises

a voice capture unit for capturing voice information;

And a signal processing unit coupled to the voice capture unit for extracting instruction information in the voice information and/or for positioning the sound source.
The panoramic photo automatic photographing system according to claim 1, further comprising an image recognizing unit coupled to said image capturing unit for classifying objects in said image and extracting objects in said image Picture feature information.
The panoramic photo automatic photographing system according to claim 1, further comprising a matching unit, connected to said image recognizing unit, and the object mark information of the mark start position and the object mark information of the mark end position and the The picture feature information is matched to re-determine the start position, end position, and shooting trajectory of the shooting.
The panoramic photo automatic photographing system according to claim 1, further comprising a storage unit for storing the instruction information.
A panoramic photograph automatic photographing method, characterized in that the panoramic photograph automatic photographing system according to any one of claims 1 to 5, comprising

Step s11, obtaining a starting position of the shooting object, an ending position of the shooting object, and a shooting track by using the voice information sent by the sound source;

Step s12, the moving unit drives the image capturing unit to move from the starting position to the ending position along the shooting track, and capture a group of photos during the moving process;

In step s13, a panoramic photo is generated according to the photo stitching.
The method for automatically photographing a panoramic photo according to claim 6, wherein the step s11 is specifically as follows:

Step s111a, receiving a voice information for taking a panoramic photo;

Step s112a, extracting instruction information included in the voice information, where the command information includes at least a photographing instruction, an initial starting position, an initial ending position, and an initial shooting trajectory;

Step s113a, generating a control signal under the action of the instruction information to drive the moving unit to move from the initial starting position to the initial shooting position along the initial shooting position, and the image capturing unit is driven by the mobile unit Taking a first set of reference photos along a 360-degree space of the shot track;

Step s114a, extracting picture feature information of each object in the first group of reference photos, and matching and re-determining the starting position and ending by matching the object tag information with the feature information of the object in the first group of reference photos. Location and shooting trajectory.
The method for automatically photographing a panoramic photo according to claim 6, wherein said step Step s11 is as follows:

Step s111b, receiving a voice message for taking a panoramic photo;

Step s112b, extracting instruction information included in the voice information, where the command information includes at least a photographing instruction and a shooting track;

Step s113b, positioning a sound source of the voice information, and controlling the image capturing unit to face and track the movement of the sound source;

In step s114b, the image capturing unit tracks the first position to which the sound source is moved as the starting position of the shooting by the moving unit, and tracks the second position to which the sound source is moved as the ending position of the shooting.
The method for automatically photographing a panoramic photo according to claim 6, wherein the step s11 is specifically as follows:

Step s111c, receiving a voice information for taking a panoramic photo;

Step s112c, extracting instruction information included in the voice information, where the command information includes at least a photographing instruction and a shooting track;

Step s113c, positioning the sound source of the voice information;

Step s114c, under the control of the voice command of the sound source, the image capturing unit moves to a first position according to the set direction as a starting position of shooting; the image capturing unit continues to move to a second position as a shooting The end position.
The method for automatically photographing a panoramic photo according to claim 6, wherein the step s11 is specifically as follows:

Step s111d, receiving a voice information for taking a panoramic photo;

Step s112d, extracting instruction information included in the voice information, where the instruction information is at least Including photographing instructions, starting position object feature combination, ending position object feature combination and shooting trajectory;

Step s113d, rotating the image capturing unit to acquire a set of reference photos;

Step s114d, extracting the name and picture feature information of each object in the reference photo, and matching the name and feature information of the object in the reference photo by the start position object feature combination to find the starting position; The end position is found by matching the end position object feature combination with the name and feature information of the object in the reference photo.