JP6384891B1 - Information terminal device, photographing device, photographing method and photographing program - Google Patents

Abstract

Photographing can be performed at a position and composition desired by a user with a simple configuration. An information terminal device includes a memory for specifying an imaging target and recording imaging target specifying information for specifying a specific operation state of the imaging target, and the imaging target of the specific operation state includes an imaging unit. A communication unit that transmits the imaging target specifying information for instructing movement and imaging of the imaging range of the imaging unit to a moving imaging apparatus that performs control to move the imaging range of the imaging unit so that the imaging range is captured And. [Selection] Figure 1

Description

  The present invention relates to an information terminal device, a photographing device, a photographing method, and a photographing program suitable for an autonomous moving body such as a robot cleaner.

In recent years, mobile bodies that can move autonomously, such as robot cleaners and drones, have been commercialized. Such mobile objects can be used for various activities, such as for activities in dangerous places, for activities in places where it is difficult for humans to enter, or for the purpose of releasing troublesome or tedious work for humans. It is used for the purpose.
For example, Patent Document 1 discloses a robot that moves in accordance with a user instruction.

JP 2006-263844 A

In the technology of Patent Document 1, it is necessary to provide the target position to the robot. However, in the future, it is becoming an era in which devices are required to execute what the user wants in cooperation. The challenge is how to carry out robot control without careful handling.
An object of the present invention is to provide an information terminal device, a photographing device, a photographing method, and a photographing program capable of photographing at a position and composition desired by a user with a simple configuration.

  An information terminal device according to an aspect of the present invention includes: a memory that records imaging target specifying information for specifying an imaging target and a specific operating state of the imaging target; and the imaging target of the specific operating state includes The imaging target specifying information for instructing movement and imaging of the imaging range of the imaging unit is transmitted to a moving imaging apparatus that performs control to move the imaging range of the imaging unit so that the imaging unit captures an image. And a communication unit.

  An imaging method according to an aspect of the present invention includes a step of specifying an imaging target and recording imaging target specifying information for specifying a specific operation state of the imaging target, and capturing the imaging target in the specific operation state. Communication for transmitting the imaging target specifying information for instructing movement and imaging of the imaging range of the imaging unit to a moving imaging apparatus that performs control to move the imaging range of the imaging unit so that the imaging unit captures an image. Steps.

  An imaging program according to an aspect of the present invention includes a step of recording an imaging target specifying information for specifying an imaging target and a specific operation state of the imaging target in a computer, and the imaging of the specific operation state. The imaging target specifying information for instructing movement and imaging of the imaging range of the imaging unit to a moving imaging apparatus that performs control to move the imaging range of the imaging unit so that the target is imaged by the imaging unit. And a communication step of transmitting.

  In the imaging device according to one aspect of the present invention, the imaging target specifying information for specifying the imaging target is given in the imaging device whose imaging range is moved by the movement of the stage to which the imaging unit is attached. The imaging target designated by the movement of the stage is detected by image analysis on the captured image being acquired by the imaging unit by the movement of the stage, and the detected imaging target is imaged by the imaging unit. And a control unit for controlling.

  An imaging device according to another aspect of the present invention includes an imaging unit, and the imaging device whose imaging range is moved by moving the stage is provided with imaging target specifying information for specifying an imaging target, so that the imaging is performed. In order to detect the imaging target specified by the target specifying information by image analysis on a captured image being acquired by the imaging unit by moving the stage, and to capture the detected imaging target by the imaging unit. And a control unit for controlling the movement of the stage.

  An information terminal device according to an aspect of the present invention includes a communication unit that transmits the imaging target specifying information to the imaging device.

  According to the present invention, it is possible to perform photographing at a position and composition desired by a user with a simple configuration.

1 is a block diagram showing a mobile imaging device and a mobile imaging control device according to an embodiment of the present invention. Explanatory drawing for demonstrating the information memorize | stored in the target object information memory 25 in FIG. Explanatory drawing for demonstrating the mode of utilization in embodiment. 6 is a flowchart for explaining the operation of the mobile photographing control device 20; 4 is a flowchart for explaining the operation of the mobile photographing apparatus 10; Explanatory drawing for demonstrating a mode that the mobile imaging | photography control apparatus 20 and the mobile imaging | photography apparatus 10 perform imaging | photography in cooperation. Explanatory drawing for demonstrating a mode that the mobile imaging | photography control apparatus 20 and the mobile imaging | photography apparatus 10 perform imaging | photography in cooperation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing a mobile photographing apparatus and a mobile photographing control apparatus according to an embodiment of the present invention.

  The mobile imaging device 10 in the present embodiment is autonomous and movable. As the mobile imaging device 10, various known devices that can move autonomously may be used. For example, as the mobile photographing device 10, a robot cleaner that can move in the room may be employed, an automatic traveling vehicle that moves on the ground may be employed, or an unmanned aircraft such as a drone may be employed. You may employ | adopt the autonomous ship etc. which move on the water.

  In FIG. 1, the mobile photographing apparatus 10 has a control unit 11. The control unit 11 may be configured by a processor using a CPU or the like (not shown), or may realize each function according to a program stored in a memory (not shown). The moving photographing apparatus 10 is provided with a moving unit 12. The moving unit 12 has various moving mechanisms for moving the mobile photographing device 10. For example, when the mobile photographing device 10 uses a robot cleaner or the like, wheels are employed as the moving mechanism. Further, when the mobile photographing device 10 uses a drone, a propeller is employed as the moving mechanism. Further, as the mobile photographing device 10, a robot that moves a leg may be adopted, or a robot that moves by a caterpillar may be adopted. The moving unit 12 includes a driving mechanism such as a motor or an engine that drives the moving mechanism. The moving unit 12 is controlled by the control unit 11 and can move the moving photographing apparatus 10 in a desired direction.

  The mobile photographing device 10 is provided with an information acquisition unit 13. The information acquisition unit 13 includes an imaging unit 13a. The imaging unit 13a is provided with an imaging element (not shown) configured by a CCD, a CMOS sensor, or the like, and a subject optical image is guided to the imaging surface of the imaging element by an optical system (not shown). In the optical system, there are configured a mechanism unit that drives a focus lens, a zoom lens, a diaphragm, and the like (not shown) and a drive unit that drives these mechanism units.

  The control unit 11 is provided with an imaging control unit 11a. The imaging control unit 11a drives and controls a driving unit in the optical system, and controls driving of a focus lens, a zoom lens, and a diaphragm configured in the optical system. It is like that. The imaging element photoelectrically converts the subject optical image to obtain a captured image of a moving image or a still image. This captured image is given to the control unit 11.

  In addition, the tracking control unit 17 is provided in the mobile imaging device 10. The tracking control unit 17 can be controlled by the control unit 11 to track a predetermined subject in the captured image acquired by the imaging unit 13a. For example, the tracking control unit 17 can control the angle of view so that the subject being tracked is included within the field of view of the imaging unit 13a even when the subject being tracked moves.

  The mobile photographing device 10 is provided with a communication unit 16, and the communication unit 16 is controlled by the control unit 11 to transmit / receive information to / from an external device such as the mobile photographing control device 20 by wire or wirelessly. Can be done. The control unit 11 can also transmit a captured image captured by the imaging unit 13 a to an external device via the communication unit 16. Further, a memory may be provided in the mobile photographing device 10 so that a captured image acquired by the imaging unit 13a is stored in this memory.

  Note that the communication unit 16 and the communication unit 22 to be described later are configured by, for example, NFC (Near Field Communication) or BLE (Bluetooth (registered trademark) Low Energy) when performing wireless communication, thereby reducing power consumption. It can be made sufficiently small.

  Moreover, the information acquisition part 13 may be provided with not only the imaging part 13a but the sound collection part (not shown). The sound collection unit can collect ambient sounds of the mobile imaging device 10 and output sound information to the control unit 11. Also about this audio | voice information, the control part 11 can also be transmitted to an external apparatus via the communication part 16, and can also be memorize | stored in the memory for recording which is not shown in figure.

  Note that the imaging unit 13a can capture a predetermined direction around the mobile imaging device 10 to obtain a captured image. The imaging unit 13a is preferably capable of imaging in an arbitrary direction with respect to the moving direction of the mobile imaging device 10. For example, when the mobile imaging device 10 is configured by a cylindrical or polygonal columnar robot cleaner, the optical system of the imaging unit 13a can be attached to the side surface of the mobile imaging device 10 so that the viewing direction is directed to a predetermined direction. As described above, it is preferable that the mobile photographing device 10 is configured to be rotatable with respect to the floor surface so that the side surface can be directed in an arbitrary direction.

The control unit 11 can move the mobile photographing device 10 according to a predetermined program. For example, when the mobile imaging device 10 is configured by a robot cleaner, the movement of the mobile imaging device 10 is controlled so as to clean the entire room where the robot cleaner is disposed. For example, the mobile imaging device 10 configured by a robot cleaner may employ a method of moving so as to clean the entire room based on a random movement pattern, and the room may be based on an image captured during movement. There is a case where a method of moving to clean the entire room using this map is sometimes used. In the mobile photographing device 10, the movement may be controlled by any of these methods.
In this way, robots and the like constituting mobile imaging devices have evolved in a direction that allows them to move autonomously while avoiding obstacles. Confirmation of the location of the user, determination of a reasonable and safe movement route using map information, etc. have been achieved by a program including learning by artificial intelligence. Examples of the sensor include a GPS, a laser radar, an altimeter, an acceleration sensor, a geomagnetic sensor, and the like, and the current position may be estimated from the travel distance and travel direction from the reference position. An imaging unit or the like can also be used.

  Also in the present embodiment, the mobile imaging device 10 includes not only the imaging unit 13a but also the map information unit 14, and the map information unit 14 uses the map information previously recorded in the recording unit. However, a map of the room may be created based on the image obtained by the imaging unit 13a during movement, and the map information may be stored in the map information storage unit 14a. Thus, if there is map information and the user's position is known, it is easy to determine how to reach the target position from the obstacle position included in the map information. This obstacle position may also be recorded in advance, or may be moved and learned. As will be described later, even if the map information is not provided, some functions in the present embodiment can be achieved, and the creation and holding of the map information is not necessarily an essential configuration in the embodiment. .

  Furthermore, in the present embodiment, the control unit 11 includes a photographing movement control unit 11b, and the photographing movement control unit 11b can control the movement of the mobile photographing apparatus 10 for imaging. It is like that. In the present embodiment, the imaging movement control unit 11 b controls the movement of the mobile imaging device 10 based on the information stored in the object information memory 15. The object information memory 15 stores information (hereinafter, referred to as object specifying information) for specifying an object such as a subject desired by the user. In the present embodiment, the control unit 11 receives the object specifying information from the mobile imaging control device 20.

  The mobile photographing control device 20 as the mobile photographing instruction device can be configured by, for example, a photographing device such as a digital camera or a smartphone having a photographing function. The mobile photographing control device 20 has a control unit 21. The control unit 21 may be configured by a processor using a CPU (not shown) or the like, and may realize each function according to a program stored in a memory (not shown). The mobile photographing control device 20 is provided with a communication unit 22, and the communication unit 22 is controlled by the control unit 21 so that information can be transmitted to and received from the communication unit 16 of the mobile photographing device 10.

  The mobile imaging control device 20 has an imaging unit 23. The image pickup unit 23 is provided with an image pickup device (not shown) configured by a CCD, a CMOS sensor, or the like, and a subject optical image is guided to the image pickup surface of the image pickup device by an optical system (not shown). In the optical system, there are configured a mechanism unit that drives a focus lens, a zoom lens, a diaphragm, and the like (not shown) and a drive unit that drives these mechanism units.

  The control unit 21 drives and controls a drive unit in the optical system to drive and control a focus lens, a zoom lens, and an aperture configured in the optical system. The imaging element photoelectrically converts the subject optical image to obtain a captured image of a moving image or a still image. This captured image is given to the control unit 21. The control unit 21 provides the captured image to the recording unit 27. The recording unit 27 records the given captured image on a predetermined recording medium. Further, the control unit 21 provides the captured image to the display unit 26. The display unit 26 has a display screen configured by an LCD or the like, and can display a given captured image on the display screen.

  The mobile photographing control device 20 is provided with an operation input unit 24. The operation input unit 24 includes a shutter button, a dial, a function button, various switches, and the like (not shown), accepts a user operation, and gives an operation signal to the control unit 21. Note that a touch panel (not shown) provided on the display screen of the display unit 26 may be employed as the operation input unit 24. The touch panel can generate an operation signal corresponding to the position on the display screen pointed by the user with a finger. An operation signal from the operation input unit 24 is supplied to the control unit 21. The control unit 21 executes processing according to an operation signal from the operation input unit 24.

  In the present embodiment, the mobile photographing control device 20 is provided with an object information memory 25. The object information memory 25 can store the object specifying information given from the control unit 21. The control unit 21 generates the object specifying information based on the information recorded in the recording unit 27 and the user operation of the operation input unit 24, gives the object specifying information to the object information memory 25, and stores it.

  The object specifying information is information for specifying an object captured by the mobile imaging device 10. Note that the phrase “capture” means a state in which the mobile imaging device 10 can take some action on the object. As the action, for example, the mobile imaging device 10 tracks the object. Or the mobile imaging device 10 images the object.

  For example, the object specifying information is for specifying a subject that is an imaging target in the mobile imaging device 10. Further, the object specifying information may include information for the mobile imaging device 10 to capture the object under the condition (aspect condition) on the condition of a predetermined aspect of the object. For example, when specifying a person as a subject, the object specifying information may be information for specifying the person, and may include information on a condition condition of a predetermined pose of the person. When such object specifying information is given, the mobile imaging device 10 can capture and capture the person in a predetermined pose. Note that the aspect condition may determine not only the pose of the object at the time of imaging but also the composition at the time of imaging. For example, the shooting position and the background may be set depending on the aspect condition.

  For example, the control unit 21 acquires information for specifying the target object as target object identification information using the recognition result of a person, a pet, an object, or the like in the captured image recorded in the recording unit 27 as the target object. Also good. The control unit 21 can recognize a person, a pet, and various objects from the captured image by using a known face recognition technique or the like. Moreover, the control part 21 may specify a target object by the character recognition with respect to the character in a captured image, and may acquire target object specific information. Further, for example, the control unit 21 may acquire the target specifying information by specifying the target based on the meta information added to the captured image recorded in the recording unit 27. Further, for example, the control unit 21 may acquire the object specifying information by specifying the object based on a user operation on the operation input unit 24.

  Further, the control unit 21 may use frequency information appearing in the captured image recorded in the recording unit 27 when specifying the object. For example, the control unit 21 sets a person who appears most frequently in the captured image recorded in the recording unit 27 as a target, and acquires information for specifying the person as the target specifying information. Also good.

  In addition, for example, the control unit 21 may set object specifying information including a mode condition such as a predetermined pose for a person obtained from a result of face recognition or the like. For example, when the person who is most frequently appearing in the captured image recorded in the recording unit 27 is a baby, the control unit 21 uses the state where the baby is standing and stands as a mode condition. The object specifying information in which the baby when the condition is satisfied is set as the object may be generated.

The control unit 21 may acquire the aspect condition to be set based on a user operation by the operation input unit 24 or may be acquired by image analysis on the captured image. For example, it is assumed that the control unit 21 performs face recognition processing on the captured image recorded in the recording unit 27 and selects an object that appears most frequently in the captured image. In this case, when the target object is a baby, the control unit 21 performs image analysis of the baby image, and the captured image of the baby is accumulated, for example, for more than six months, or sits on a butt. When it is determined that there is no image of the baby that is holding and standing, the aspect condition may be that the baby is holding and standing.
In this way, it is possible to determine what kind of captured image is preferred by analyzing the tendency of the image on the Internet, and learning this from the results of posting in these databases and the results of writing of the viewers, etc. Alternatively, it may be programmed in advance. A photographing spot is determined from the preferred photographed image. If an image with a sofa in the background is preferred, the location of the sofa becomes the shooting spot. Note that it is not always necessary to move so precisely, and the object photographing spot where the object is to be photographed and the camera photographing spot where the camera is easy to photograph the object located there are similar but not the same. It must be noted. However, here, since it is possible to take a picture if it is close, the explanation is made as being substantially the same place. It also expresses the vicinity of the shooting position.

  FIG. 2 is an explanatory diagram for explaining information stored in the object information memory 25 in FIG. The example of FIG. 2 shows an example in which there are three records R1 to R3 as the object specifying information. Each record R1 to R3 includes information of items i1 to i4. The item i1 is image information as a result of face recognition, the item i2 is information of a name corresponding to the face recognition result, the item i3 is information of mode conditions, and the item i4 is information of priority. Each record R1 to R3 in FIG. 2 shows information about the father, mother, and baby, respectively. In the example of FIG. 2, the object set with the highest priority 1 is the baby shown by the user family 2, It shows that “holding standing” designated as a desired image is set as a mode condition.

When the object is a baby, it is possible to set a mode condition such as a scene where the baby stands for the first time, the first time the baby starts walking, a scene of playing with a stuffed animal, etc. . For example, when a robot cleaner is adopted as the mobile photographing device 10, the robot cleaner can take images for a relatively long time while moving in a room where there are no adults. There is a high possibility of capturing the scene. Moreover, the robot cleaner can take an image from a position close to the floor, and sometimes a quaint photo that is different from a normal photo taken by an adult may be taken.
The robot cleaner may record in advance the place where it is easy to get caught or where the stuffed toy is, or it may be grasped in advance by image judgment on a daily basis during movement such as during cleaning. . If there is a sofa or the like in the popular image on the net, it may be determined from the image or the distance distribution whether there is a similar one.

  The control unit 21 controls the communication unit 22 to search for an autonomous mobile body that has a communication function and can perform some action on an object such as an imaging function. When the control unit 21 succeeds in searching for such a moving object, the control unit 21 performs communication with the moving object. When this communication is established, the control unit 21 transmits the object specifying information stored in the object information memory 25 to the mobile body. For example, when communication is established with the mobile imaging device 10, the control unit 21 transmits the object specifying information stored in the object information memory 25 to the mobile imaging device 10.

  Note that the control unit 21 may transmit the object specifying information to a moving body such as the mobile imaging device 10 when the user performs a transmission operation using the operation input unit 24.

  In the present embodiment, the object specifying information is generated based on the captured image acquired by the user's shooting operation or the user's input operation, and is considered to reflect the user's preference. . In other words, the mobile shooting control device 20 generates target specifying information that reflects the user's preference by using a normal camera that captures a subject desired by the user. Then, the mobile imaging control device 20 transmits the generated object identification information to the mobile imaging device 10 that can perform imaging while moving autonomously, so that the mobile imaging control device 20 can detect the position, angle of view, timing, The user can obtain his / her favorite captured image without performing a shooting operation.

  For example, in a meeting of people who love animal pets, by taking a picture of each pet in advance and recording it in the recording unit 27, the object specifying information is generated based on the picture of the pet, and this object It is also possible to automatically image each pet coming to the meeting based on the object identification information.

  When receiving the object specifying information via the communication unit 16, the control unit 11 of the mobile imaging device 10 gives the received object specifying information to the object information memory 15 to be stored. The imaging movement control unit 11b reads the object identification information stored in the object information memory 15 and controls the movement of the mobile imaging apparatus 10. For example, the imaging movement control unit 11b moves the mobile imaging device 10 so as to capture the specified object with the aspect condition specified by the object specifying information. For example, the imaging movement control unit 11b of the mobile imaging device 10 performs image analysis on the captured image acquired by the imaging unit 13a, and whether or not an image of the object specified by the object specifying information exists in the image. If it exists, it is possible to control the tracking control unit 17 and the moving unit 12 and continue to capture the object, for example.

  As described above, the information acquisition unit 13 of the mobile imaging device 10 has a function of capturing an object and a detection function for capturing the object. For this detection function, the information acquisition unit 13 may employ various sensors in addition to the imaging unit 13a. For example, as the information acquisition unit 13, an appropriate sensor such as a radar device, an atmospheric pressure sensor, an acceleration sensor, or a gyroscope may be employed.

  Further, for example, the shooting movement control unit 11b can move the moving shooting apparatus 10 to a position where the specified aspect condition of the target specified by the target specifying information can be imaged as a subject.

  For example, when the target specified by the target specifying information exists in the imaging range, the shooting movement control unit 11b tracks the mobile target while keeping tracking the target in the imaging range. 10 is moved. Furthermore, when the position where the object is imaged is defined depending on the aspect condition, the imaging movement control unit 11b moves to the position regardless of whether the object is included in the imaging range. The apparatus 10 may be moved.

  For example, the control unit 21 of the mobile photographing control device 20 often picks up a baby standing on the couch, because the baby often uses the stepped portion such as the sofa or the stairs to stand up. The aspect condition may be set so that imaging is performed at a stepped portion such as a stairs. In addition, for example, the control unit 21 has information on popular scenes for each age, and can capture images of popular scenes according to the age and sex of the target using a known gender and age estimation algorithm based on face recognition. Aspect conditions may be set to be performed. Such aspect conditions suitable for imaging may be templated according to age, sex, etc., and the control unit 21 reads such a template from a memory (not shown) or a computer on the cloud. Thus, the aspect condition may be set.

  Note that the control unit 21 may access the computer on the cloud via the communication unit 22 and use the artificial intelligence on the cloud to specify the target and set the aspect condition.

  In addition, when the imaging unit 23 captures an object set by the object identification information during imaging, the control unit 21 transmits information indicating that the object has been captured to the control unit 11 of the mobile imaging device 10. May be configured such that the mobile imaging control device 20 and the mobile imaging device 10 perform imaging in cooperation.

  Next, the operation of the embodiment configured as described above will be described with reference to FIGS. FIG. 3 is an explanatory diagram for explaining a state of use in the embodiment. FIG. 4 is a flowchart for explaining the operation of the mobile photographing control device 20, and FIG. 5 is a flowchart for explaining the operation of the mobile photographing device 10. 6 and 7 are explanatory diagrams for explaining how the mobile imaging control device 20 and the mobile imaging device 10 perform imaging in cooperation with each other.

  FIG. 3 shows an example in which an imaging device 32 is employed as the mobile imaging control device 20 and a robot cleaner 31 is employed as the mobile imaging device 10 to image a baby 33 as a subject. The photographing device 32 includes each circuit of the mobile photographing control device 20 of FIG. The imaging device 32 includes a lens barrel 32 a and a lens 32 b that constitute the imaging unit 23 in FIG. 1, and acquires captured images of various subjects including the baby 33. The control unit 21 provided in the photographing device 32 generates the object specifying information based on the acquired captured image.

  The robot cleaner 31 includes each circuit of the mobile photographing device 10 of FIG. The robot cleaner 31 includes wheels 31a constituting the moving unit 12, and is movable in a desired direction by rotating the wheels 31a. The robot cleaner 31 has a cylindrical housing, and an imaging unit 13a having an imaging range in a predetermined radial direction from the side surface of the housing is provided on the side surface.

  In step S <b> 1 of FIG. 4, the control unit 21 of the mobile shooting control apparatus 20 using the shooting device 32 determines whether or not a shooting mode is designated. If the shooting mode is designated, the control unit 21 displays a through image in step S2. That is, the control unit 21 causes the image pickup unit 23 to pick up an image of the subject and gives the picked-up image to the display unit 26 as a through image for display. Next, the control part 21 determines the presence or absence of the imaging operation by the user (step S3), and when the imaging operation is not performed, the process returns to step S1.

  When the photographing operation by the user is performed, the control unit 21 proceeds to step S4 and determines each subject or the like in the captured image by image recognition. For example, the control unit 21 determines a specific person or a pet in the captured image. When a recognizable determination result is obtained from the captured image, the control unit 21 gives the recorded image to the recording unit 27 together with information on the determination result, for example, person information (step S5). If the recognizable determination result is not obtained, the control unit 21 gives only the captured image to the recording unit 27 for recording (step S6). The control unit 21 may use a preset database, for example, a built-in database, a database obtained by a computer on the cloud, or the like for subject recognition.

  In the next step S7, the control unit 21 generates object specifying information. For example, the control unit 21 performs image analysis of the image recorded in the recording unit 27, performs trend analysis of the analog image, and determines a subject that the user likes to capture as a desired image. The control unit 21 generates target object specifying information for specifying the subject (target object) that is the target of the desired image and stores it in the target object information memory 25.

  For example, when the aspect condition for a specific object is set by the operation of the operation input unit 24 by the user, the control unit 21 also stores the aspect condition in the object information memory 25. Thus, for example, the object specifying information shown in FIG. 2 is obtained. Note that each of the records R1 to R3 of the object specifying information in FIG. 2 is obtained by a user's shooting operation, but all or a part of each information of the items i1 to i4 is obtained by image analysis of the control unit 21. It can be obtained. In addition, you may acquire all of each information of items i1-i4 based on user operation.

  If it is determined in step S1 that the shooting mode is not specified, the control unit 21 proceeds to step S11 and determines whether or not the reproduction mode is specified. When the reproduction mode is designated, the control unit 21 selects and reproduces an image based on the user's selection operation from the images recorded in the recording unit 27 in step S12, and displays the image on the display screen of the display unit 26. To display. Even in this reproduction mode, the control unit 21 generates the object specifying information. For example, the control unit 21 performs image analysis of the image being reproduced, performs trend analysis of the analog image, and determines the subject that the user likes and reproduced as the desired image. The control unit 21 generates target object specifying information for specifying the subject (target object) that is the target of the desired image and stores it in the target object information memory 25.

  When the control unit 21 determines in step S11 that the reproduction mode is not designated, the control unit 21 proceeds to step S21 and determines whether or not the object specifying information is stored in the object information memory 25. If the object specifying information is not stored in the object information memory 25, the process returns to step S1. When the object specifying information is stored, in step S22, the control unit 21 tries to communicate by searching for a mobile body that has a communication function and can perform some action on the object. For example, the control unit 21 attempts communication (cooperative request communication) with the mobile imaging device 10 capable of imaging (hereinafter referred to as cooperative imaging) according to the object specifying information from the mobile imaging control device 20. The control unit 21 may try the cooperation request communication every time the object specifying information is stored, or may try the cooperation request communication at a predetermined time interval. In addition, the control unit 21 may perform cooperation request communication in response to access from the mobile imaging device 10 that is the communication partner. Further, the control unit 21 may perform cooperation request communication based on a user operation.

  In step S23, the control unit 21 determines whether communication with the mobile imaging device 10 has been established. When the cooperation request communication is established with the mobile imaging device 10, the control unit 21 proceeds to step S <b> 24 and transmits the object specifying information to the mobile imaging device 10. In addition, the control part 21 returns a process to step S1, when cooperation request | requirement communication cannot be established between cooperation imaging | photography partners in step S23.

  The mobile body that has received the object specifying information executes a predetermined action by detecting the object specified by the object specifying information. For example, the mobile imaging device 10 images a predetermined imaging range by the imaging unit 13a, and performs imaging according to the aspect condition indicated by the object specifying information when the object exists in the imaging range. Further, in the present embodiment, the mobile imaging control device 20 can instruct the subject to wait for the imaging even when the mobile imaging device 10 is not imaging the object.

  In step S25, the control unit 21 determines whether or not such an ambush mode is designated. If the ambush mode is not designated, the process proceeds to step S28, and when the captured image of the object is given as a linked shooting result from the mobile imaging device 10, the captured image is recorded in the recording unit 27. On the other hand, when the ambush mode is designated, the control unit 21 detects the object designated by the object specifying information. For example, the control unit 21 detects the object by imaging the object by the imaging unit 23. When the control unit 21 detects the object, the process proceeds to step S27 and transmits an ambush instruction. When the mobile imaging device 10 receives the ambush instruction, the mobile imaging device 10 moves to the ambush position set for the target object regardless of whether the target object is within the imaging range of the imaging unit 13a. The camera enters a shooting standby state. In addition, the mobile imaging device 10 may be configured to move to an ambush position for an object given the highest priority when the object specifying information includes a plurality of objects.

In the example of FIG. 4, an example is shown in which the control unit 21 transmits an ambush instruction in the ambush mode. However, the contents of the ambush instruction may be included in the object specifying information in advance. In this case, when the mobile imaging device 10 receives the object specifying information including the ambush instruction, the mobile imaging device 10 is set to the object regardless of whether or not the object exists within the imaging range of the imaging unit 13a. It is also possible to move to a waiting position where the subject is in a shooting standby state.
In this case, it is preferable that a shooting spot is recorded in the map information of the mobile imaging device 10 and that it can be determined how far the vehicle can move without an obstacle. Therefore, it is preferable that the mobile photographing device has information on the moving distance of the moving route and the speed at which the moving image can be moved, and can perform control in consideration of the time until it reaches the position. As long as it is an ambush, it is necessary to move faster than the object to be moved and go to that place, so it may be given up in consideration of energy saving before moving. In addition, because of this situation, it is more reasonable to shoot when the baby or pet has entered the room and judge the voice or footsteps with a sensor such as a microphone and immediately ambush Can be.

  The mobile imaging device 10 determines whether or not it is the normal work mode in step S31 of FIG. In the normal work mode, the mobile photographing device 10 performs work involving movement. For example, when the mobile photographing device 10 is the robot cleaner 31, the mobile photographing device 10 cleans while moving on a floor such as a room (step S32). In this case, the map information unit 14 of the mobile imaging device 10 creates a map such as a room and stores the map information in the map information storage unit 14a.

  FIG. 6 shows the inside of the room 41. In the room 41, a side board 42 and a television receiver (TV) 43 are arranged along the wall. A table 44 is arranged in the center of the room 41, and a sofa 45 is arranged in front of the table 44. Note that an imaging device 32 is placed on the side board 42, and the imaging device 32 can image a part of the imaging range of the room 41. In addition, a robot cleaner 31 that is the mobile photographing device 10 exists on the floor surface of the room 41.

  As described above, an imaging position where a position having a step can be imaged is suitable for imaging the baby standing. When creating the map information, the robot cleaner 31 registers the position information of such shooting spots in the map information storage unit 14a (step S33).

  If the control unit 11 of the robot cleaner 31 determines in step S31 that the normal work mode is not designated, it enters a standby state for the cooperation request communication in step S41. Next, the control part 11 determines whether cooperation request | requirement communication generate | occur | produced (step S42). The control part 11 returns a process to step S31, when cooperation request | requirement communication has not generate | occur | produced.

  If it is determined in step S42 that the cooperation request communication has occurred, the control unit 11 responds to the cooperation request communication in step S43, receives the object specifying information from the mobile imaging control device 20, and receives the object information memory. 15 (step S44), the process proceeds to step S45.

  In the present embodiment, the control unit 11 of the robot cleaner 31 determines whether or not the cooperation request communication from the mobile shooting control device 20 has occurred even during the work (step S34). When it is determined in step S34 that the cooperation request communication has occurred, the control unit 11 interrupts the work, shifts the process to step S43, and makes a response to the cooperation request communication. When the cooperation request communication has not occurred, the control unit 11 shifts the process from step S34 to step S35, and determines whether the work is finished. If the work has not been completed, the control unit 11 returns the process to step S31. If the work has been completed, the control unit 11 determines whether or not the object specifying information is stored in the object information memory 15 in step S36. When the object specifying information is not stored in the object information memory 15, the control unit 11 returns the process to step S31, and when it is stored, the process proceeds to step S45.

  When the control unit 11 receives the object specifying information in step S44 or determines that the object specifying information is stored in step S36 at the end of the operation, the control unit 11 indicates the object indicated by the object specifying information. Is detected (step S45).

  In this step S45, the imaging movement control unit 11b in the control unit 11 detects the object by, for example, imaging the object by the imaging unit 13a. Note that, when detecting the object, the imaging movement control unit 11b causes the object to exist within the imaging range when the object specified by the object identification information does not exist within the imaging range. You may perform control which moves the mobile imaging device 10 according to a predetermined rule, or changes an imaging direction. For example, if the target object cannot be detected even after a predetermined time has elapsed, or if the target object cannot be detected even after performing imaging while moving along a predetermined route, the control unit 11 performs processing in step S31. Return to.

When the control unit 11 detects the object, the control unit 11 determines the imaging method according to the moving state of the object, the map state, the aspect condition, and the ambush instruction (step S47). Here, the steps are simplified, but in reality, there are cases where ambush shooting is good, but various situations are judged and given up. That is, there may be a case where the object to be photographed is fast or a route cannot be searched due to an obstacle. Even when the route along which the device moves is only the same route as the object, it is not preferable because it may collide if it is not determined that there is an obstacle. In other words, as one of the important invention points of the present application, for example, if the robot cleaner is a mobile photographing device, it is easy to photograph the specified object by effectively using the position information possessed by the robot cleaner. It can be said that it is in the point of judging where it will be. For this purpose, it is necessary to determine and determine the route based on the positional relationship between the position of the device and the shooting spot, and determine the distance therefrom. There is a contrivance such as searching for routes other than the route that the moving object selects (simply search for multiple routes and delete the object moving route from there), and from the speed of your aircraft The procedure includes calculating the required time, determining the speed of the object from a plurality of image changes taken and determining the route. Thus, the time at which the target object arrives at the position is predicted, and compared with the time required for the subject machine, the determination is made until this determination is possible. If necessary, there is a flow to give up from the viewpoint of risk avoidance and energy saving, but it is omitted here.
In FIG. 5, parallel running, tracking, and ambush are illustrated as examples of the photographing method. For example, when the control unit 11 detects that the target object is moving across the imaging range of the imaging unit 13a, the control unit 11 shifts the processing to step S54 so as to maintain the target object in the imaging range. Then, the moving unit 12 and the tracking control unit 17 are controlled to run in parallel with the object, and imaging of the object is started. When performing the imaging (request imaging) specified by the object specifying information, the control unit 11 transmits a captured image as a result of the cooperative imaging to the mobile imaging control device 20 via the communication unit 16. The control unit 21 of the mobile imaging control device 20 gives the captured image transmitted in step S28 of FIG. 4 to the recording unit 27 for recording.

  Now, for example, it is assumed that the ambush mode is designated in the mobile photographing control device 20. Further, it is assumed that the ambush object is the baby (user family 2) shown in the object specifying information in FIG. 2 and the baby 33 in FIG. In this case, grabbing is specified as an aspect condition. The control unit 21 of the mobile imaging control device 20 detects the baby 33 in step S26 of FIG. As shown in FIG. 7, it is assumed that the baby 33 enters the imaging range of the imaging device 32 in the room 41. If it does so, the control part 21 will determine with the baby 33 which is a target object having been detected by the image analysis of the captured image from the imaging part 23. FIG. As a result, the control unit 21 transmits an ambush instruction to the mobile imaging device 10. Here, the speed is determined from the repeatedly captured images, and the route is predicted from the moving direction.

  When receiving the ambush instruction, the control unit 11 of the mobile imaging device 10 moves the process from step S47 to step S57, and specifies a specific position suitable for picking up the image specified by the object specifying information, that is, step S33. Moves to a position having a level difference registered in. In FIG. 7, this specific position is a position at which the step portion of the sofa 45 can be imaged by the imaging unit 13a. When the control unit 11 performs an image capturing operation (requested image capturing) specified by the object specifying information (step S58), the captured image as a result of the cooperative image capturing is transmitted to the mobile image capturing control device 20 via the communication unit 16. (Step S59). The control unit 21 of the mobile imaging control device 20 gives the captured image transmitted in step S28 of FIG. 4 to the recording unit 27 for recording. Next, in step S60, the control unit 11 returns the mobile imaging device 10 to the original position, and returns the process to step S31.

  Although not shown in FIG. 5, if it is not possible to capture the baby 33 while standing for a predetermined time after moving to a specific position, the process proceeds from step S58 to step S60. Alternatively, the baby 33 may be imaged at a pose other than standing and the imaging result may be transmitted to the mobile imaging control device 20.

  Further, in step S47, when the parallel running or the ambush is not selected as the photographing method, the control unit 11 selects tracking and performs the process of step S51. That is, when the imaging movement control unit 11b detects an object within the imaging range of the imaging unit 13a, the imaging movement control unit 11b tracks the object and starts imaging (step S51). When the control unit 11 performs imaging (request shooting) specified by the object specifying information (step S52), the control unit 11 transmits a captured image as a result of the cooperative shooting to the mobile shooting control device 20 via the communication unit 16 (step S52). S53). The control unit 21 of the mobile imaging control device 20 gives the captured image transmitted in step S28 of FIG. 4 to the recording unit 27 for recording.

  As described above, in the present embodiment, the mobile shooting control device generates object specifying information for setting a subject, a pose, a composition, and the like that the user desires to capture based on a captured image acquired by a user's shooting operation. To do. When there is a mobile photographing device capable of cooperative photographing through cooperative request communication, the mobile photographing control device transmits object specifying information to the mobile photographing device. The mobile imaging device can take an image while moving, and can take an image of a desired subject in a desired pose or composition at a position desired by the user. Thereby, even if the user's confidence is not at the shooting location, or at a location where it is difficult for the user to take an image, it is possible to perform the imaging desired by the user.

  In the above-described embodiment, an example has been described in which the mobile image capturing control device generates object specifying information for specifying the image capturing position, the object, the pose, the composition, and the like of the mobile image capturing device. However, the object specifying information may be generated based on a captured image by a user operation, and the object specifying information may be generated in any circuit. For example, the object specifying information may be generated by one of the control unit of the mobile shooting control apparatus and the control unit of the moving shooting apparatus, and these control units cooperate to generate the object specifying information. May be. Furthermore, you may produce | generate object specific information using not only one or both of these control parts but the computer on a cloud.

For example, when the object specifying information is generated by the mobile imaging device, the mobile imaging control device may transmit the captured image acquired by the user operation to the mobile imaging device. In this case, the mobile imaging device may recognize the object by image analysis on the received captured image and generate the object specifying information according to the appearance frequency of the recognition result. In addition, for example, when only the above-mentioned baby holding position is set as the aspect condition in the object specifying information generated in the mobile shooting control device, the specific position suitable for imaging the baby holding position is described below. The determination may be made by the photographing movement control unit of the mobile photographing device.
At this time, the place where the baby will grab and stand is presumed to be entered in advance or predicted, and further consideration should be taken by the imaging device first to the place. Is it possible to be? Therefore, it is necessary to predict the baby's speed and route, but the route can be determined from the map and the current position, and the speed is the amount of baby movement in each shooting frame and the average baby movement speed. It can be determined from. As a result, the time of arrival at the place where the person catches up is known. The present application relates to an imaging unit that can move and image, a sensor that detects movement of a specified object, map information, and a time at which the object moves to a specific location according to the movement detection and the map information. It can be expressed as a mobile photographing device or a mobile photographing device characterized by comprising a predicting unit for predicting. In addition, it is better for the device to move along a route different from the route, the route can also be determined by the obstacle information in the map information, and the moving speed of the device is known because there are design numerical values etc. The mobile imaging device includes: an object route prediction unit that predicts a movement route of the object; and an imaging unit that determines a movement route of the imaging unit at a location where the object at the specific location can be imaged. It can be said that the apparatus includes a route determination unit, an arrival time determination unit at which the imaging unit arrives based on the imaging unit route determination result and the moving speed of the imaging unit.

  Furthermore, in the above-described embodiment, the mobile imaging control apparatus has been described as an example having an imaging function. However, the present embodiment determines the object or scene that the user wants to take based on the captured image acquired by the user operation, that is, the captured image reflecting the user's preference, and moves and captures the mobile imaging device. It is characterized by control. Therefore, as long as it is possible to read a captured image acquired by such a user operation from a recording medium, an apparatus that does not have an imaging function can function as a mobile imaging control apparatus. For example, a personal computer may be adopted as the mobile shooting control device. Further, the mobile shooting control device is not limited to a shooting device such as a digital camera, but may be a smartphone, a tablet PC, or the like. A smartphone often has not only a shooting function but also a communication function such as wifi, and generates object specifying information based on a captured image acquired by a user operation, and the generated object specifying information is transferred to a moving shooting apparatus. Can be sent to.

  Further, the object specifying information can be obtained not only when an image is acquired by capturing a subject but also when an image is reproduced. That is, the object specifying information may be generated based on not only an image that the user likes to capture but also an image that the user likes to reproduce. Therefore, as the mobile photographing control device, not only a device having an imaging function but also a device having a reproduction function may be adopted. For example, the object specifying information may be acquired by determining an image desired by the user by trend analysis of an image acquired by accessing a website by a computer.

  In the above-described embodiment, a robot cleaner has been described as an example of the mobile photographing device. The robot cleaner acquires room map information for cleaning the room. Therefore, when an instruction is given to the robot cleaner, it is possible to take a picture using a map of the room.

  Although FIG. 1 shows an example in which the mobile imaging device and the mobile imaging control device are configured separately, they may be configured integrally. For example, as a moving body, there is an apparatus that can perform imaging by a user's remote operation, such as a robot cleaner that can instruct photographing with a smartphone. In this case, the moving body records the image acquired by the shooting operation as the cut-in subject information in the recording unit in the moving body, generates the object specifying information based on the recorded captured image, and the object. The movement and imaging may be controlled based on the specific information.

  Moreover, in the said embodiment, although the example which image-captures was demonstrated as an action with respect to the target designated by target object specific information, not only imaging but various actions may be performed. For example, a specific person is set as an object, the object specifying information generated based on the photograph of this person is given to a moving body such as a drone, the specific person is searched, and the position information is tracked. You may control to make it transmit. Alternatively, the object specifying information generated based on the photograph of a specific tree may be given to the drone to fly in an appropriate space, and when the tree is detected, an action such as spraying agricultural chemicals may be executed.

In the above embodiment, the digital camera is used as the imaging device. However, the camera may be a digital single lens reflex camera or a compact digital camera, and a video camera such as a video camera or movie camera. Of course, a camera built in a portable information terminal (PDA: Personal Digital Assist) such as a mobile phone or a smartphone may be used. In addition, the moving body is not limited to a robot cleaner, and can be applied to all movable devices.
The term “movement” cannot be simply associated, but a machine such as a scanner in which the machine itself does not move but only the imaging unit moves is within the assumed range. For example, in applications such as observing microorganisms, the microscope and stage may move. However, depending on the application, such applications as scientific observations, capsule endoscopes, CT scanners, etc. may be wider. There is also an imaging device that captures a range or the whole, and the technique of the present application can be effectively used when determining whether tracking is possible based on a difference between a scan movement and an object.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  It should be noted that even if the operation flow in the claims, the description, and the drawings is described using “first,” “next,” etc. for convenience, it is essential to carry out in this order. It doesn't mean. In addition, it goes without saying that the steps constituting these operation flows can be omitted as appropriate for portions that do not affect the essence of the invention.

  Of the techniques described here, the control mainly described in the flowchart is often settable by a program and may be stored in a recording medium or a recording unit. The recording method for the recording medium and the recording unit may be recorded at the time of product shipment, may be a distributed recording medium, or may be downloaded via the Internet.

    DESCRIPTION OF SYMBOLS 10 ... Mobile imaging device, 11, 21 ... Control part, 11a ... Shooting control part, 11b ... Moving control part for imaging | photography, 12 ... Moving part, 13 ... Information acquisition part, 13a, 23 ... Imaging part, 14 ... Map information part , 15, 25 ... object information memory, 16, 22 ... communication unit, 20 ... mobile imaging control device, 24 ... operation input unit, 27 ... recording unit.

Claims (8)

A memory for specifying imaging target and recording imaging target specifying information for specifying a specific operation state of the imaging target;
Instructs a moving imaging apparatus that controls to move the imaging range of the imaging unit so that the imaging target in the specific operation state is imaged by the imaging unit. A communication unit for transmitting the imaging target specifying information for
An information terminal device comprising: The information terminal device according to claim 1, further comprising an imaging target specifying information generation unit that generates the imaging target specifying information based on image information that the user preferably captures or reproduces. The information terminal device according to claim 1, wherein the communication unit receives an imaging result of the imaging unit of the mobile imaging device. Recording imaging target specifying information for specifying an imaging target and specifying a specific operation state of the imaging target;
Instructs a moving imaging apparatus that controls to move the imaging range of the imaging unit so that the imaging target in the specific operation state is imaged by the imaging unit. A communication step for transmitting the imaging target specifying information for
A photographing method characterized by comprising: On the computer,
Recording imaging target specifying information for specifying an imaging target and specifying a specific operation state of the imaging target;
Instructs a moving imaging apparatus that controls to move the imaging range of the imaging unit so that the imaging target in the specific operation state is imaged by the imaging unit. A communication step for transmitting the imaging target specifying information for
Shooting program to execute. In an imaging device in which the imaging range moves by moving the stage to which the imaging unit is attached,
Given imaging target specifying information for specifying an imaging target, the imaging target specified by the imaging target specifying information is detected by image analysis on a captured image being acquired by the imaging unit by moving the stage. A control unit that controls the movement of the stage in order to cause the imaging unit to capture the detected imaging target;
An imaging device comprising: In an imaging device that has an imaging unit and the imaging range moves by moving the stage,
Given imaging target specifying information for specifying an imaging target, the imaging target specified by the imaging target specifying information is detected by image analysis on a captured image being acquired by the imaging unit by moving the stage. A control unit that controls the movement of the stage in order to cause the imaging unit to capture the detected imaging target;
An imaging device comprising: 8. An information terminal device comprising a communication unit that transmits the imaging target specifying information to the imaging device according to claim 6 or 7.

Images