JP2015026894A - Imaging program, imaging method and information processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging program and the like which can simply image a target image.SOLUTION: An imaging program causes a computer to execute: processing of acquiring images of a subject which are serially imaged and acquiring distance information between the subject at the time of imaging the images and a camera which images the images; processing of extracting from the images, a constituent element in which a portion of the subject and a position of the portion on the images are related with each other; processing of referring to a storage part which stores learning information in which the constituent element of relating a portion of a subject with a position of the portion on a target image at the time of imaging the target image including the portion of the subject, and the distance information between the subject and the camera are related with each other; and processing of outputting an image among the acquired images, which has the constituent element coinciding with that in the learning information and distance information having a difference with that in the learning information being within an allowable range.

Description

  The present invention relates to an imaging program, an imaging method, and an information processing apparatus.

  Some cameras have a support function for displaying on a finder or the like a target part included in the subject, for example, an aim for matching the face, so that the user of the camera can photograph the subject with the intended composition.

JP 2007-174548 A JP 2005-269562 A

  However, the above technique cannot easily capture a target image. For example, when the user takes a picture of his / her head or back, the user cannot view while photographing even if the aim is displayed on the finder or the like. For this reason, the user shoots without confirming how the subject appears on the camera. Therefore, even if the above-described shooting support function is used, the user will have trouble shooting repeatedly until the subject can be shot accidentally with the composition intended by the user.

  In one aspect, an object is to provide an imaging program, an imaging method, and an information processing apparatus that can easily capture a target image.

  In one embodiment, the imaging program acquires, in a computer, an image in which a subject is continuously captured, and acquires distance information between the subject and the camera that has captured the image at the time of capturing the image. Is executed. The imaging program causes the computer to execute a process of extracting a component in which the part of the subject and the position of the part on the image are associated from the image. In the imaging program, a component in which the target subject part and the position of the part on the target image are associated with each other when the target image including the target subject part is captured corresponds to the computer. A process of referring to the storage unit for storing the attached learning information is executed. The imaging program stores learning information in which distance information between the target subject and the camera that captured the target image is associated with the computer at the time of capturing the target image including the target subject portion. A process of referring to the storage unit to be stored is executed. The imaging program causes the computer to execute a process of outputting an image whose constituent elements match the learning information in the acquired image and whose difference in distance information is within an allowable range.

  A desired image can be easily captured.

FIG. 1 is a block diagram illustrating an example of the configuration of the mobile terminal device according to the embodiment. FIG. 2 is an explanatory diagram illustrating an example of the learning information DB. FIG. 3 is an explanatory diagram showing an example of the progress information DB. FIG. 4 is an explanatory diagram illustrating an example of an image of the progress information DB. FIG. 5 is an explanatory diagram illustrating an example of a part. FIG. 6 is an explanatory diagram illustrating an outline of an example of the learning process. FIG. 7 is a flowchart illustrating an example of learning processing of the mobile terminal device according to the embodiment. FIG. 8 is an explanatory diagram illustrating an outline of an example of the similar image capturing process. FIG. 9 is a flowchart illustrating an example of the similar image capturing process of the mobile terminal device according to the embodiment. FIG. 10 is a flowchart illustrating an example of a learning information update process. FIG. 11 is an explanatory diagram illustrating an example of a change in learning information due to accumulation of progress information. FIG. 12 is an explanatory diagram illustrating an example of a computer that executes an imaging program.

  Hereinafter, embodiments of an imaging program, an imaging method, and an information processing apparatus disclosed in the present application will be described in detail based on the drawings. The disclosed technology is not limited by the present embodiment. Further, the following embodiments may be appropriately combined within a consistent range.

  FIG. 1 is a block diagram illustrating an example of the configuration of the mobile terminal device according to the embodiment. A mobile terminal device 10 shown in FIG. 1 includes a camera 11, a display unit 12, an operation unit 13, a proximity sensor 14, a gyro sensor 15, a speaker 16, a motor 17, a communication unit 18, and a storage unit 19. And a control unit 20. As the mobile terminal device 10, for example, a smartphone, a mobile phone, a PHS (Personal Handy phone System), a PDA (Personal Digital Assistant, or Personal Data Assistance) can be used.

  The camera 11 is a module that captures an image. The camera 11 has an image sensor. As the imaging device, a complementary metal oxide semiconductor (CMOS) image sensor, a charge coupled device (CCD) image sensor, or the like can be used. The camera 11 photoelectrically converts light received by the image sensor and performs A / D (Analog / Digital) conversion to generate an image. The camera 11 outputs the captured image to the control unit 20.

  The display unit 12 is a display device for displaying various information. For example, the display unit 12 is realized by a liquid crystal display or the like. The operation unit 13 is an input device that receives various operations from the user. For example, the operation unit 13 is realized by a cursor key, a numeric keypad, or the like. In addition, when the touch panel is employ | adopted for the portable terminal device 10, the display part 12 and the operation part 13 are integrated. In addition to the touch panel, the mobile terminal device 10 employing the touch panel may have hardware keys such as a power button, a volume button, and a home button.

  The proximity sensor 14 measures the distance between the mobile terminal device 10 and the subject. The proximity sensor 14 is, for example, the intensity of ultrasonic waves or infrared rays reflected by irradiating the subject with ultrasonic waves or infrared rays, or the time from when the subject is irradiated with ultrasonic waves or infrared rays until it is reflected and returned. Measure the distance by measuring. When ultrasonic waves or infrared rays are used for the proximity sensor 14, since the ultrasonic waves and infrared rays are directional, the proximity sensor 14 measures the distance between the camera 11 and a subject near the center of the image to be captured. The proximity sensor 14 may be of any other method as long as a distance of about 1 mm to 500 mm can be measured. When measuring the intensity of ultrasonic waves or infrared rays, the proximity sensor 14 outputs a voltage within a predetermined range to the control unit 20 as measurement information, for example. Moreover, when measuring the reflection time, the proximity sensor 14 outputs the reflection time to the control unit 20 as measurement information, for example.

  The gyro sensor 15 detects the angular velocity or acceleration of the mobile terminal device 10. As the gyro sensor 15, a so-called MEMS (Micro Electro Mechanical Systems) gyro can be used, and by using the triaxial gyro, the angular velocities of the X, Y, and Z axes can be detected. The gyro sensor 15 outputs the detected angular velocity of each axis to the control unit 20 as angular velocity information. Here, the mobile terminal device 10 can set the X and Y axes to 0 °, for example, with reference to the case where the mobile terminal device 10 is placed on a horizontally installed desk. For the Z axis, for example, a magnetic north or true north can be set to 0 ° using a magnetic sensor (not shown).

  Further, the gyro sensor 15 may detect the inclination of the mobile terminal device 10 by measuring the gravitational acceleration using a three-axis acceleration sensor, and output the acceleration information instead of the angular velocity information. Note that the gyro sensor 15 can increase the accuracy of the angle and the like by combining the MEMS gyro and the acceleration sensor.

  The speaker 16 outputs a guidance voice for guiding the position of the mobile terminal device 10 to the user, a shooting completion notification, and the like. When an audio signal that is an electrical signal is input from the control unit 20, the speaker 16 outputs a sound wave corresponding to the audio signal to the space.

  The motor 17 is a small electric motor, and by attaching a half-moon-shaped weight to the shaft, the casing of the mobile terminal device 10 to which the motor 17 is fixed is vibrated by rotation. When an instruction to vibrate is input from the control unit 20, the motor 17 rotates to vibrate the casing of the mobile terminal device 10.

  The communication unit 18 is realized by a wireless communication module that supports, for example, a mobile phone line, a wireless local area network (LAN), Bluetooth (registered trademark), and near field communication (NFC). The communication unit 18 is a communication interface that is connected to the Internet via, for example, a wireless base station and manages communication of various types of information.

  The storage unit 19 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 19 includes a learning information database (hereinafter referred to as learning information DB) 21 and a progress information database (hereinafter referred to as progress information DB) 22. Further, the storage unit 19 stores information used for processing in the control unit 20.

  The learning information DB 21 stores learning information including distance information, angle information, composition information, and the like of an image intended by the user. The learning information DB 21 stores learning information by dividing records for different compositions. Here, FIG. 2 is an explanatory diagram illustrating an example of the learning information DB. As shown in FIG. 2, the learning information DB 21 includes distance information 101, distance information + side tolerance 102, distance information −side tolerance 103, angle information 104, angle information + side tolerance 105, and angle information Items such as the -side allowable error 106 are managed in association with each other. In addition, the learning information DB 21 manages items such as composition information 107, components 108, parts 109, positions 110, first shooting date / time 111, information update date / time 112, and images 113 in association with each other.

  The distance information 101 indicates the shooting distance at which the target image is shot. The distance information 101 indicates an average value of the shooting distance of each image when learning progresses and information of a plurality of images is reflected. The + side allowable error 102 of the distance information indicates the + side allowable error of the shooting distance at which the target image is taken. The + side allowable error 102 of the distance information is updated with the new allowable error when a new allowable error is calculated based on the difference between the previous distance information and the distance information of the newly captured image. The -side allowable error 103 of the distance information indicates an allowable error on the-side of the shooting distance at which the target image is shot. The -side allowable error 103 of the distance information is updated in the same procedure as the + side allowable error 102 of the distance information.

  The angle information 104 indicates an angle of the mobile terminal device 10 when a target image is captured. The angle information 104 indicates an average value of angles corresponding to each image when learning progresses and information of a plurality of images is reflected. Here, the angle information 104 can be orientation information representing the orientation of the mobile terminal device 10, for example. The angle information 104 can represent, for example, only the angle between the display unit 12 of the mobile terminal device 10 and the horizontal plane, that is, how much the mobile terminal device 10 is tilted from the angle of each axis as the orientation information. Further, the angle information 104 may represent only a rough direction such as downward, diagonally downward, upward, and diagonally upward as the orientation information.

  The + side allowable error 105 of the angle information indicates an allowable error on the + side of the angle of the mobile terminal device 10 when the target image is captured. The + side allowable error 105 of the angle information is updated with the new allowable error when a new allowable error is calculated based on the difference between the previous angle information and the angle information of the newly captured image. The -side allowable error 106 of the angle information indicates an allowable error on the negative side of the angle of the mobile terminal device 10 when the target image is captured. The -side allowable error 106 of the angle information is updated in the same procedure as the + side allowable error 105 of the angle information. It should be noted that the allowable error of the distance information and the angle information is updated to a new allowable error when the value is within a preset maximum allowable error.

  The composition information 107 further includes one or more components 108. In the example of FIG. 2, the composition information 107 includes three constituent elements 108, but is not limited thereto. The component 108 has a portion 109 and a position 110. A part 109 indicates a target part of the subject. The part 109 is information such as “head of head”, “ear”, “nose”, and the like. It should be noted that the feature part to be noted such as the affected part may include additional information or flag information such as “the top of the head (affected part)”. A position 110 indicates the position of the part 109 on the target image. The position 110 is, for example, information such as “center”, “lower right”, and “lower left”. The first shooting date and time 111 indicates the date and time when an image having a certain composition is first stored in the learning information DB 21 as a target image. The information update date 112 indicates the date when each item is updated with a newly taken image, that is, when learning is performed. The image 113 shows a target image file.

  Returning to the description of FIG. 1, the progress information DB 22 stores progress information including the shooting date and time, distance information, angle information, composition information, and the like of a similar image that is continuously captured and is similar to the target image. To do. Here, FIG. 3 is an explanatory diagram showing an example of the progress information DB. As shown in FIG. 3, the progress information DB 22 manages items such as the shooting date / time 121, the distance information 122, the angle information 123, the composition information 124, the component 125, the part 126, the position 127, and the image 128 in association with each other. Yes.

  The shooting date / time 121 indicates the date / time when a similar image was shot. The distance information 122 indicates the distance between the mobile terminal device 10 and the subject when a similar image is captured. The angle information 123 indicates the angle of the mobile terminal device 10 when a similar image is captured. The composition information 124 further includes one or more components 125. In the example of FIG. 3, the composition information 124 includes three components 125, but is not limited thereto. The component 125 has a portion 126 and a position 127. A part 126 indicates a part of the target subject on the similar image. The part 126 is information such as “head of head”, “ear”, “nose”, and the like. It should be noted that the feature part to be noted such as the affected part may include additional information or flag information such as “the top of the head (affected part)”. A position 127 indicates the position of the part 126 on the similar image. The position 127 is information such as “center”, “lower right”, and “lower left”. The image 128 shows a similar image file.

  Here, an example of a specific example of the image 128 is shown in FIG. FIG. 4 is an explanatory diagram illustrating an example of an image of the progress information DB. The images 31 to 34 are, for example, images of the affected part of the top of the head, and are images in which time has passed in order from the image 31 to the image 34. For example, the images 31 to 34 are preferably images taken with the same composition as much as possible in order to continuously observe the affected area on the top of the head. That is, the progress information DB 22 is a database for recording changes over time in a predetermined site such as an affected part.

  Returning to the description of FIG. 1, the control unit 20 is realized by a program stored in an internal storage device being executed using the RAM as a work area, for example, by a CPU, an MPU (Micro Processing Unit), or the like. . The control unit 20 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 20 includes an acquisition unit 23, a learning unit 24, an extraction unit 25, a determination unit 26, and an output unit 27, and implements or executes information processing functions and operations described below. Note that the internal configuration of the control unit 20 is not limited to the configuration illustrated in FIG. 1, and may be another configuration as long as information processing described later is performed.

  The acquisition unit 23 receives an image from the camera 11 and acquires an image captured by the camera 11. The acquisition unit 23 receives measurement information from the proximity sensor 14 and receives one or more of angular velocity information and acceleration information from the gyro sensor 15. The acquisition unit 23 calculates the distance between the mobile terminal device 10 and the subject based on the voltage or the reflection time of the measurement information, and sets it as the distance information of the image. Moreover, the acquisition part 23 calculates the angle of the portable terminal device 10 at the time of image acquisition based on one or more of angular velocity information and acceleration information, and makes it into angle information. Here, the acquisition unit 23 calculates the angle by integrating the angle of the angular velocity information. In addition, the acquisition unit 23 calculates the angle of each axis of the mobile terminal device 10 based on the gravitational acceleration of each axis of the acceleration information.

  Here, as the orientation information indicating the orientation of the mobile terminal device 10, the acquisition unit 23, for example, the angle between the display unit 12 of the mobile terminal device 10 and the horizontal plane from the angle of each axis, that is, how much the mobile terminal device 10 is. You may calculate the angle showing only whether it inclines. Furthermore, the acquisition unit 23 may represent only rough directions such as downward, diagonally downward, upward, and diagonally upward as the orientation information. The acquisition unit 23 can calculate the movement amount (movement distance) of the mobile terminal device 10 by integrating the acceleration.

  The acquisition unit 23 reads learning information from the learning information DB 21 and displays an image example of a target image on the display unit 12 when performing a similar image capturing process according to a user instruction. The acquisition unit 23 activates the camera 11, the proximity sensor 14, and the gyro sensor 15 when the user selects an example of an image to be photographed, and starts acquiring images, measurement information, angular velocity information, or acceleration information. The acquisition unit 23 calculates distance information of the image based on the measurement information, and calculates angle information or orientation information of the mobile terminal device 10 at the time of image acquisition based on the angular velocity information or the acceleration information. The acquisition unit 23 outputs the image to the learning unit 24 and the extraction unit 25. Further, the acquisition unit 23 outputs the distance information of the image and the angle information or orientation information of the mobile terminal device 10 at the time of image acquisition to the learning unit 24 and the determination unit 26. Further, the acquisition unit 23 outputs information on the image example selected by the user to the determination unit 26.

  The acquisition unit 23 may output an image captured by the camera 11 to the storage unit 19 as a moving image or a still image. The acquisition unit 23 outputs, for example, a moving image having an FHD (Full High Definition video) image quality such as 1080i or an HD (High Definition video) image quality such as 720p as a moving image. For example, when the number of pixels of the image sensor of the camera 11 is 8 million pixels, the acquisition unit 23 outputs a still image having pixels of up to 3264 × 2448 pixels. Note that the still image can have any number of pixels equal to or less than the number of pixels of the image sensor as long as necessary information is captured.

  The learning unit 24 receives an image, image distance information, and angle information or orientation information of the mobile terminal device 10 at the time of image acquisition from the acquisition unit 23. The learning unit 24 extracts a predetermined number of images at a predetermined interval, for example, an interval of 0.5 seconds, from the image distance information and the angle information or orientation information of the mobile terminal device 10 at the time of image acquisition. The plurality of extracted images are output as candidate images to the display unit 12 and displayed. That is, the learning unit 24 presents a continuous image at a predetermined interval to the user as a candidate image.

  The learning unit 24 extracts a part and a position of the part as a constituent element of the subject by image processing for the candidate image selected as the target image by the user. For example, the learning unit 24 divides an image showing a person's head into a hair portion and other regions, and extracts an affected part, eyebrows, ears, nose, and a toothpick. When each part is extracted, the learning unit 24 generates composition information including one or more components in which each part is associated with the position of each part. Here, the part is illustrated in FIG. FIG. 5 is an explanatory diagram illustrating an example of a part. The learning unit 24 divides the image 35 into the hair unit 36 and other regions. The learning unit 24 extracts the toothpick 37 from the hair unit 36 as a part. Further, the learning unit 24 extracts the ear 38, the nose 39, and the eyebrow 40 as parts from other regions.

  The learning unit 24 associates the distance information of the image, the angle information or orientation information of the mobile terminal device 10 at the time of image acquisition, and the composition information with respect to the image for which the composition information has been generated, and stores it in the progress information DB 22 as progress information. Store. Further, the learning unit 24 adds to the progress information of the image an allowable error that is a preset allowable range with respect to the distance information of the image and the angle information or the orientation information of the mobile terminal device 10 at the time of image acquisition. Learning information is generated and stored in the learning information DB 21. That is, the learning unit 24 is a generation unit that generates learning information.

  Further, when update information is input from the determination unit 26, the learning unit 24 reads the progress information to be updated with reference to the progress information DB 22, and calculates the difference between the distance information and the angle information with the corresponding learning information. To do. The learning unit 24 calculates a new allowable error by adding the calculated difference to the allowable error of the distance information and the angle information of the learning information. If the new allowable error is within the maximum allowable error, the learning unit 24 reflects the new allowable error in the learning information. If the new allowable error is outside the maximum allowable error, the learning unit 24 discards the calculated new allowable error without reflecting it. When there is progress information for which the difference between the distance information and the angle information has not been calculated, the learning unit 24 refers to the next progress information and similarly calculates a new allowable error. If there is no progress information for which the difference between the distance information and the angle information has not been calculated, the learning unit 24 ends the learning information update process. Note that the learning unit 24 may calculate an allowable error of the orientation information instead of the angle information.

  Returning to the description of FIG. 1, when an image is input from the acquisition unit 23, the extraction unit 25 performs image processing such as pattern matching on the subject of the image. The extraction unit 25 extracts the part of the subject and the position of the part on the image from the image by image processing, and uses the extracted part and the position of the part as components. The extraction unit 25 collects one or more extracted components and generates composition information. The extraction unit 25 outputs the generated composition information to the determination unit 26.

  When the composition information is input from the extraction unit 25, the determination unit 26 reads the learning information stored in the learning information DB 21. The determination unit 26 determines whether the constituent element of the composition information matches the constituent element of the learning information. First, the determination unit 26 determines whether or not the component parts match each other. The determination unit 26 determines whether there are a plurality of constituent elements, and determines whether or not the positions of the parts match if there is at least one or more matching parts. That is, the determination unit 26 determines whether or not at least one constituent element of the composition information and the learning information matches.

  Further, the distance information of the image and the angle information or the orientation information of the mobile terminal device 10 at the time of image acquisition are input to the determination unit 26 from the acquisition unit 23. The determining unit 26 compares the distance information of the image with the distance information of the learning information when at least one of the constituent elements of the composition information and the learning information matches. If the difference between the distance information is within an allowable range, that is, within the allowable error range on each of the + side and the − side, the determination unit 26 adds distance information, angle information or orientation information, and composition information to the image. Are associated with each other and output to the output unit 27. At this time, in order to notify the user that the image has been output, the determination unit 26 notifies, for example, by voice that “shooting is completed” or vibrates the mobile terminal device 10. It may be.

  If the predetermined number of constituent elements of the composition information and the learning information do not match, or if the difference between the distance information is outside the allowable range, the determination unit 26 notifies the user of the composition information of the image. Or it guides distance information toward learning information. The determination unit 26 guides the direction in which the mobile terminal device 10 is moved by voice in order to guide the user. For example, if the determination unit 26 determines that the target portion of the subject is at the bottom of the image, the determination unit 26 guides the user to move the mobile terminal device 10 downward, for example, “Move it down”. . Further, the determination unit 26 may guide the mobile terminal device 10 by vibrating it in order to guide the user. For example, the determination unit 26 increases the vibration as the target portion of the subject is at the center of the image, and decreases the vibration as the end of the image is out of the image capturing range. By preventing vibration, the user can be guided. For example, when the mobile terminal device 10 is too close to the subject, the determination unit 26 guides, for example, “Please keep the camera away”. That is, the determination unit 26 is also a notification unit that notifies the guidance.

  The determination unit 26 determines whether images corresponding to all the image examples have been captured based on the information on the image examples selected by the user input from the acquisition unit 23. The determination unit 26 guides the user to a shooting position corresponding to the next image example when shooting of images corresponding to all the image examples is not completed. The determination unit 26 outputs update information to the learning unit 24 when imaging of images corresponding to all image examples is completed.

  The output unit 27 receives an image from the determination unit 26, and distance information, angle information or orientation information, and composition information associated with the image. The output unit 27 stores the image, distance information associated with the image, angle information or orientation information, and composition information in the progress information DB 22 as progress information. Further, the output unit 27 outputs and displays the image on the display unit 12 so that the user can confirm the captured image. Further, the output unit 27 may perform an encoding process on the image to compress the file size of the image. The output unit 27 may output the encoded image to the storage unit 19 or may store the encoded image in another server via the communication unit 18 and the Internet.

  Next, the operation of the mobile terminal device 10 according to the embodiment will be described.

  First, an outline of a learning process from a state where there is no learning information will be described with reference to FIG. FIG. 6 is an explanatory diagram illustrating an outline of an example of the learning process. The user holds the mobile terminal device 10 in his / her hand and takes images of the top of the head and the back of the head that cannot be seen by himself / herself, and acquires an image that is a candidate for a target image. At this time, the mobile terminal device 10 may guide whether the entire head is within the imaging range or whether a part of the head is captured. The mobile terminal device 10 displays a frame 41 for displaying candidate images as thumbnails on the display unit 12 and displays each candidate image. The user selects the image 42 by touching the image with a finger or the like. The mobile terminal device 10 generates learning information by associating the selected image 42 with image distance information, angle information or orientation information of the mobile terminal device 10 at the time of image acquisition, composition information, and the like. Store in DB21.

  FIG. 7 is a flowchart illustrating an example of learning processing of the mobile terminal device according to the embodiment. In the mobile terminal device 10, when the learning process is started by an instruction from the user, the acquisition unit 23 activates the camera 11, the proximity sensor 14, and the gyro sensor 15 to acquire images, measurement information, angular velocity information, or acceleration information. Start (step S1). The acquisition unit 23 calculates distance information of the image based on the measurement information, and calculates angle information or orientation information of the mobile terminal device 10 at the time of image acquisition based on the angular velocity information or the acceleration information. Here, the acquisition unit 23 determines whether the distance information of the image has been calculated based on the measurement information, that is, whether the distance information has been acquired (step S2). If the distance information cannot be acquired (No at Step S2), the acquisition unit 23 calculates the distance information of the image again based on the new measurement information.

  When the distance information can be acquired (step S2: affirmative), the acquisition unit 23 determines whether the angle information or the direction information can be calculated based on the angular velocity information or the acceleration information, that is, the angle information or the direction information is acquired. It is determined whether or not it has been completed (step S3). If the angle information or the direction information cannot be acquired (No at Step S3), the acquisition unit 23 returns to Step S2, and if the distance information can be acquired, the angle information is again obtained based on the new angular velocity information or acceleration information. Alternatively, orientation information is calculated. When the angle information or the direction information can be acquired (Step S3: Yes), the acquisition unit 23 outputs the image, the distance information of the image, and the angle information or the direction information to the learning unit 24.

  The learning unit 24 determines whether or not the acquisition unit 23 has extracted a predetermined number of images at predetermined intervals for the image distance information and angle information or orientation information input from the acquisition unit 23. (Step S4). If the acquisition unit 23 cannot extract a predetermined number of images at predetermined intervals (No at Step S4), the learning unit 24 returns to Step S2 to the acquisition unit 23, and the image, image distance information, and Instructing reacquisition of angle information or orientation information. The learning unit 24 outputs and displays the extracted predetermined number of images as candidate images on the display unit 12 when the acquisition unit 23 can extract the predetermined number of images at predetermined intervals (step S4: Yes). Present to the user (step S5).

  The learning unit 24 determines whether a target image has been selected by the user from the candidate images presented on the display unit 12 of the mobile terminal device 10 (step S6). If the target image is not selected by the user (No at Step S6), the learning unit 24 returns to Step S2 and returns the image, the distance information of the image, and the angle information or orientation to the acquisition unit 23. Instructs reacquisition of information. When the target image is selected by the user (step S6: affirmative), the learning unit 24 extracts a part and a position of the part as constituent elements of the subject to form constituent elements, and obtains composition information from the constituent elements. Generate (step S7).

  The learning unit 24 determines whether composition information has been generated (step S8). If the composition information could not be generated (No at Step S8), the learning unit 24 returns to Step S2 and re-generates the image, the distance information of the image, and the angle information or the orientation information. Instruct acquisition. When the composition information can be generated (step S8: affirmative), the learning unit 24 associates the image distance information, the angle information or the orientation information, and the composition information with each other as progress information. The information is stored in the information DB 22 (step S9). Further, the learning unit 24 generates learning information by adding to the progress information of the image an allowable error that is a predetermined allowable range with respect to the distance information and angle information or orientation information of the image, and generates learning information. (Step S10).

  The mobile terminal device 10 can acquire learning information by acquiring the target image for automatic shooting and composition information corresponding to the image by learning processing.

  Next, an outline of processing for capturing an image similar to a target image as progress information will be described with reference to FIG. FIG. 8 is an explanatory diagram illustrating an outline of an example of the similar image capturing process. The user selects an image example 43 to be photographed from the learning information image displayed on the display unit 12 of the mobile terminal device 10. Here, as the image example 43, for example, it is assumed that image examples 43A, 43B, and 43C are selected. Learning information is associated with each of the image examples 43A to 43C. The user holds the mobile terminal device 10 in his / her hand and moves the mobile terminal device 10 toward the top of the head and the back of the head which cannot be seen by himself / herself in order to capture images similar to the image examples 43A to 43C.

  For example, when the target part of the user who is the subject enters the imaging range, the mobile terminal device 10 guides the user so that the target part is in the center of the imaging range. The mobile terminal device 10 automatically performs imaging when the user moves the mobile terminal device 10 according to the guidance and the target site is in the center of the imaging range. For example, the mobile terminal device 10 guides in a direction in which the learning information of the image example 43A and the composition information of the image being captured are the same, and captures a similar image. When the imaging of the image example 43A is completed, the portable terminal device 10 performs guidance and imaging for the image examples 43B and 43C. Note that the mobile terminal device 10 captures images of the image examples 43A to 43C in any order, and captures images from the closest image example. The mobile terminal device 10 generates progress information by associating images similar to the captured image examples 43A to 43C with distance information at the time of imaging, angle information or orientation information of the mobile terminal device 10, composition information, and the like. And stored in the progress information DB 22.

  FIG. 9 is a flowchart illustrating an example of the similar image capturing process of the mobile terminal device according to the embodiment. In the portable terminal device 10, when a similar image capturing process is started in accordance with a user instruction, the acquisition unit 23 reads out learning information from the learning information DB 21 and displays an image example of a target image on the display unit 12 (step 1). S51). When the user selects an example image to be photographed, the acquisition unit 23 outputs information on the selected example image to the determination unit 26 (step S52). In addition, the acquisition unit 23 activates the camera 11, the proximity sensor 14, and the gyro sensor 15, and starts acquiring images, measurement information, angular velocity information, or acceleration information (step S53). The acquisition unit 23 calculates distance information of the image based on the measurement information, and calculates angle information or orientation information of the mobile terminal device 10 at the time of image acquisition based on the angular velocity information or the acceleration information. The acquisition unit 23 outputs the image to the extraction unit 25. The acquisition unit 23 outputs the distance information of the image and the angle information or the orientation information of the mobile terminal device 10 at the time of image acquisition to the determination unit 26.

  When the image is input from the acquisition unit 23, the extraction unit 25 extracts the part of the subject and the position of the part on the image from the image, and uses the extracted part and the position of the part as components. The extraction unit 25 collects one or more extracted components and generates composition information (step S54). The extraction unit 25 outputs the generated composition information to the determination unit 26.

  When the composition information is input from the extraction unit 25, the determination unit 26 reads the learning information stored in the learning information DB 21. The determination unit 26 determines whether or not at least one constituent element of the composition information matches at least one constituent element of the learning information (step S55). The determination unit 26 compares the distance information of the image with the distance information of the learning information when at least one of the respective constituent elements matches (step S55: Yes), and the difference between the distance information is within the allowable range. It is determined whether or not (step S56). That is, the determination unit 26 determines whether the distance between the subject and the mobile terminal device 10 is within the allowable range of the distance information in the learning information. When the predetermined number of components does not match each other (step S55: negative), or when the difference between the distance information is outside the allowable range (step S56: negative), the determination unit 26 determines that the user is in the imaging position. (Step S60), and the process returns to step S54. That is, the determination unit 26 guides the user with the composition information or distance information of the image directed to the learning information. When the difference between the distance information is within the allowable range (step S56: Yes), the determination unit 26 associates the distance information, the angle information or the orientation information, and the composition information with the image, and outputs the output unit 27. Output to. Further, in order to notify the user that the image has been output, the determination unit 26 notifies, for example, “Shooting has been completed” by voice, or notifies the user by vibrating the mobile terminal device 10.

  When the image, distance information, angle information or orientation information, and composition information are input from the determination unit 26, the output unit 27 associates the distance information, angle information, orientation information, and composition information with the image. The progress information is stored in the progress information DB 22 (step S57).

  The determination unit 26 determines whether images corresponding to all the image examples have been captured based on the information of the selected image examples input from the acquisition unit 23 (step S58). If the images corresponding to all the image examples have not been captured (No at Step S58), the determination unit 26 guides the user to the next imaging position (Step S60) and returns to Step S54. That is, the determination unit 26 guides the user by directing the composition information or distance information of the image toward the learning information of the next image example. When the determination unit 26 has captured images corresponding to all the image examples (step S58: Yes), the determination unit 26 outputs update information instructing the learning unit 24 to execute learning information update processing (step S59). . In addition, the determination unit 26 notifies the user that imaging of images corresponding to all image examples has been completed, for example, notifies the user that “all shooting has been completed” or a portable terminal. The device 10 is notified by vibrating.

  FIG. 10 is a flowchart illustrating an example of a learning information update process. When the update information is input from the determination unit 26, the learning unit 24 reads the progress information to be updated with reference to the progress information DB 22, and calculates the difference between the distance information and the angle information with the corresponding learning information. (Step S591). The learning unit 24 calculates a new allowable error by adding the calculated difference to the allowable error of the distance information and the angle information of the learning information (step S592).

  The learning unit 24 determines whether or not the new allowable error is within the maximum allowable error (step S593). If the new allowable error is within the maximum allowable error (step S593: Yes), the learning unit 24 reflects the new allowable error in the learning information (step S594). When the new allowable error is outside the maximum allowable error (No at Step S593), the learning unit 24 discards the calculated new allowable error without reflecting it. The learning unit 24 determines whether there is progress information for which the difference between the distance information and the angle information has not been calculated (step S595). If there is progress information for which the difference between the distance information and the angle information has not been calculated (step S595: affirmative), the learning unit 24 refers to the next progress information (step S596), returns to step S591, and is similarly new. Calculate the tolerance. If there is no progress information for which the difference between the distance information and the angle information has not yet been calculated (step S595: No), the learning unit 24 ends the learning information update process.

  The learning unit 24 of the mobile terminal device 10 accumulates the progress information through the learning information update process, so that the distance between the subject that is the imaging condition and the mobile terminal device 10 and the angle of the mobile terminal device 10 or Regarding the orientation, the allowable error can be expanded within the range of the maximum allowable error. As a result, a target image can be taken more easily.

  The mobile terminal device 10 presents an example image to the user from the learning information DB 21 and an image example to be captured is selected. The mobile terminal device 10 generates composition information from the captured image, and stores the image in the progress information DB 22 as progress information when the composition information matches the distance information and the like. In addition, when the composition information does not match the distance information, the mobile terminal device 10 guides the user in the matching direction. As a result, the user can easily take a desired image by moving the mobile terminal device 10 in accordance with the guidance to be guided.

  Next, the relationship between accumulation of progress information and learning information will be described with reference to FIG. FIG. 11 is an explanatory diagram illustrating an example of a change in learning information due to accumulation of progress information. First, the user executes a learning process of the mobile terminal device 10 and captures an image 45 in which the affected part 44 that is a target part of the temporal region is located at the center. The image 45 is stored in the learning information DB 21 as a target image.

  The mobile terminal device 10 performs imaging with reference to the learning information DB 21 in the second and subsequent imaging. At this time, as shown in images 46, 47, and 48, when the affected part 44 is included in the image, the mobile terminal device 10 accumulates in the progress information as an allowable image. On the other hand, as shown in the images 49 and 50, the mobile terminal device 10 accumulates an unacceptable image, that is, progress information when the affected part 44 is not included in the image at all or only partially included. Not performed.

  The mobile terminal device 10 accumulates the progress information. For example, even if the image is an image 51 that is initially determined not to match the learning information, the mobile terminal device 10 accumulates the progress information, and the tolerance of the learning information is small. By updating and enlarging, it accumulates in the progress information as an acceptable image. In other words, the mobile terminal device 10 allows imaging even when the target image 45 and the direction of the subject are changed. Further, the portable terminal device 10 can accumulate the progress information, and if the affected part 44 in the temporal region is included in the image, even the image 52 captured from the front can be allowed.

  In this way, the mobile terminal device 10 acquires images obtained by continuously capturing the subject, and acquires distance information between the subject and the camera that captured the image at the time of capturing the image. The mobile terminal device 10 extracts a component in which the position of the subject and the position of the part on the image are associated from the image. The mobile terminal device 10 learns by associating a component in which the position of the target object and the position of the part on the target image are associated with each other at the time of capturing the target image including the target object region. A storage unit for storing information is referred to. The portable terminal device 10 stores learning information in which distance information between a target subject and a camera that has captured the target image is associated with the target image including a target image including a target subject region. Refer to the section. The mobile terminal device 10 outputs an image in which the constituent elements match the learning information in the acquired image and the difference in the distance information is within an allowable range. As a result, a target image can be easily taken.

  Also, the mobile terminal device 10 extracts a plurality of components from the image, and outputs an image in which at least one of the components matches with the learning information and the difference in distance information is within an allowable range To do. As a result, even if all the constituent elements of the learning information do not match, if the target part of the constituent elements is included in the image, it can be more easily photographed as being similar to the target image.

  In addition, when the mobile terminal device 10 acquires an image whose constituent elements do not match the learning information or an image whose difference in distance information is outside the allowable range, the mobile terminal device 10 displays the constituent elements or distance information of the images. Notify guidance to guide towards learning information. As a result, when the user moves the mobile terminal device 10 according to the guidance, the target image can be taken more easily.

  In addition, the mobile terminal device 10 uses, as learning information, information further associated with orientation information of a camera that captures the image at the time of capturing the target image. The mobile terminal device 10 acquires camera orientation information at the time of capturing an image, and outputs an image in which the difference between the orientation information and the learning information is within an allowable range. As a result, the orientation of the mobile terminal device 10 is shown to the user by considering not only the component of the image (composition) and the distance between the subject at the time of imaging and the mobile terminal device 10 but also the orientation of the mobile terminal device 10. Can also be guided.

  Further, the mobile terminal device 10 acquires images obtained by continuously capturing the subject as images that are candidates for the target image, and distance information between the subject and the camera that captured the image at the time of capturing the candidate image. And the orientation information of the camera that captures the image at the time of capturing the candidate image is acquired. The mobile terminal device 10 presents candidate images to the user and extracts components from the candidate images selected by the user. The mobile terminal device 10 uses the candidate image from which the component is extracted as a target image, and generates learning information in which the component image and distance information at the time of capturing the target image are associated with the target image. As a result, the mobile terminal device 10 can acquire the target image and corresponding learning information.

  In the above embodiment, the determination unit 26 receives the distance information of the image and the angle information or the orientation information from the acquisition unit 23 together with the composition information input from the extraction unit 25. However, the determination unit 26 is not limited thereto. Not. For example, the determination unit 26 may request angle information or orientation information from the acquisition unit 23 after determining whether the composition information and the distance information match.

  Moreover, in the said Example, although the user's own head, the back, etc. were mentioned as an example as a to-be-visible subject for a user, it is not limited to this. For example, the subject can be applied to any subject that cannot be seen by the user, such as the back side of an apparatus installed adjacent to a wall.

  In the above embodiment, a plurality of image examples of the target image are used. However, the present invention is not limited to this, and one image example may be selected by the user.

  Moreover, in the said Example, although the image similar to the target image was stored in progress information DB22 of the memory | storage part 19, it is not limited to this. For example, the mobile terminal device 10 may continuously take images of the affected area for follow-up observation, and may appropriately transmit the images to the doctor's computer via the communication line via the communication unit 18.

  In the above embodiment, the affected part is cited as the target part of the subject. However, the affected part may not be identified by treatment. For this reason, the mobile terminal device 10 captures an image if the other constituent elements match even if the constituent element including the most prioritized portion of the learning information does not match in the composition information of the image. And get an image. Thereby, for example, the portable terminal device 10 can obtain a target image by acquiring an image if the ear is present in the image, even when the affected part of the temporal region is healed and cannot be determined. .

  Moreover, in the said Example, the acquisition part 23 measured the distance between the portable terminal device 10 and a to-be-photographed based on the measurement information input from the proximity sensor 14, However, It is not limited to this. For example, the acquisition unit 23 inputs the dimension a of the subject, the focal length b of the camera 11, and the camera 11 if the dimension a of the subject is known in advance by the user. The distance can be calculated based on the dimension c of the part on the image. That is, the distance X between the mobile terminal device 10 and the subject can be calculated from a: c = x: b and X = x + b. Thereby, the acquisition part 23 can make distance X into the distance information of an image.

  Moreover, in the said Example, although the portable terminal device 10 was illustrated as information processing apparatus, it is not limited to this. The information processing apparatus may be, for example, a stationary personal computer.

  In addition, each component of each part illustrated does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each unit is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions. Can be configured. For example, in the above embodiment, the composition information is generated in the learning process by the learning unit 24, but may be executed by the extraction unit 25 that generates the composition information in the similar image capturing process.

  Furthermore, various processing functions performed in each device may be executed entirely or arbitrarily on a CPU (or a microcomputer such as an MPU or MCU (Micro Controller Unit)). In addition, various processing functions may be executed in whole or in any part on a program that is analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware based on wired logic. Needless to say, it is good.

  By the way, the various processes described in the above embodiments can be realized by executing a program prepared in advance by a computer. Therefore, in the following, an example of a computer that executes a program having the same function as in the above embodiment will be described. FIG. 12 is an explanatory diagram illustrating an example of a computer that executes an imaging program.

  A computer 200 that executes the imaging program illustrated in FIG. 12 includes an interface unit 211, a RAM 212, a ROM (Read Only Memory) 213, and a processor 214. The interface unit 211 acquires, for example, an image in which a subject is captured and distance information between the subject and the camera at the time of capturing the image via various interfaces such as USB (Universal Serial Bus) or a communication line. The processor 214 controls the entire computer 200.

  The ROM 213 stores in advance an imaging program that exhibits the same function as in the above-described embodiment. Note that the imaging program may be recorded on a recording medium that can be read by a drive (not shown) instead of the ROM 213. The recording medium may be, for example, a portable recording medium such as a CD-ROM, a DVD disk, or a USB memory, or a semiconductor memory such as a flash memory. As the imaging program, as shown in FIG. 12, there are an acquisition program 213A, an extraction program 213B, and an output program 213C. Note that the programs 213A to 213C may be integrated or distributed as appropriate. The RAM 212 stores a database that stores learning information, progress information, and the like.

  Then, the processor 214 reads these programs 213A to 213C from the ROM 213, and executes each of these read programs. As shown in FIG. 12, the processor 214 functions as the acquisition process 214A, the extraction process 214B, and the output process 214C with respect to each program 213A to 213C.

  The processor 214 acquires images in which the subject is continuously captured, and acquires distance information between the subject and the camera that captured the image at the time of capturing the image. The processor 214 extracts the component in which the position of the subject and the position of the part on the image are associated from the image. The processor 214 refers to a storage unit that stores a component that associates the position of the target subject and the position of the target image on the target image when capturing the target image including the target subject part. . The processor 214 has a storage unit that stores learning information in which distance information between the target subject and the camera that captured the target image is associated with the target image including the target subject region. refer. The processor 214 outputs an image in which the constituent elements match the learning information in the acquired image and the difference in the distance information is within an allowable range. As a result, a target image can be easily taken.

DESCRIPTION OF SYMBOLS 10 Portable terminal device 11 Camera 12 Display part 13 Operation part 14 Proximity sensor 15 Gyro sensor 16 Speaker 17 Motor 18 Communication part 19 Storage part 20 Control part 21 Learning information DB
22 Progress information DB
23 acquisition unit 24 learning unit 25 extraction unit 26 determination unit 27 output unit

Claims (7)

On the computer,
Acquire an image of the subject continuously captured,
Obtaining distance information between the subject and the camera that captured the image at the time of capturing the image;
Extracting a component in which the position of the subject and the position of the part on the image are associated from the image,
The component in which the position of the target subject and the position of the part on the target image are associated with each other at the time of capturing the target image including the target subject part, and the target subject and With reference to a storage unit that stores learning information associated with distance information between cameras that have captured the target image, the constituent elements match the learning information in the acquired image, and An imaging program for executing a process of outputting an image in which a difference in distance information is within an allowable range. As a process of extracting the component,
Extracting a plurality of the constituent elements from the image;
As a process of outputting the image,
2. The imaging program according to claim 1, wherein a process of outputting an image in which at least one component matches with the learning information and a difference in distance information is within an allowable range is executed. . In the computer,
When an image whose component does not match with the learning information or an image whose difference in distance information is outside the allowable range is acquired, the component or distance information of the image is guided toward the learning information The imaging program according to claim 1, further comprising executing a process of notifying a guidance to be performed. The learning information is further associated with orientation information of the camera at the time of capturing the target image,
In the computer,
Further executing a process of acquiring orientation information of the camera at the time of capturing the image,
As a process of outputting the image,
The imaging program according to any one of claims 1 to 3, wherein a process of outputting an image in which a difference in mutual orientation information is within an allowable range with the learning information is executed. In the computer,
An image obtained by continuously capturing the subject is acquired as an image as a candidate for the target image;
Obtaining distance information between the subject and the camera that captured the image at the time of capturing the candidate image;
Obtaining orientation information of the camera at the time of capturing the candidate image;
Further executing a process of presenting the candidate image to the user,
As the process of extracting the component, the component is extracted from the candidate image selected by the user,
Processing for generating learning information in which the candidate image from which the component is extracted is set as the target image and the target image is associated with the distance information at the time of capturing the component and the target image. The imaging program according to any one of claims 1 to 4, wherein the imaging program is executed. Computer
Acquire an image of the subject continuously captured,
Obtaining distance information between the subject and the camera that captured the image at the time of capturing the image;
Extracting a component in which the position of the subject and the position of the part on the image are associated from the image,
The component in which the position of the target subject and the position of the part on the target image are associated with each other at the time of capturing the target image including the target subject part, and the target subject and With reference to a storage unit that stores learning information associated with distance information between cameras that have captured the target image, the constituent elements match the learning information in the acquired image, and An imaging method comprising: executing an output process of an image in which a difference in distance information is within an allowable range. Acquire an image of the subject continuously captured,
An acquisition unit that acquires distance information between the subject and the camera that captured the image at the time of capturing the image;
An extraction unit that extracts a component in which the position of the subject and the position of the part on the image are associated from the image;
The component in which the position of the target subject and the position of the part on the target image are associated with each other at the time of capturing the target image including the target subject part, and the target subject and With reference to a storage unit that stores learning information associated with distance information between cameras that have captured the target image, the constituent elements match the learning information in the acquired image, and An output unit that outputs an image in which the difference in distance information is within an allowable range;
An information processing apparatus comprising:

Images