JP2015082823A - Imaging control apparatus, imaging control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which, when a user's ambient environment is automatically imaged by detecting images that are meaningful to the user, a method of imaging the user's ambient environment at every constant period largely consumes a battery.SOLUTION: On the basis of a direction in which a user is facing and a direction in which an object to be imaged is positioned, an image is obtained by imaging the direction in which the user is facing. In addition, on the basis of a result obtained by analyzing the image, a direction in which the object to be imaged is positioned is set.

Description

  The present invention relates to a photographing control apparatus, a photographing control method, and a program suitable for efficiently and automatically photographing a user's surrounding environment.

  In recent years, a camera based on SenseCam technology (Non-Patent Document 1) has been developed as a lifelog camera. Some cameras based on the SenseCam technology record the user's field of view for a long time by continuously photographing the front of the user at predetermined time intervals. In addition, various sensors (temperature sensor, illuminance sensor, acceleration sensor, geomagnetic sensor, color sensor, etc.) are used to automatically shoot only when changes in the outside world are detected, or to suppress automatic shooting according to the brightness of the outside world. There is something.

  In addition, there is known a photographing method in which a digital camera or a camera-equipped mobile terminal is equipped with various sensors such as a geomagnetic sensor, a gyro sensor, and an acceleration sensor, so that photographing can be performed without the user intentionally pressing the shutter button. ing. For example, Patent Document 1 discloses a technique in which a mobile terminal stores an image capturing direction in advance in a mobile terminal so that the mobile terminal captures an image in the same direction as the stored direction. Japanese Patent Application Laid-Open No. 2003-228561 discloses a technique for photographing in accordance with a place, direction, and order in which the camera is set by setting in advance the place, direction, and order in which the user takes images.

JP 2013-85048 A JP 2013-55508 A

Microsoft Research / SenseCam, [online], [October 2, 2013 search], Internet <URL: http://research.microsoft.com/en-us/um/cambridge/projects/sensecam/>

  However, there is no technique suitable for efficiently and automatically photographing the user's surrounding environment. For example, according to the technique of Non-Patent Document 1, it is possible to continuously perform automatic shooting in front of the user. However, since this technique continuously performs automatic shooting at predetermined time intervals, the battery consumption increases as the shooting interval decreases. In addition, there are some that reduce the amount of shooting by automatically shooting when a change in the external environment is detected, but it is not known whether the subject to be shot is included in the shot image, so shooting a meaningless shot May end up. On the other hand, when there is little change in the external world, such as when the written content of the whiteboard is updated, there is a possibility that the change will not be detected and shooting may be missed.

  Further, for example, according to the techniques disclosed in Patent Document 1 and Patent Document 2, it is possible to automatically capture images when the camera is directed in a capturing direction set in advance by the user. However, these techniques require the user to explicitly set the direction in which the user wants to shoot, and the shooting direction cannot be set automatically.

  An imaging control apparatus according to the present invention includes an imaging unit that acquires a captured image by capturing an image of a direction that the user is facing based on a direction in which the user is facing and a direction in which an imaging target is located, and the captured image And setting means for setting a direction in which the photographing target is located based on the result obtained by the analysis.

  ADVANTAGE OF THE INVENTION According to this invention, a user's surrounding environment can be automatically image | photographed efficiently.

It is an image figure which a user wears and uses an imaging | photography control apparatus. It is a hardware block diagram of a photography control device. It is a software block diagram of an imaging | photography control apparatus. It is a flowchart of the process which image | photographs imaging | photography object based on imaging conditions. It is an example of the imaging conditions described in the first embodiment. 6 is a flowchart of imaging processing described in the first embodiment. This is an example in which a rectangular area described in the first embodiment is set as an imaging condition. It is a flowchart of the imaging | photography process demonstrated in 2nd Example. It is an example of the imaging conditions demonstrated in 2nd Example. It is a flowchart of the imaging | photography process demonstrated in a 3rd Example. It is a flowchart of the imaging | photography process demonstrated in the 4th Example. It is an example of the imaging conditions at the time of marker detection demonstrated in the 4th Example. It is a flowchart of the process which image | photographs at the time of the relative position calculation demonstrated in 5th Example. It is a flowchart of the imaging | photography process demonstrated in a 6th Example. It is an example of the imaging conditions demonstrated in a 6th Example. It is a flowchart of the process which image | photographs the imaging | photography object demonstrated in a 6th Example based on imaging conditions. It is a flowchart of the imaging | photography process demonstrated in 7th Example.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, the constituent elements described in this embodiment are merely examples, and are not intended to limit the scope of the present invention only to them.

<First embodiment>
A first embodiment of the present invention will be described below with reference to the drawings. The imaging control apparatus 100 shown in FIGS. 1 to 3 is worn by a user in the office, and is always in a power-on state unless the user turns off the power.

  FIG. 1 is an image diagram in which a user wears the photographing control apparatus 100 according to the present embodiment on the head and uses it. An example of the imaging control device 100 is a glasses-type device that is worn on the user's head. The user can wear the glasses-type imaging control device 100 as shown in FIG. 1 on the head like actual glasses. In addition, the camera module 102 provided in the imaging control apparatus 100 can capture the direction in which the user is facing. In the present embodiment, the glasses-type shooting control device 100 is used, but any device may be used as long as it controls the shooting of a camera module capable of shooting the direction in which the user is facing. For example, it is not limited to the glasses type, and it may be applied to the user's neck or may be worn on the chest and photographed in front. In addition, as long as the imaging control device performs wireless communication with the camera module, the imaging control device itself may not be a wearable device as described above.

  In this embodiment, it is assumed that the user is in the office, and a planar rectangular object such as a whiteboard is assumed as an object to be tracked and photographed. In the following description, the direction in which the user is facing (that is, the direction in which the user is viewing) and the direction in which the imaging control device 100 is facing (that is, the direction in which the lens of the camera module 102 is facing) are the same. To do.

  FIG. 2 is a block diagram illustrating a hardware configuration example of the imaging control apparatus 100 in FIG. The imaging control apparatus 100 includes a CPU 101, a camera module 102, a network module 103, a ROM 104, an input module 105, a RAM 106, a display module 107, and a storage device 108. The imaging control apparatus 100 includes a gyro sensor 109, an acceleration sensor 110, an orientation sensor 111, and a timer 112. Each element is connected by a system bus 113. In addition, power is supplied to each unit by the battery 114.

  The CPU 101 is a unit for controlling the overall photographing control apparatus and executing a program to be described later. The ROM 104 stores a boot loader program and an imaging control program used for starting control. Since the RAM 106 is a volatile memory and can be accessed at high speed, information stored in the ROM 104 and information used temporarily are stored here. When the imaging control apparatus 100 is turned on, the CPU 101 reads and executes the boot loader program stored in the ROM 104, retrieves the imaging control program stored in the ROM 104, and stores it in the RAM 106. Then, the CPU 101 executes a shooting control program stored in the RAM 106 and executes each function provided in the shooting control apparatus 100.

  The camera module 102 includes known components such as a lens, a CCD, and an imaging device called a CMOS, and converts a captured image into a digital image. In this embodiment, the angle of view of the camera module 102 is set to 60 degrees in the horizontal direction and 30 degrees in the vertical direction. However, the present invention is not limited to this, and can be implemented with any angle of view.

  The network module 103 transmits / receives data to / from devices such as various servers and personal computers connected via the network 9000. The network module 103 can transmit the image data captured by the camera module 102 to a personal computer server or the like and store them in these devices. Note that when a configuration in which the camera module 102 is separated from the imaging control device is adopted, the network module 103 may be configured to transmit and receive data to and from the camera module 102.

  The input module 105 is used by a user wearing the imaging control apparatus 100 to operate the apparatus. The input module 105 includes well-known UI parts such as buttons and a touch panel, and the user can perform various operations including power on / off of the imaging control apparatus 100 using the input module 105.

  The display module 107 includes known display components such as an LED and a liquid crystal panel, and the user can check the state of the imaging control apparatus 100 by the light emission of the LED and the display of characters.

  The storage device 108 is used as a storage location for images taken by the camera module 102 and various setting data. It can also be used as a storage location for result data executed by the program.

  The gyro sensor 109 can output the angular velocity of the imaging control device 100. The gyro sensor 109 can calculate a relative angle from the reference direction by integrating the output angular velocities. The angle is generally expressed as pitch, yaw, or roll, and the inclination of the imaging control apparatus 100 can be expressed as a rotation angle by pitch, yaw, or roll expression. As the gyro sensor 109, a semiconductor element using a Coriolis force is known, but the present invention is not limited to this and can be implemented using any method.

  The acceleration sensor 110 can output the acceleration of the imaging control device 100. From the output acceleration value, the vertical direction of the imaging control apparatus 100 is known. The acceleration sensor 110 has a movable part in the semiconductor chip, and the acceleration applied from the outside moves the fin of the movable part, the distance between the fin of the non-movable part changes, and the capacitance changes. . The acceleration sensor 110 is known to be a piezoresistive type, a heat detection type, or the like, but the present invention can be implemented using any of these known methods.

  The azimuth sensor 111 can measure directions from east, west, south, and north by geomagnetism, and converts the measured information into digital information. With the azimuth sensor 111, it can be grasped in which direction of the east, west, south, and north the physical direction of the imaging control device 100 is directed.

  By performing signal processing on the output values of the gyro sensor 109 and the acceleration sensor 110 described above, it is possible to detect the user's walking motion. Specifically, the walking motion is detected using the periodicity and amplitude of the output values of the gyro sensor 109 and the acceleration sensor 110 when the user walks. Further, by combining the gyro sensor 109, the acceleration sensor 110, and the azimuth sensor 111, a relative position from a certain position of the photographing control apparatus 100 can be calculated. Since the user's walking motion and relative position detection processing are well-known techniques (see, for example, Japanese Patent Application Laid-Open No. 2012-145457), description thereof is omitted here.

  The timer 112 measures the current time by performing signal processing on data output from the clock generator.

  FIG. 3 is a block diagram of software modules of the imaging control apparatus 100. A program 300 is stored in the ROM 104 or the RAM 106 and is executed by the CPU 101 in order to realize the processing described in the present embodiment.

  The shooting control unit 301 controls the camera module 102 to perform a shooting instruction and received image data reception processing. Specifically, the camera module 102 is instructed to shoot according to the determination of the shooting determination unit 307, which will be described later, and the image data shot at that time is received. In this embodiment, the imaging control unit 301 performs preprocessing such as geometric correction and color adjustment of received image data.

  The shooting target determination unit 302 determines whether there is an object to be tracked and shot in the shot image data by image processing. In this embodiment, the shooting target determination unit 302 detects a rectangular area (that is, a rectangular surface) such as a whiteboard from the captured image data, and determines that the rectangular area is a shooting target when the rectangular area is detected. . This is because the rectangular area is representative of what the user wants to record as an image in an office, for example. For example, what is desired to be recorded can be specified as a rectangular area, such as a work record on a whiteboard, a projection of a projector displaying a presentation material, a monitor screen displaying a material, a notebook note, and a distribution material. Therefore, in the present embodiment, the rectangular area is determined as an imaging target. However, if all rectangular areas are targeted, all surrounding rectangular areas such as wall patterns and doors that are not significant to the user are tracked and photographed. For this reason, the rectangular area determined as the object to be photographed is approximate to a predetermined aspect ratio such as 4: 3, 16: 9 (for example, the calculated aspect ratio falls within 10% of the predetermined aspect ratio, respectively) It should be of a certain size or larger. The predetermined size here can be measured by the number of pixels of the captured image. In the detection of the rectangular area, the edge line segment in the captured image is extracted by the Hough transform, and a rectangular frame (that is, a rectangular area frame) is formed by combining two vertical lines and two horizontal lines from the set of extracted edge line segments. Done by generating. The aspect ratio of the rectangular area can be numerically obtained from a generated square frame by a known mathematical formula. In addition, since detection of a rectangular area and calculation of an aspect ratio are well-known techniques (for example, refer to Unexamined-Japanese-Patent No. 2012-238098), detailed description is omitted.

  The image recording unit 303 extracts a rectangular area that is an object to be imaged by the imaging object determination unit 302, records the image after correcting the keystone so that the user can easily see it later. Since trapezoidal correction is a known technique, a detailed description thereof is omitted here. Note that although the rectangular area is extracted here, the captured image itself including the object to be captured may be recorded.

  The image holding unit 304 holds the captured image recorded by the image recording unit 303 in the storage device 108.

  The photographing condition setting unit 305 sets photographing conditions for determining whether or not to photograph next in a certain direction. Specifically, when there is a rectangular area that is determined as a shooting target by the shooting target determination unit 302, the directions of the left and right and upper and lower ends of the rectangular area are calculated as angles based on the direction of the shooting control apparatus 100. Sets the center direction of the area. When the shooting target determination unit 302 determines that there is no shooting target, based on the direction of the shooting control device 100 and the angle of view of the camera, a range without a rectangular area is set as the angle of the left, right, top, and bottom ends. The set time is also set at the same time. An example of shooting condition setting will be described later with reference to FIGS. 6 and 7, and a more detailed description thereof is omitted here.

  The shooting condition holding unit 306 holds the shooting condition setting data set by the shooting condition setting unit 305 in the storage device 108.

  The shooting determination unit 307 determines whether to shoot based on the shooting conditions held by the shooting condition holding unit 306, the current direction detected by the direction detection unit 308, and the current time output by the timer 112. . Specifically, the shooting determination unit 307 determines whether the current direction is the direction of the center of the rectangular area from the shooting conditions. If the current direction is the direction of the center of the rectangular area and a predetermined time has elapsed, the shooting determination unit 307 determines that shooting is to be performed, and otherwise determines that shooting is not to be performed. Since the shooting determination will be described later with reference to FIG. 4, a more detailed description is omitted here.

  The direction detection unit 308 detects the direction in which the imaging control apparatus 100 is currently facing based on the outputs of the direction sensor 111, the gyro sensor 109, and the acceleration sensor 110. Specifically, the direction from the earth's geomagnetism to the east, west, south, and north is measured using the direction sensor 111 on the same principle as the compass. However, the reliability of geomagnetism may be lowered indoors due to building reinforcement. Therefore, when it is determined that the reliability of the direction sensor 111 is low, the vertical component and the horizontal component are extracted from the output values of the acceleration sensor 110 and the gyro sensor 109, and the relative horizontal angle with respect to the reference direction is calculated. . Since the calculation method of the horizontal angle and the elevation angle by the gyro sensor 109 and the acceleration sensor 110 is a known technique (see, for example, Japanese Patent Application Laid-Open No. 2012-145457), a more detailed description is omitted. In the following description, for the sake of simplicity, the yaw axis is the axis in the zenith direction (opposite to the vertical direction), the roll axis is the horizontal axis that the user is looking upright, and the pitch axis is 90 clockwise from the roll axis. The horizontal axis that is rotated in degrees. The angle around the yaw axis is represented by 0 degree to 359 degrees where 0 degrees is north, 90 degrees east, 180 degrees south, and 270 degrees west, and the clockwise direction of the axis is positive. The angle around the pitch axis is expressed as 90 to -90 degrees, with 90 degrees directly above (that is, 90 degrees of elevation), 0 degrees horizontally, and -90 degrees directly below (that is, 90 degrees depression). Positive. The angle around the roll axis is expressed as 90 degrees to -90 degrees, with 0 degrees when the user is not tilting his head, 90 degrees when tilted to the right and 90 degrees when tilted to the right and -90 degrees when tilted to the left. The clockwise direction of the shaft is defined as the positive rotation direction.

  The walking detection unit 309 detects the user's walking motion using the gyro sensor 109 and the acceleration sensor 110. Whether or not the user is walking can be determined from the amplitude of the vertical component of the direction detection unit 308 described above. Further, it is known that periodic features appear in the vertical component and horizontal component of each sensor in human walking motion, and the number of steps can be measured by pattern matching between this feature and each sensor output value. Since the walking motion detection process is a known technique (see, for example, Japanese Patent Laid-Open No. 2012-145457), a more detailed description is omitted.

  The image comparison unit 310 calculates feature points / features of a plurality of images held in the image holding unit 304, and compares the similarities of the images. A technique called DoG (Difference Of Gaussian) is generally used for feature point detection. For the feature quantity extraction of each feature point, an image processing technique called SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) is generally used. Since image comparison based on feature points and feature amounts is a known technique, a more detailed description thereof is omitted here. An example using the image comparison unit 310 will be described in Example 2.

  The display output unit 311 performs display output to the display module 107 according to the comparison result of the image comparison unit 310. An example using the display output unit 311 will be described in a second embodiment.

  The input determination unit 312 determines a user input operation from the input module 105. For example, it is possible to determine whether the button of the input module 105 is pressed once (single push), continuously pressed twice (double push), or long pressed.

  The position calculation unit 313 calculates the relative position of the user with respect to a certain point. The relative position of the user with respect to a certain point is calculated based on the moving speed estimated from the stride estimated from the number of steps and the magnitude of the acceleration in the gravity direction of the user, and the direction in which the user is facing. Since the technique for calculating the relative position of the user from the gyro sensor 109, the acceleration sensor 110, and the azimuth sensor 111 is a known technique (see, for example, Japanese Patent Laid-Open No. 2012-145457), a more detailed description is omitted.

  Next, with reference to FIG. 4 and FIG. 5, processing for photographing a photographing target based on photographing conditions will be described. FIG. 4 is a flowchart of processing for photographing based on photographing conditions. FIG. 5 is an example of shooting conditions held in the shooting condition holding unit 306.

  The flowchart in FIG. 4 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100.

  In step S401, the direction detection unit 308 detects the direction in which the user is facing. Since the details of the direction detection have been described above, description thereof is omitted here.

  In step S402, the walking detection unit 309 detects the user's walking. Since the details of the walking detection have been described above, the description thereof is omitted here.

  In step S403, the walking detection unit 309 determines whether the user is walking based on the detection result in step S402. Here, if it is determined that the user is walking two or more steps, the process proceeds to step S405; otherwise, the process proceeds to step S404. Here, the number of steps is two or more, but the present invention is not limited to this and can be set to an arbitrary number of steps.

  In step S <b> 404, the shooting condition setting unit 305 clears the shooting conditions held in the shooting condition holding unit 306. That is, when it is determined in step S403 that the user is walking, the shooting condition setting unit 305 deletes the shooting condition. In this embodiment, the shooting condition is set depending on whether or not the user is facing the direction of the rectangular area to be shot. When the user is walking, the direction of the rectangular area to be imaged changes, so when it is determined that the user is walking here, the imaging condition setting unit 305 is an imaging condition holding unit. The shooting conditions held in 306 are deleted. After the shooting conditions are cleared in step S404, the process proceeds to step S412 described later.

  In step S405, the direction detection unit 308 determines whether the angle around the roll axis of the imaging control apparatus 100 is within −10 degrees to 10 degrees. If the angle is within 10 degrees to 10 degrees, the process proceeds to step S406; otherwise, the process proceeds to step S412. Here, −10 degrees to 10 degrees is a value that can be arbitrarily set, and is not limited thereto. The reason why the angle around the roll axis is determined to be within −10 degrees to 10 degrees is to determine whether the inclination of the user's head is small or not and perform shooting. In the present embodiment, control is performed using this step S405 so as not to consider the tilt of the head as much as possible, for example, so as not to shoot in a state where the head is greatly tilted in the office. It should be noted that setting including the roll axis may be performed in the imaging conditions described later, and control using the roll axis may be performed. In that case, step S405 may be omitted.

  In step S <b> 406, the shooting determination unit 307 reads the shooting conditions held in the shooting condition holding unit 306. Specifically, the shooting conditions are held as data as shown in FIG.

  Here, how to read the data in FIG. 5 will be described. The shooting conditions in FIG. 5 indicate in which direction around the user a rectangular area (that is, a rectangular surface) exists or does not exist. That is, the shooting conditions in FIG. 5 indicate whether or not a rectangular area (that is, a rectangular surface) exists or does not exist in an already shot area. The ID column is a number for uniquely identifying each row. The rectangle presence / absence column indicates whether or not a rectangular area exists. The rectangular center direction indicates the angle in the central direction of the rectangular area as viewed from the user. The rectangular area (upper and lower left and right) columns indicate in which direction the left and right and upper and lower line segments of the rectangular frame of the rectangular area (that is, the frame of the rectangular area) are in an angle viewed from the user. The non-rectangular region (left and right upper and lower ends) column indicates a region that is known to have no rectangle as the angle of the left and right and upper and lower ends of the region viewed from the user. The update time column indicates the time when each row was last updated.

  Specifically, the row of ID01 indicates the direction in which the rectangular area exists, and it can be seen that the center of the rectangular area is in the 115 degree direction around the yaw axis and in the 3.5 degree direction around the pitch axis. The ID01 rectangular area (left and right upper and lower ends) column indicates that the center point of the leftmost line segment of the rectangular rectangular frame is 100 degrees around the yaw axis, and the center point of the right end line segment is 130 degrees around the yaw axis. Show. In addition, the point farthest from the center of the square frame of the upper end line segment is 10 degrees around the pitch axis, and the point farthest from the center of the square frame of the lower end line segment is −3 degrees around the pitch axis. Yes. Here, for the sake of simplicity, the range of the rectangular area is represented as a rectangular area (100 degrees, 130 degrees, 10 degrees, -3 degrees). The time when ID01 was last updated is 2013/8/1 9:30:00. The ID02 line indicates that there is no rectangular area in the range of 0 degrees to 100 degrees around the yaw axis and 30 degrees to -40 degrees around the pitch axis. Here, for the sake of simplicity, a range in which no rectangular area exists will be expressed as a non-rectangular area (0 degrees, 100 degrees, 30 degrees, and -40 degrees). A specific method for calculating the angle will be described later with reference to FIGS.

  In step S <b> 407, the shooting determination unit 307 determines whether or not a shooting area in the direction in which the camera is currently facing (that is, a range in which the camera module 102 is shooting) includes an unshooted area. The unphotographed area is an area excluding, for example, each rectangular area or non-rectangular area (that is, a photographed area) of each photographing condition shown in FIG. In step S407, the photographing determination unit 307 determines whether or not the unphotographed area is included in the photographing area in the direction in which the unphotographed area is currently directed at a predetermined angle or more. For example, the direction in which the imaging control device 100 is currently facing is 250 degrees around the yaw axis and 0 degrees around the pitch axis. At that time, from the angle of view of the camera, the imaging region is in the range of 220 to 280 degrees around the yaw axis and 15 to -15 degrees around the pitch axis. Hereinafter, for the sake of simplicity, this imaging area is represented as an imaging area (220 degrees, 280 degrees, 15 degrees, and -15 degrees). When the current direction is the shooting area (220 degrees, 280 degrees, 15 degrees, and -15 degrees), the shooting area is not included in the areas set as the shooting conditions in FIG. That is, a rectangular area (100 degrees, 130 degrees, 10 degrees, -3 degrees), a non-rectangular area (0 degrees, 100 degrees, 30 degrees, -40 degrees) and a non-rectangular area (130 degrees, 200 degrees, 20 degrees, and -30 degrees). Therefore, the shooting determination unit 307 determines that the shooting area includes an un-shot area, and proceeds to step S410.

  Note that in the case of the shooting region (10 degrees, 70 degrees, 15 degrees, and -15 degrees), the shooting area is included in the non-rectangular areas (0 degrees, 100 degrees, 30 degrees, and -40 degrees), and thus the shooting determination is performed. The unit 307 determines that the non-photographed area is not included in the photographing area, and advances the process to step S408. If the imaged area and the imaged area overlap, if the unphotographed area includes 20 degrees or more around the yaw axis or the pitch axis, the image capturing determination unit 307 captures the unphotographed area. It is determined that it is included in the region, otherwise it is determined that it is not included. However, although it is 20 degrees or more here, it is not limited to 20 degrees and can be set to an arbitrary angle.

  In step S408, the shooting determination unit 307 determines whether or not a predetermined time has elapsed since the last shooting time. For example, if the predetermined time is 10 seconds, the process proceeds to step S409 if 10 seconds or more have elapsed since the last photographing time, and the process proceeds to step S412 otherwise. The predetermined time can be arbitrarily set, and is not limited to 10 seconds.

  In step S409, the shooting determination unit 307 determines whether or not the current direction is the shooting direction. In other words, the shooting determination unit 307 determines whether the shooting area in the current direction is a direction with a rectangular area or a direction with a non-rectangular area. For example, if the current direction is 115 degrees around the yaw axis and 3.5 degrees around the pitch axis (hereinafter referred to as the current direction (115 degrees, 3.5 degrees for simplicity)) in the shooting conditions of FIG. The current direction is the rectangle center direction. Accordingly, the shooting determination unit 307 determines that the current direction is the shooting direction, and proceeds to step S411. Otherwise, the process proceeds to step S410. Here, in order to allow errors such as sensor values and camera shake, if the current direction is within ± 10 degrees around each axis of the yaw axis or the pitch axis with respect to the center direction of the rectangle, the imaging determination unit 307 It is determined as the shooting direction. Here, the numerical value of ± 10 degrees is a value that can be arbitrarily set as a margin, and is not limited to this.

  In step S410, the shooting determination unit 307 determines whether or not the current time has passed a predetermined time or more from the update time associated with the current shooting direction (that is, the direction set as the non-rectangular region). Determine. That is, it is determined whether or not a predetermined time or more has passed since the last shooting time in the non-shooting direction. In other words, in order to check whether a new rectangular area has appeared in the current direction recognized as a direction not to be photographed, photographing is performed even when facing a non-rectangular area when a predetermined time has elapsed. Here, it is assumed that the predetermined time is 3 minutes. For example, if the current time is 2013/8/1 9:36:00 when facing the non-rectangular area of ID03 in FIG. 5, 3 minutes and 50 seconds have passed since the update time of ID03. The determination unit 307 determines that the predetermined time or more has elapsed, and proceeds to step S411. Otherwise, the process proceeds to step S412. In this case, the update time of ID03 is updated at the current time 2013/8/1 9:36:00. Although the predetermined time is 3 minutes here, it can be set to any value and is not limited to 3 minutes.

  In step S411, the shooting control unit 301 performs shooting processing. Details of the photographing process will be described with reference to the flowchart of FIG.

  In step S412, the input determination unit 312 determines whether the photographing control apparatus 100 has been terminated. Specifically, the shooting control process is terminated when the power is turned off by, for example, pressing and holding the power button of the shooting control device, and otherwise the process returns to step S401.

  next. The process for setting the shooting conditions during shooting will be described with reference to FIGS. FIG. 6 is a diagram illustrating an example of a flowchart of the photographing process in step S411 in FIG. FIG. 7 is an image diagram for setting a rectangular area as a shooting condition and an example of the shooting condition.

  The flowchart in FIG. 6 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100.

  In step S <b> 601, the imaging control unit 301 captures an image by controlling the camera module 102 in the current direction. For example, in step S601, data indicating a captured image such as 701 in FIG. 7A or 702 in FIG. 7D can be acquired.

  In step S602, the shooting target determination unit 302 detects a rectangle in the image shot in step S601. For example, a frame surrounded by a broken line 702 in FIG. 7B is a detected rectangular frame. Since the method of detecting the rectangle has been described in the section of the photographing target determination unit 302 in FIG. 3, the description thereof is omitted here.

  In step S603, the imaging target determination unit 302 determines whether a rectangle is detected by the rectangle detection in step S602. If a rectangle is detected, the process proceeds to step S604; otherwise, the process proceeds to step S607. For example, if a rectangle surrounded by a broken line 702 is detected as shown in FIG. 7B, the process proceeds to step S604.

  In step S604, the imaging target determination unit 302 determines whether or not the detected rectangle matches a predetermined condition. Specifically, it is determined whether the rectangle is within a predetermined aspect ratio or is a predetermined size or more. For example, it is determined whether or not a broken line 702 representing a rectangle in FIG. 7B meets a predetermined condition. Details have been described in the section of the photographing target determination unit 302 in FIG. 3, and thus description thereof will be omitted. If the predetermined condition is met, the process proceeds to step S605; otherwise, the process proceeds to step S607. Here, it is assumed that the rectangle in FIG. 7B meets a predetermined condition.

  In step S605, the shooting condition setting unit 305 sets the rectangular area detected in step S602 as the shooting condition as a rectangular area to be tracked and shot. For example, from the image obtained by photographing the rectangle in FIG. 7A, the photographing conditions are set as ID01 to ID05 in FIG. ID02 is a rectangular imaging condition represented by a broken line 702. The rectangular area (left and right upper and lower end) columns of ID02 represent the directions viewed from the user of the left and right and upper and lower line segments of the broken line 702 in FIG. Specifically, the center points of the left and right line segments of the broken line 702 in FIG. 7B are represented as 52.5 degrees (704) and 75 degrees (705), respectively, and are farthest from the rectangular centers of the upper and lower line segments. The points are expressed as 13.1 degrees (706) and -3.8 degrees (707), respectively. Further, the shaded areas (708 to 711) in FIG. 7C correspond to ID01 and ID03 to ID05, respectively. Specifically, the shaded area 708 corresponds to ID01. The shaded area 709 corresponds to ID03. The shaded area 710 corresponds to ID04. The shaded area 711 corresponds to ID05. The shaded area 708 has no rectangle at 30 ° to 52.5 ° around the yaw axis and 15 ° to −15 ° around the pitch axis. It is set as the angle. ID03 to ID05 are set similarly to ID01.

  In step S <b> 606, the image recording unit 303 performs keystone correction on the rectangular area detected in step S <b> 602 and records / holds it in the image holding unit 304. As described in the section of the image recording unit 303 in FIG. 3, trapezoidal correction is a known technique, and thus detailed description thereof is omitted here. Although an example in which the rectangular area is extracted and recorded is described here, the captured image data itself including the subject to be captured may be recorded.

  In step S <b> 607, the shooting condition setting unit 305 sets the shooting area as the shooting condition as an area to be tracked and not shot. For example, FIG. 7D shows an example of a photographed image when no rectangle is captured. At this time, an imaging condition is set as ID06 in FIG. 7E as an area where the direction of the captured image is tracked and the image is not captured. Specifically, the non-rectangular region (left and right upper and lower ends) is set as a camera photographing range of 100 degrees to 160 degrees, 10 degrees to -20 degrees.

  As described above, in the first embodiment, for example, the first area including the rectangle is automatically captured in the first direction in which the imaging area is the imaging area, and further, the area does not overlap with the first area and includes the rectangle. The second direction in which the second region that is not present becomes the photographing region is automatically photographed. Thereafter, when the camera module 102 is pointed in the first direction, the first direction is automatically shot, while when the camera module 102 is pointed in the second direction, the second direction is not shot. To control. Further, when the camera module 102 is directed in a third direction that does not overlap with the first direction and the second direction, control is performed so that the third direction is automatically captured.

  In the first embodiment, the processing for capturing a rectangular area or an unknown area from the shooting conditions and the example of setting the shooting conditions from the direction that is currently facing the captured image are shown. Thereby, the direction in which the subject is to be photographed can be tracked and photographed using each sensor with low power consumption based on the photographing condition, so that the number of photographing can be reduced and battery consumption can be prevented.

<Second embodiment>
A second embodiment for carrying out the present invention will be described below. In the first embodiment, an example is shown in which a rectangular area is detected from a captured image, and shooting conditions are set with the detected rectangular area as a shooting target. In the second embodiment, an example is shown in which image pattern matching between a past rectangular area and a current rectangular area is performed, and an image having no change is excluded from the rectangular area to be captured and tracked.

  Hereinafter, a description will be given centering on differences from the first embodiment. A method for setting shooting conditions according to the second embodiment will be described with reference to FIGS. FIG. 8 is a diagram illustrating an example of a flowchart of imaging processing according to the second embodiment. FIG. 9 is an example of imaging conditions in the second embodiment. The flowchart in FIG. 8 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100.

  Steps S801 to S805, step S813, and step S814 are the same as steps S601 to S607 in FIG. 6 described in the first embodiment, and therefore, here, the difference from the first embodiment will be mainly described.

  The difference from the first embodiment is that steps S806 to S812 detect a rectangular area with little change by pattern matching between the current rectangular area and a past rectangular area, and track the detected rectangular area with little change. This is where the user selects whether to shoot. Accordingly, the user can be notified of a rectangular area having no change, and the subsequent shooting can be excluded when the user inputs a button. FIG. 9 is the same as FIG. 5 of the first embodiment except for the photographing prohibition flag string. The difference from the first embodiment is that the shooting prohibition flag can be set in the shooting condition by the user's button input.

  In step S806, the imaging condition setting unit 305 determines whether or not there is a rectangular area detected in the past in the same area as the detected rectangular area (that is, a range surrounded by the angle of the rectangular area (left and right upper and lower ends) in FIG. 9). judge. Specifically, the newly detected rectangle is compared with the respective angles of the upper, lower, upper and lower corners of the past rectangular area held in the shooting conditions, and if each value is within ± 5 degrees, it is determined as the same rectangular area To do. The value of ± 5 degrees can be arbitrarily set, and is not limited to this.

  In step S807, the shooting condition setting unit 305 determines that the oldest image is 60 seconds among the rectangular area images held in the image holding unit 304 in association with the shooting conditions of the rectangular area detected in the past determined in step S806. It is determined whether or not it has been previously saved. If it has been saved 60 seconds or more ago, the process proceeds to step S808, and if not, the process proceeds to step S813. When a certain time has passed, there is a high possibility that it is clear that there is some change in the rectangular area or there is no change. Moreover, it is inefficient if pattern matching described later is always performed. Therefore, in this embodiment, pattern matching, which will be described later, is performed when a predetermined time has elapsed since the saved rectangular area image was saved. In addition, although demonstrated here as 60 seconds, you may use arbitrary values. Although the comparison target is described as the oldest image here, it may be compared with the newest image.

  In step S808, the image comparison unit 310 performs pattern matching between the detected rectangular area image and the oldest rectangular area image captured in the past, and calculates a matching rate. Since the image pattern matching has been described in the section of the image comparison unit 310 in FIG. 3, the description thereof is omitted here.

  In step S809, if the coincidence rate of the pattern matching of the image is equal to or less than a predetermined threshold, the image comparison unit 310 regards that there is a change, and proceeds to step S813, otherwise proceeds to step S810. For example, when the coincidence rate is 95% or more, it is considered that there is no change, and the process proceeds to step S810. The matching rate can be obtained from the calculated number of feature points and the ratio of the feature points that match each image. If there is a change, the process proceeds to step S813 in order to capture the rectangle in which the change has occurred, and the rectangular area is set as the capture condition as the capture area. On the other hand, if there is no change, the process proceeds to step S810 in order to confirm with the user whether or not to continue photographing the rectangular area.

  In step S810, the display output unit blinks the display module 107 (for example, LED) for a predetermined time to notify the user that there is no change in the rectangular area image. For example, by blinking for 3 seconds, the user is notified that there is no change in the rectangular area image.

  In step S811, the input determination unit 312 determines whether or not an input by the user to the imaging control device 100 is within a predetermined time from the start of LED blinking. For example, when a double push by the user is detected within 5 seconds after the LED blinks, it is determined that there is a predetermined input, and the process proceeds to step S813. Otherwise, the process proceeds to step S812.

  In step S812, the shooting condition setting unit 305 sets a shooting prohibition flag in the shooting condition of the rectangular area detected in step S802. Specifically, the photographing prohibition flag is set to ON as in ID01 in FIG. As a result, even if ID01 is a rectangular region, it is excluded from the subject to be tracked and photographed. When the shooting prohibition flag is set to ON, the shooting determination unit 307 determines that the rectangular area is not shot.

  In the second embodiment, a change in a rectangular area that is tracked and photographed is detected by image pattern matching, and if there is a rectangular area that does not change, the user is allowed to choose not to photograph that area thereafter. As a result, the continuous shooting of images without change is stopped, so that the number of shootings can be reduced.

<Third embodiment>
Hereinafter, a third embodiment for carrying out the present invention will be described. In the first and second embodiments, an example is shown in which shooting conditions are set so that all detected rectangles that meet the conditions are tracked. The third embodiment shows an example in which, when a plurality of rectangles are detected, the largest rectangle is selected and set as the shooting condition.

  Hereinafter, a description will be given centering on differences from the first embodiment. FIG. 10 is an example of a flowchart of imaging processing in the third embodiment. The flowchart in FIG. 10 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100. Note that steps S1001 to S1004 and steps S1007 to S1008 are the same as steps S601 to S607 of FIG. 6 described in the first embodiment, and thus description thereof is omitted here.

  The difference from the first embodiment is that, in steps S1005 to S1006, the imaging target determination unit 302 extracts the largest rectangular area when there are a plurality of rectangles. As a result, when there are a plurality of rectangles, the largest rectangular area is extracted, so that only a representative rectangular area can be photographed and stored. For example, when a user who is looking at a whiteboard looks at the screen of the PC at hand, it is assumed that a rectangular area of the whiteboard is extracted on the back of the PC screen. In this case, the user can achieve the purpose of automatic shooting by shooting and saving the PC screen that is focused on and is extracted as a larger rectangular area at a closer distance. Therefore, in this embodiment, when there are a plurality of rectangles, the largest rectangular area is extracted.

  In step S1005, the shooting target determination unit 302 determines whether there are a plurality of rectangular regions determined in step S1004. If there are more than one, the process proceeds to step S1006; otherwise, the process proceeds to step S1007.

  In step S1006, the imaging target determination unit 302 calculates the length of a plurality of rectangular square frames, and extracts the rectangle with the longest frame as the largest rectangular region.

  In the third embodiment, only the largest representative rectangle is set to be photographed. This eliminates small rectangles that are additionally detected, thereby reducing extra shooting.

<Fourth embodiment>
The fourth embodiment for carrying out the present invention will be described below. In the first, second, and third embodiments, the detected rectangle is set as the shooting condition. In the fourth embodiment, an imaging condition setting method when a marker prepared in advance from a captured image is detected will be described.

  Hereinafter, a description will be given centering on differences from the first embodiment. A method for setting shooting conditions according to the fourth embodiment will be described with reference to FIGS. 11 and 12. FIG. 11 is an example of a flowchart of photographing processing in the fourth embodiment. FIG. 12 is an example of a photographed image and photographing conditions in the fourth embodiment. The flowchart in FIG. 11 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100. Steps S1101 and S1110 to S1115 are the same as steps S601 to S607 described with reference to FIG. 6 of the first embodiment, and thus description thereof is omitted here.

  The difference from the first embodiment is that when the shooting prohibition marker or the shooting request marker is detected in steps S1102 to S1109, the shooting area is set as a non-shooting area or a shooting area, respectively. is there. As a result, it is possible to prohibit the subsequent shooting without recording the shot image of the area where the shot image is not desired to be left by the shooting prohibition marker. Further, it is possible to prompt photographing even in an area where a rectangle is not detected by a marker for photographing request.

  In step S1102, the shooting target determination unit 302 detects a marker from the shot image. For example, the marker is like 1202 in FIG. 12 (a) or 1204 in FIG. 12 (b). In general, the marker is detected by detecting a predetermined frame from a captured image by image processing. Also, different types of markers can be identified by the design within the marker frame. Since the marker detection technique is a known technique, further detailed description is omitted.

  In step S1103, the shooting target determination unit 302 determines whether a marker is detected in step S1102. If a marker is detected, the process proceeds to step S1104; otherwise, the process proceeds to step S1110.

  In step S1104, the photographing target determination unit 302 determines whether or not the detected marker is a photographing prohibition marker 1202 as illustrated in FIG. If it is a photographing prohibition marker, the process proceeds to step S1105; otherwise, the process proceeds to step S1106.

  In step S1105, the shooting condition setting unit 305 sets shooting prohibition conditions as shooting conditions. Specifically, as shown by ID01 in FIG. 12C, the marker row is prohibited from being shot. Also, as shown in FIG. 12A, the center direction of the photographing prohibition marker 1202 is calculated by the current direction and image processing, and the center direction of the marker is an angle around the yaw axis and the pitch axis. ) Marker direction. As a result, the shooting determination unit 307 performs control so that shooting is not performed when the shooting image of the shooting control apparatus 100 faces a direction in which it is determined that the marker 1202 is included.

  In step S1106, the shooting target determination unit 302 determines whether the detected marker is a shooting request marker 1204 as illustrated in FIG. If it is a photographing request marker, the process proceeds to step S1107, and if not, the process proceeds to step S1108.

  In step S1107, the shooting condition setting unit 305 sets shooting request conditions as shooting conditions. Specifically, a marker row is set as an imaging request, as indicated by ID02 in FIG. Similarly to step S1105, the center direction of the imaging request marker 1204 in FIG. 12B is calculated, and the angles around the yaw axis and the pitch axis are set in the marker direction in FIG. Thus, as in the rectangular area shooting, when it is determined that the current direction of the shooting control apparatus 100 matches the marker direction, control is performed to perform shooting.

  In step S1108, the image recording unit 303 records the captured image captured in step S1101 in the image holding unit 304.

  In step S1109, when the imaging control apparatus 100 detects another marker other than the markers described in this embodiment, the imaging control apparatus 100 performs other processing. The other processing does not define the processing, and no processing may be performed.

  In the fourth embodiment, the method for setting the shooting condition when shooting the shooting prohibition marker or the shooting request marker is described. As a result, shooting conditions are set according to the markers attached in the office, so that shooting can be controlled more flexibly.

<Fifth embodiment>
The fifth embodiment for carrying out the present invention will be described below. In the first, second, third, and fourth embodiments, an example is shown in which shooting conditions are set based on analysis of a shot image and the current direction. In the fifth embodiment, a case will be described in which after the user moves from a certain place and the photographing condition is cleared, the user returns to the original place again.

  Hereinafter, a description will be given centering on differences from the first embodiment. FIG. 13 is a flowchart of processing for photographing based on photographing conditions in the fifth embodiment. The flowchart in FIG. 13 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100. Steps S1301 to S1303 and S1308 to S1316 are the same as steps S401 to S412 in FIG. 4 of the first embodiment, and thus description thereof is omitted here.

  The difference from the first embodiment is that, in the case of YES in step S1303, the imaging conditions that have been cleared when the user returns to the original position by calculating the relative position of the user in steps S1304 to S1307. Is about to restore. As a result, even when the user moves, if the user returns to the original position, the shooting can be controlled again with the shooting conditions held so far.

  In step S <b> 1304, the position calculation unit 313 calculates the relative position of the user with respect to a certain point from the direction in which the imaging control device 100 is facing and the moving speed. Since the calculation of the relative position of the user has been described in the section of the position calculation unit 313 in FIG. 3, the description here is omitted.

  In step S1305, the imaging condition setting unit 305 determines whether or not the current relative position calculated in step S1304 is a position at the start of walking. If the relative position is the position at the start of walking, the process proceeds to step S1307; otherwise, the process proceeds to step S1306.

  In step S <b> 1306, the shooting condition setting unit 305 takes a temporary backup of the shooting conditions held in the shooting condition holding unit 306. When a predetermined time (for example, 5 minutes) elapses, the backup is regarded as the user does not return to the original position and is cleared.

  In step S1307, since the shooting condition setting unit 305 determines that the user has returned to the position before movement in step S1304, the shooting condition cleared in step S1308 is restored from the backup. However, the backup of the imaging conditions performed in step S1306 is cleared after a predetermined time (for example, 5 minutes) has elapsed. Therefore, the restoration is valid only within the time when the backup performed in step S1306 remains.

  In the fifth embodiment, an example is shown in which the shooting conditions are restored when the user's walking is detected and the shooting conditions are cleared and then the original position is restored. Thus, even if the user moves temporarily, shooting can be started under the same conditions as long as the user returns to the original position, so that it is not necessary to set the shooting conditions again. Therefore, since the previously set direction for non-photographing can be used as it is, extra photographing can be reduced.

<Sixth embodiment>
The sixth embodiment for carrying out the present invention will be described below. In the first, second, third, fourth, and fifth embodiments, the example is shown in which the shooting conditions are set when the entire rectangle is shot. In the sixth embodiment, an example of shooting condition setting when a rectangular area is partially captured and a method of shooting determination will be described.

  Hereinafter, a description will be given centering on differences from the first embodiment. The shooting condition setting and shooting process of the sixth embodiment will be described with reference to FIGS. FIG. 14 is a flowchart of imaging processing according to the sixth embodiment. FIG. 15 is an image diagram of captured images and an example of shooting conditions in the sixth embodiment. The flowchart in FIG. 14 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100.

  Steps S1401 to S1403 and S1406 to S1409 are the same as steps S601 to S607 in FIG. 6 of the first embodiment, and a description thereof will be omitted.

  When the difference from the first embodiment is NO in step S1403, in steps S1404 to S1405, it is determined whether or not a rectangular object is included at both ends of the captured image, and an area including the target is determined as an imaging target. The shooting conditions are set as follows. Examples of a rectangular object include a U-shaped frame line as shown in FIG. The U-shaped frame line can be said to be a rectangle having one side as the end of the captured image. A rectangular object is called a quasi-rectangle. If there is such an object, the image is taken immediately at the timing when the U-shaped frame line enters as large as possible without being cut off from the photographed image (for example, FIG. 15B). Thereby, since the area | region like a rectangle is image | photographed quickly and correctly, it can be determined immediately whether the object like a rectangle is a rectangle. Further, in the embodiments so far, since the rectangular area is not a rectangle, the shooting condition may be set as a non-shooting area. Therefore, there is a possibility that the shooting may be missed. Decrease.

  In step S <b> 1404, the shooting target determination unit 302 determines whether there is a U-shaped frame line that covers either end of the shot image. For example, a broken line 1502 in FIG. 15A is an image of a U-shaped frame line that is applied to the left end of the photographed image 1501. Since the detection of the frame line can be performed in the same manner as the detection of the rectangle, the detailed description is omitted. If it is determined that there is a U-shaped frame line, the process proceeds to step S1405; otherwise, the process proceeds to step S1409.

  In step S1405, the shooting condition setting unit 305 sets shooting conditions so that a rectangular area including a U-shaped frame line is shot next. Specifically, when a broken line 1502 as shown in FIG. 15A is detected, the imaging condition is set with the rectangular presence / absence column of ID02 in FIG. In addition, since the upper right and lower frame lines of the rectangle are detected, the value is set by setting the direction of the left line of the rectangular region (left and right upper and lower end) columns as-(unknown).

  Next, a process of shooting based on shooting conditions in the sixth embodiment will be described with reference to FIG. The flowchart in FIG. 16 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100. Steps S1601 to S1606 and S1608 to S1613 are the same as steps S401 to S412 in FIG. 4 of the first embodiment, and thus description thereof is omitted here.

  The difference from the first embodiment is that, in S1607, it is determined whether or not the current direction is a direction in which a U-shaped region is greatly reflected. In step S <b> 1607, the shooting determination unit 307 determines whether or not the current direction is a direction in which a U-shaped region of the shooting condition is enlarged. For example, it is determined whether or not the shooting direction is a direction as shown in FIG. That is, it is determined whether or not the method includes three sides of the quasi-rectangle that are located in the opposite direction to the edge of the photographed image that includes three sides that have been confirmed and that has formed one undefined side. To do. Specifically, according to FIG. 15C, which is the imaging condition set in step S1405 in FIG. 14, the right and top directions of the U-shaped region are directions of 75 degrees, 13.1 degrees, and -3.8 degrees. It is known that the left end is an undetermined frame line. Therefore, the shooting determination unit 307 determines whether or not the right end of the U-shaped region is as close to the right end of the shooting direction as possible so that the left end of the rectangle can be seen. Here, since the right end of the U-shaped region is 75 degrees, it is determined that the photographing is performed when the photographing direction around the yaw axis is in the range of 80 degrees to 85 degrees. The reason why the angle is set to 80 ° to 85 ° instead of 75 ° is because a sensor error or blur is taken into consideration, and the numerical value is not limited to this.

  When YES is determined in the step S1607, as shown in FIG. 15D without waiting for a predetermined time (for example, 10 seconds) at the timing when the user faces the direction in which the right end of the rectangle 1503 is captured by the photographing process in the subsequent step S1612. Shooting conditions can be set.

  In the sixth embodiment, an example is shown in which when a rectangular area is shown halfway, shooting for determining whether the area is rectangular is performed quickly and accurately. As a result, even when the rectangle appears halfway, it is possible to set conditions for immediately capturing without waiting for a predetermined time.

<Seventh embodiment>
The seventh embodiment for carrying out the present invention will be described below. In the first, second, third, fourth, fifth, and sixth embodiments, an example in which the direction of the rectangle and the shooting marker is set as the shooting direction is shown. In the seventh embodiment, an example of shooting condition setting when a specific object such as a whiteboard or a display is recognized in an image by so-called specific object recognition will be described.

  Hereinafter, a description will be given centering on differences from the first embodiment. The shooting process according to the seventh embodiment will be described with reference to FIG. FIG. 17 is a flowchart of imaging processing according to the seventh embodiment.

  The flowchart in FIG. 17 is realized by the CPU 101 executing a program stored in the ROM 104 of the imaging control apparatus 100.

  Steps S1701 and S1706 are the same as steps S601 and S607 in FIG. 6 of the first embodiment, and a description thereof will be omitted here.

  The difference from the first embodiment is that, in steps S1702 to S1705, the detected rectangle is not set as an object to be imaged, but a specific object recognized by specific object recognition is set as an object to be imaged. Since specific object recognition can be set as a target for photographing a previously specified object, any object other than a rectangle can be set as a target for photographing.

  In step S1702, the shooting target determination unit 302 recognizes a specific object from the shot image. In the specific object recognition, a specific object can be recognized from a captured image by specifying an object significant to the user such as a whiteboard or a display in advance. In the specific object recognition, feature quantities (for example, HOG, haar-like, etc.) of an object to be recognized are learned in advance from a large number of correct images and non-correct images, and a discriminator is generated. From the classifier and a given image, it can be determined whether a specific object is present in the image. In addition, since specific object recognition is a well-known technique, the detailed description beyond this is omitted.

  In step S1703, the shooting target determination unit 302 determines whether a specific object such as a whiteboard or a display has been recognized. If the specific object is recognized, the process proceeds to step S1704; otherwise, the process proceeds to step S1706.

    In step S1704, the shooting condition setting unit 305 sets the specific object recognized in step S1702 as shooting conditions as shooting conditions. In general, in specific object recognition, a rectangular frame indicates where a specific object is in a captured image. In the present embodiment, the rectangular center direction of the specific object extracted as a rectangular frame is set as a shooting condition as a shooting direction by the same method as in FIG. 7B of the first embodiment.

  In step S1705, the shooting condition setting unit 305 records the image of the specific object recognized in step S1702 in the image holding unit 304. In this example, the area to be recorded is an area where a specific object is extracted with a rectangular frame, but an entire image including the specific object photographed in step S1701 may be used.

  In the seventh embodiment, when a specific object such as a whiteboard or a display is recognized, an example in which the direction of the specific object is set as a shooting direction is shown. Thereby, since an arbitrary object specified in advance can be recognized, an object other than a rectangle can be set as an object to be imaged.

<Other examples>
In the first to seventh embodiments, the case has been described in which the shooting control apparatus 100 sets the rectangular area, the predetermined marker, and the specific object as shooting directions that are significant for the user. However, in addition to the above, the shooting condition may be set as a direction in which the user explicitly captures a direction, a region in which characters are written, a region where there is much change, and the like. In the embodiment, the shooting direction is set in advance as the shooting condition. However, if the shooting target and the position of the user can be measured, the shooting direction may be calculated from the mutual positional relationship.

  In Example 1 or the like, when the angle at which the user is facing and the angle indicating the area where the shooting target is located coincide with each other, the angle indicating the area where the shooting target is positioned is set as the shooting condition, and shooting is performed. An example of performing is described. Here, it has been described that the angle at which the user is facing and the angle indicating the region where the photographing target is not necessarily completely coincide with each other, and a predetermined error can be allowed. That is, when the angle at which the user is facing and the angle indicating the area where the shooting target is located correspond to each other, the angle indicating the area where the shooting target is positioned can be set as a shooting condition and shooting can be performed. . Similarly, a certain angle (region, direction) matches another angle (region, direction) that allows a predetermined error, and a certain angle (region, direction) allows a predetermined error. Corresponding to the angle (region, direction).

  Although the second to seventh embodiments have been described mainly with respect to differences from the first embodiment, the present invention is also applicable to embodiments in which the first to seventh embodiments are appropriately combined. Is possible.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (30)

Photographing means for capturing a captured image by capturing the direction the user is facing based on the direction in which the user is facing and the direction in which the photographing target is located;
A shooting control apparatus comprising: setting means for setting a direction in which a shooting target is located based on a result obtained by analyzing the shot image.   The imaging control apparatus according to claim 1, wherein when the captured image includes a rectangle, the setting unit sets a direction in which the rectangle is positioned as a direction in which the imaging target is positioned.   The said setting means sets the direction in which the said rectangle is located as a direction in which the said imaging | photography object is located, when the said picked-up image contains the rectangle which has predetermined | prescribed aspect ratio. Shooting control device.   The said setting means sets the direction in which the said rectangle is located as a direction in which the said imaging | photography object is located, when the said picked-up image contains the rectangle which has predetermined | prescribed aspect ratio. One item is a photographing control device.   5. The setting unit according to claim 1, wherein when the captured image includes a plurality of rectangles, the setting unit sets a direction in which a large rectangle among the plurality of rectangles is positioned as a direction in which the imaging target is positioned. An imaging control device according to claim 1.   The said setting means sets the direction which image | photographed the said picked-up image as a direction where a picked-up object is not located, when the said picked-up image does not contain a rectangle, The any one of Claim 1 to 5 characterized by the above-mentioned. Shooting control device.   The imaging control apparatus according to claim 1, wherein the setting unit sets a shooting time in association with the direction.   The photographing unit does not photograph the direction in which the user is facing when the direction in which the user is facing corresponds to a direction set by the setting unit as a direction in which a photographing target is not located. The imaging control device according to any one of 1 to 7.   When the shooting unit corresponds to a direction set by the setting unit as a direction in which a shooting target is located, and a direction that is not set by the setting unit, The imaging control apparatus according to claim 1, wherein the imaging direction of the user is captured.   10. The photographing control apparatus according to claim 1, wherein the photographing unit does not photograph the direction in which the user is facing when a predetermined time has not elapsed since the last photographing.   The photographing unit is a case where a direction in which the user is facing corresponds to a direction set by the setting unit as a direction in which a photographing target is not located, and a predetermined direction after photographing the direction in which the user is facing The imaging control apparatus according to claim 1, wherein when the time has elapsed, the direction in which the user is facing is captured. .   The imaging control apparatus according to claim 1, wherein the setting unit deletes the previous setting when a user's walking is detected.   13. The photographing control apparatus according to claim 12, wherein the setting unit restores the deleted setting when it is detected that the user has returned to the original position.   14. The apparatus according to claim 1, further comprising a storage unit configured to store an image including the shooting target when the setting unit sets the direction in which the shooting target is located. Shooting control device. When the direction in which the user is facing corresponds to the direction set by the setting unit as the direction in which the imaging target is located, the captured image captured by the imaging unit and the direction captured in the past are stored in the storage unit A comparison means for comparing the image with
15. The photographing control apparatus according to claim 14, further comprising notification means for notifying a user that there is no change as a result of comparison by the comparison means.   The imaging control device according to claim 15, wherein the setting unit is configured to prohibit imaging in a direction in which the imaging target is located, in accordance with an instruction from a user according to a result of notification by the notification unit.   17. The photographing control apparatus according to claim 15 or 16, wherein the comparison unit performs comparison using pattern matching of image feature amounts.   18. The photographing according to claim 1, wherein when the photographed image includes a photographing prohibition marker, the setting unit sets a direction in which the photographing prohibition marker is located as a direction in which photographing is not performed. Control device.   19. The photographing according to claim 1, wherein when the photographed image includes a photographing request marker, the setting unit sets a direction in which the photographing request marker is located as a photographing direction. Control device.   In a case where the setting unit includes a quasi-rectangle that is a rectangle having one end of the captured image as one side, the position of the quasi-rectangle in which the one side of the quasi-rectangle is undetermined is a direction in which the shooting target is located. The imaging control device according to claim 1, wherein the imaging control device is set as follows.   If the direction in which the user is facing includes three determined sides of the quasi-rectangle and the three sides correspond to the direction of the position away from the position of the end, the user 21. The photographing control apparatus according to claim 20, wherein a photographing direction is taken.   The setting unit sets a direction in which the photographed image is photographed as a direction in which the photographing object is located when a result obtained by analyzing the photographed image is a specific object. The imaging control device according to any one of 21. Photographing means for capturing a captured image by capturing the direction the user is facing based on the direction in which the user is facing and the direction in which the photographing target is located;
An imaging apparatus comprising: setting means for setting a direction in which an imaging target is located based on a result obtained by analyzing the captured image. Photographing means;
A first direction in which a first area including a rectangle is an imaging area is imaged by the imaging means, and a second area that does not overlap with the first area and does not include a rectangle is an imaging area. After shooting the direction of with the shooting means,
When the photographing means is oriented in the first direction, the first direction is photographed by the photographing means,
And a control unit that controls the second direction not to be imaged by the imaging unit when the imaging unit is oriented in the second direction. The control means includes
After photographing the first direction with the photographing means and photographing the second direction with the photographing means,
The control unit is configured to control the third direction to be photographed by the photographing unit when the photographing unit is directed in a third direction that does not overlap with the first direction and the second direction. 24. A photographing control apparatus according to 24.   26. The photographing control apparatus according to claim 25, wherein the third direction corresponds to both a region including a rectangle and a region not including a rectangle.   27. The photographing control apparatus according to claim 24, wherein the photographing unit performs the photographing without an instruction from a user. An imaging step of capturing a captured image by capturing the direction the user is facing based on the direction in which the user is facing using the capturing unit and the direction in which the capturing target is located
A setting control step of setting a direction in which the shooting target is located based on a result obtained by analyzing the shot image. The first direction including the first area including the rectangle is imaged by the imaging unit, and the second area that does not overlap the first area and does not include the rectangle is the imaging area. After shooting the direction with the shooting means,
When the photographing means is oriented in the first direction, the first direction is photographed by the photographing means,
An imaging control method, comprising: a control step of controlling the second direction not to be imaged by the imaging unit when the imaging unit is directed in the second direction.   The program for functioning a computer as an imaging | photography control apparatus as described in any one of Claim 1 to 22.

Images