JP2010141385A - Device for selecting main object - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device capable of selecting a desired object from among a plurality of objects by specifying a movement locus of a target object. <P>SOLUTION: This device includes: an imaging part for imaging an object into a continuous frame group; a detection part for detecting the location of the object in each in the frame group; a movement locus calculation part for calculating the movement locus including the location of the object; a display part for displaying the movement locus; an input part for detecting pressure by a user on the display part; an input data generation part for calculating a pressing position based on the pressure; and a comparison part for selecting the object by comparing the movement locus with the pressing position. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an apparatus for selecting a subject on a screen.

The apparatus described in, for example, FIG. 11 of Patent Document 1 has a touch panel on a liquid crystal display (LCD) that displays a captured image. The user touches this touch panel to select a subject. The apparatus selects a subject corresponding to a pressed position (touched position) on the touch panel.
JP 11-136568 A

  A user using the above-described device touches a subject desired to be selected on the touch panel. However, when selecting a subject, if the subject moves quickly or randomly, it is difficult to press the subject accurately on the touch panel. When trying to touch the subject on the touch panel, another object may overlap the desired subject. Even in such a case, it is difficult to accurately press the subject on the touch panel.

  An object of the present invention is to provide an apparatus that allows easy selection of a subject.

  An object selection device according to the present invention includes an imaging unit that images a subject in a continuous frame group, a detection unit that detects the position of the subject in each of the frame groups, and an operation that calculates an operation locus including the position of the subject. A trajectory calculation unit, a display unit for displaying the motion trajectory, an input unit associated with the display unit for receiving input from a user, an input data generation unit for calculating a pressed position based on the input, and the operation A comparison unit that selects the subject by comparing the locus and the pressed position;

  According to an embodiment, the comparison unit selects the subject by comparing a distance between the motion locus and the pressed position.

  According to an embodiment, the comparison unit selects the subject by comparing the direction of the motion locus with the direction of the locus of the pressed position.

  According to an embodiment, the comparison unit selects the subject by comparing the shape of the motion locus with the shape of the locus of the pressed position.

  The apparatus according to the present invention compares a trajectory composed of points touched by the user or a group of points touched by the user with the motion trajectory of the subject. Based on this comparison, the target subject selected by the user from the plurality of subjects can be identified. The present invention has an effect that a user can easily select a desired subject from a plurality of subjects.

  An apparatus according to an embodiment of the present invention compares the distance between a point touched by a user and a motion locus of a subject. Based on this comparison, the target subject selected by the user from the plurality of subjects can be identified. This embodiment has an effect that it is not necessary to touch the subject or the motion locus of the subject itself, and the user can easily select a desired subject from a plurality of subjects by touching the vicinity thereof.

  An apparatus according to another embodiment of the present invention compares the distance between a locus made up of points touched by a user and the movement locus of a subject. Based on this comparison, the target subject selected by the user from the plurality of subjects can be identified. This embodiment has an effect that it is not necessary to touch the subject or the motion locus of the subject itself, and the user can easily select a desired subject from a plurality of subjects by touching the vicinity thereof.

  An apparatus according to still another embodiment of the present invention compares the direction of a trajectory composed of point clouds touched by a user and the direction of a motion trajectory of a subject. Based on this comparison, the target subject selected by the user from the plurality of subjects can be identified. This embodiment does not require touching the subject or the motion trajectory of the subject itself, and allows the user to easily select a desired subject from a plurality of subjects by tracing along the direction of the motion trajectory of the subject. Have.

  According to various embodiments of the present invention, a desired subject can be selected from a plurality of subjects by tracing the motion locus of the target subject or the vicinity thereof. In these embodiments, when the subject to be selected moves quickly or randomly, and even when the subject to be selected overlaps another object, by tracing the vicinity of the motion locus of the subject on the touch panel The target subject can be easily selected.

  Exemplary embodiments are described in detail below with reference to the drawings.

(System configuration)
FIG. 1 shows a configuration of a system 100 according to an exemplary embodiment. As an example, the system 100 can be a digital video camera. However, the system 100 can be applied to any electronic device having a display that displays a moving image or a still image and a touch panel associated with the display. Examples of such an electronic device include a television, a digital camera, a personal computer, and the like.

  The system 100 includes an imaging unit 110, a subject position detection unit (also simply referred to as “detection unit”) 120, an operation locus calculation unit 130, a display unit 140, an input unit 150, an input data calculation unit 160, a comparison unit 170, and a control unit. 180 included.

  The imaging unit 110 captures a moving image through the lens system, and outputs an electric signal 112 representing the captured moving image to the display unit 140. Such a moving image is composed of a plurality of continuous frame groups (for example, 60 frames per second) associated with the imaging time. A moving image typically includes a moving subject 102. The imaging unit 110 includes an imaging element. Examples of the imaging device include a CCD (charge coupled device) image sensor, a CMOS (complementary metal oxide semiconductor) image sensor, an imaging tube, and the like.

  The display unit 140 displays the captured moving image based on the signal 112 received from the imaging unit 110. The display unit 140 may display information other than the captured video (for example, time, remaining recording time, etc.). Typically, display unit 140 is an LCD (liquid crystal display) panel. Alternative implementations of the display unit 140 include plasma display panels and EL (electroluminescence) panels.

  The subject position detection unit 120 receives the signal 114 output from the imaging unit 110. Signal 114 typically represents a moving image that includes subject 102. The subject position detection unit 120 identifies the position of the subject 102 in each of a plurality of consecutive frame groups that form the moving image. Specifically, the subject position detection unit 120 detects, for example, a feature point of the subject 102 and specifies the position of the feature point. The subject position detection unit 120 outputs the specified feature point position information 122 to the motion trajectory calculation unit 130 as the position of the subject 102.

  The motion trajectory calculation unit 130 calculates motion trajectories 132 and 134 of the subject 102 based on the positions of the subject 102 in a plurality of frame groups. There may be a plurality of subjects 102 in a single frame. In such a case, motion trajectories 132 and 134 are calculated for each of the plurality of subjects 102.

  The display unit 140 displays the motion trajectory 132 of the subject 102 by overlapping with the above-described captured moving image represented by the signal 112. When there are a plurality of subjects 102 in the captured moving image, the display unit 140 displays a plurality of motion trajectories 132 corresponding to the plurality of subjects 102.

  The input unit 150 includes a plate-like position input device (for example, called a touch pad) associated with the display screen of the display unit 140. Such a position input device can be realized using any one of matrix switches, resistive films, surface acoustic waves, infrared rays, and capacitance. In FIG. 1, the display unit 140 and the input unit 150 are expressed as two functional blocks. However, the display unit 140 and the input unit 150 do not need to be separate elements (for example, electronic components), and can be integrally formed.

  The user presses a position corresponding to a desired subject on the input unit 150 in order to select one of a plurality of subjects typically displayed on the display unit 140. The input unit 150 generates a signal 152 indicating the position pressed by the user on the display unit 140 and outputs the signal 152 to the input data generation unit 160.

  In this specification, the “pressing position” typically refers to a single or a plurality of positions (groups) on the input unit 150 pressed by the user. When the user presses at one point on the input unit 150, the pressed position represents a single coordinate. When the user presses in a series of point groups so as to trace a trajectory on the input unit 150, the pressed position represents a plurality of coordinate groups corresponding to the input trajectory.

  The input data generation unit 160 calculates the coordinates of the position pressed by the user based on the signal 152 output by the input unit 150. The input data generation unit 160 outputs a signal 162 indicating the pressed position to the comparison unit 170.

  The comparison unit 170 compares the subject motion trajectory 134 calculated by the motion trajectory calculation unit 130 with the pressed position coordinates 162 calculated by the input data generation unit 160, thereby obtaining a “target subject” (also called a main subject). ) Is selected. The target subject is a subject that the user desires to adjust parameters (for example, brightness, color tone, focus) of the subject, and is typically a specific one of a plurality of subjects. Depending on the type of comparison performed by the comparison unit 170, the system 100 may select the desired subject by various techniques. These techniques are described below for Embodiments 1-6. The comparison unit 170 outputs a signal 172 indicating the current position of the selected subject to the control unit 180.

  Based on the received position information 172, the control unit 180 adjusts an image corresponding to the position in the frame. That is, the control unit 180 gives an instruction 182 to make a desired adjustment to the control object 190. Examples of such adjustment include brightness correction, color tone correction, and focus adjustment. The adjustment performed by the control unit 180 on the image may be performed automatically, or may be performed based on the user giving some instruction.

  For example, the control unit 180 can correct the brightness of the target subject in the backlight state to be brighter. Alternatively, the control unit 180 can drive the lens system so that a target subject that is not in focus in the current state is in focus. Therefore, in the former example, the control unit 180 outputs an instruction 182 for adjusting the brightness of the pixel represented by the position information 172 to the image processing apparatus that is the control target 190. In the latter example, the control unit 180 outputs an instruction 182 for adjusting the focus of the pixel represented by the position information 172 to the lens driving system that is the control target 190.

  All or some of the functions of the subject position detection unit 120, the motion trajectory calculation unit 130, the input data generation unit 160, the comparison unit 170, and the control unit 180 may be hardware, software, or any appropriate hardware and software. It can be realized by a combination. Examples of such hardware include an IC (Integrated Circuit), an LSI (Large Scale Integrated Circuit), and an ASIC (Application Specific Integrated Circuit).

(Embodiment 1)
The system 100 described with reference to FIGS. 1 to 3 selects a desired subject in a point mode. In the point mode, the user selects a target subject by touching a point near the subject on the input unit 150. FIG. 2 shows a plurality of subjects T1, T2, T3 imaged by the imaging unit 110 and corresponding motion trajectories A1, A2, A3. The subject position detection unit 120 detects the positions of a plurality of subjects T1, T2, and T3 in each of successive frames imaged by the imaging unit 110. The motion trajectory calculation unit 130 calculates corresponding motion trajectories A1, A2, and A3 based on the positions of the plurality of subjects T1, T2, and T3.

  Generally, when the subject moves during imaging, the position of the subject within the captured frame also changes. Even if the subject itself is stationary, if the imaging unit 110 pans or tilts, the position of the subject in the frame changes. In various embodiments, both of these cases are considered. In any situation, it is assumed that the position of the subject T1 in the frame changes over a plurality of frames. The subject position detection unit 120 outputs the position of the subject T1 in each frame to the motion trajectory calculation unit 130.

  The motion trajectory calculation unit 130 generates an motion trajectory A1 based on a plurality of positions of the subject T1, and outputs the motion trajectory A1 to the display unit 140. Similarly, the display unit 140 also displays operation trajectories A2 and A3 corresponding to the subjects T2 and T3, respectively. If the subjects T1, T2, and T3 are substantially stationary, the corresponding motion locus can be one point or a dense point group. The motion trajectory is obtained by applying each position of the subject in successive frames to a curve, for example. Examples of such curves include quadratic curves, cubic curves, spline curves, and Bezier curves. The motion trajectory may be calculated by a method other than curve fitting. For example, a curve approximation by a polygon obtained by connecting a plurality of subject positions by straight lines may be used.

  The pressed position coordinates calculated by the input data generation unit 160 correspond to the input position coordinates Ar in FIG. In FIG. 2, the distances between the input position coordinates Ar and the operation trajectories A1, A2, and A3 are DA1, DA2, and DA3, respectively. For example, when the user wants to select the subject T2, the user presses a point Ar such that the distance DA2 is shorter than DA1 and DA3. In other words, the system 100 determines which of the distances DA1, DA2, and DA3 is the minimum (DA2 is the minimum in FIG. 2), and selects the subject (T2 in FIG. 2) corresponding to the minimum distance.

  In many cases, the user needs to select a specific one (target subject) from a plurality of subjects. The system 100 can realize easy subject selection by the user. The system 100 enables control (for example, brightness correction) according to the subject selected by the user. The comparison unit 170 gives the position corresponding to the selected subject (for example, T2) to the control unit 180. Based on the received position information, the control unit 180 performs adjustment in accordance with the image corresponding to the position in the frame. Examples of such adjustment include brightness correction, color tone correction, and focus adjustment.

  For example, it is assumed that the user is photographing three children (a plurality of subjects) walking with the system 100 (for example, a video camera). The user wants to adjust the brightness of the face of the target subject (for example, one of three children) while viewing a plurality of subjects on the display unit 140. The user touches near the motion locus (for example, A2) of the target subject (for example, T2) on the input unit 150. As a result, the system can recognize that the user has selected the target subject. The controller 180 adjusts the brightness so that the brightness of the pixel corresponding to the position of the selected subject falls within an appropriate range.

  FIG. 3 shows an operation flow of a process 300 executed by a controller (for example, a microprocessor) described later. The process 300 allows the comparison unit 170 to select a target subject (eg, T2 in FIG. 2) from a plurality of subjects (eg, T1, T2, T3 in FIG. 2) in the point mode. In FIG. 3, the distance DA1 is the minimum distance among the distances between the pressing position Ar and the points on the motion locus A1. In the present specification, for the sake of simplicity, the distance DA1 is referred to as “distance of the operation locus A1 from the pressing position Ar”. Similarly, the distances DA2 and DA3 are distances of the operation trajectories A2 and A3, respectively. In the point mode, an operation locus corresponding to the minimum distance among the distances DA1, DA2, and DA3 is selected.

  In 310, the comparison unit 170 receives the input position coordinate Ar and the motion locus Aj corresponding to the subject Tj. The subscript j represents the subject. For example, when the comparison unit 170 receives motion trajectories Aj (j = 1, 2, 3) corresponding to the subjects T1, T2, and T3, the flow 300 is executed for the subscripts j = 1, 2, and 3. The n sampling points on the motion trajectory corresponding to the three subjects Tj are generally expressed as Ajk (j = 1, 2, 3; k = 1,..., N) (j, k: natural number). The subscript k represents the order of the sampling points of the subject position that moves continuously depending on time within the frame.

  At 320, the controller sets the initial value of k to 1 and sets the initial value of DAj to, for example, 65535. The initial value of DAj may be a sufficiently large value for calculating the minimum value described later. Therefore, the initial value of DAj can be any appropriate natural number. In 330, the controller determines whether the subscript k is n or less. If the determination in 330 is YES (subscript k is n or less), the controller repeatedly executes the operations of 340 to 360. In 340, the controller calculates a distance Dk between the pressed position coordinate Ar and a point Ajk on the motion trajectory corresponding to the subject Tj.

  At 350, the controller determines whether Dk is less than DAj. If the determination at 350 is YES (Dk is less than DAj), the controller assigns the value of Dk to DAj at 360 and proceeds to 365. If the determination at 350 is NO (Dk is not smaller than DAj), the process proceeds to 365.

  At 365, the controller increments the value of k by 1 before proceeding to 330. If the determination at 330 is NO (subscript k is greater than n), the controller sets the distance between the input position coordinate Ar and the motion trajectory Aj of the subject Tj to DAj at 370. When there are p subjects (p: natural number), the operations 310 to 370 are repeatedly executed to obtain the distance DAj corresponding to the motion locus Aj (j = 1,..., P; p: natural number).

  In 380, the controller selects a subject Tj corresponding to the smallest distance among DAj (j = 1,..., P; p: natural number). The comparison unit 170 outputs the position information 172 of the subject Tj selected in 380 to the control unit 180.

  In the case of FIG. 2, the motion trajectories A1, A2, A3 are composed of the same number (n) of sampling points. However, the number of sampling points is not limited to n, and more generally, the motion trajectories A1, A2, and A3 can be composed of different numbers of sampling points.

(Embodiment 2)
The system 100 described with reference to FIGS. 1, 4 and 5 selects a desired subject according to the trace mode. In the trace mode, the user selects a target subject by tracing (tracing) the motion locus of the subject on the input unit. The description of the system 100 with reference to FIG. 1 also applies to the second embodiment.

  Compared to the point mode with reference to FIG. 2 and FIG. 3, in the trace mode, the user selects a subject by tracing while keeping pressing on the motion trajectory of the target subject. FIG. 4 shows a trace locus AF of such a user. The input data generation unit 160 calculates the input trajectory AF. The distances between the input trajectory AF and the motion trajectories A1, A2, and A3 corresponding to the plurality of subjects T1, T2, and T3 are called distances DA1, DA2, and DA3.

  FIG. 5 shows an operational flow of a process 500 executed by a controller (eg, a microprocessor). The process 500 allows the comparison unit 170 to select a target subject (eg, T2 in FIG. 4) from a plurality of subjects (eg, T1, T2, T3 in FIG. 4) in the trace mode. In FIG. 4, the distance DA1 is the minimum distance among the distances between a specific sampling point AFi on the input locus AF and a point on the operation locus A1. In the present specification, for the sake of simplicity, the distance DA1 is referred to as “distance of the operation locus A1 from the sampling point AFi”. Similarly, the distances DA2 and DA3 are distances of the operation trajectories A2 and A3 from the sampling point AFi, respectively.

  In general, there are m (m: natural number) sampling points AFi (i = 1,..., M) on the input locus AF. If the user does not move the finger touching the input unit 150, there may be only one sampling point AFi (i = 1). The trace mode selects an operation locus corresponding to the minimum distance among the distances DA1, DA2, and DA3. Here, the sum of the distances DAj of the motion trajectory Aj from a plurality of sampling points AFi (i = 1,..., M) is SDAj (j = 1, 2, 3 in FIG. 4). In the trace mode, an operation locus (A2 in FIG. 4) corresponding to the minimum distance among the distances SDA1, SDA2, and SDA3 is selected.

  In 510, the comparison unit 170 receives an input locus AF composed of a plurality of sampling points AFi (i = 1,..., M) and an operation locus Aj corresponding to the subject Tj. The subscript i represents the order of sampling points. The subscript j represents the subject. For example, when the comparison unit 170 receives motion trajectories Aj (j = 1, 2, 3) corresponding to the subjects T1, T2, and T3, the flow 500 is executed for the subscripts j = 1, 2, and 3. The n sampling points on the motion trajectory corresponding to the three subjects Tj are generally expressed as Ajk (j = 1, 2, 3; k = 1,..., N) (j, k: natural number). The subscript k represents the order of the sampling points of the subject position that moves continuously depending on time within the frame.

  In 520, the controller sets the initial value of i to 1, sets the initial value of k to 1, sets the initial value of DAj to, for example, 65535, and sets the initial value of SDAj to 0. The initial value of DAj may be a sufficiently large value for calculating the minimum value described later. Therefore, the initial value of DAj can be any appropriate natural number. In 525, the controller determines whether the subscript i is less than or equal to m. If the determination at 525 is YES (subscript i is less than or equal to m), the controller repeatedly executes the operations of 530 to 560. In 540, the controller calculates a distance Dki between the sampling point AFi and the point Ajk on the motion trajectory corresponding to the subject Tj.

  In 530, the controller determines whether the subscript k is n or less. If the determination at 530 is YES (subscript k is n or less), the controller repeatedly executes the operations of 540 to 560. In 540, the controller calculates a distance Dki between the sampling point AFi and the point Ajk on the motion trajectory corresponding to the subject Tj.

  At 550, the controller determines whether Dki is less than DAj. If the determination at 550 is YES (Dk is less than DAj), the controller assigns the value of Dk to DAj at 560 and proceeds to 565. If the determination at 550 is NO (Dk is not smaller than DAj), the process proceeds to 565.

  At 565, the controller increments the value of k by 1 before proceeding to 530. If the determination at 530 is NO (subscript k is greater than n), at 570, the controller sets the sum of the current SDAj and DAj as the new SDAj. At 580, the controller increments the value of i by one.

  If the determination at 525 is NO (subscript i is not less than or equal to m), the controller proceeds to 585. In 585, the controller sets SDAj as the sum of the distances between the sampling points AFi on the input trajectory AF and the motion trajectory Aj of the subject Tj. When p subjects Tj (p: natural number) exist, operations 510 to 580 are repeatedly executed to obtain the distance SDAj corresponding to the motion locus Aj (j = 1,..., P).

  In 590, the controller selects a subject Tj corresponding to the smallest distance among SDAj (j = 1,..., P). The comparison unit 170 outputs the position information 172 of the subject Tj selected in 590 to the control unit 180.

  In the case of FIG. 4, the motion trajectories A1, A2, A3 are composed of the same number (n) of sampling points. However, the number of sampling points is not limited to n, and more generally, the motion trajectories A1, A2, and A3 can be composed of different numbers of sampling points.

(Embodiment 3)
The system 100 described with reference to FIGS. 1 and 6 selects a desired subject according to the trace mode. In the trace mode, the user selects a target subject by tracing the motion locus of the subject on the input unit. The description of the system 100 with reference to FIG. 1 also applies to the third embodiment.

  In the trace mode shown in FIG. 6, the user selects a subject by tracing the movement locus of the target subject as in the second embodiment. FIG. 6 shows such a trace AF AF by the user. The locus AF is typically composed of a plurality of sampling points AFi (i = 1,..., M; m = natural number). The input data generation unit 160 calculates the input trajectory AF.

  The comparison unit 170 calculates the normal AFNi at the sampling point AFi on the input locus AF based on the operation locus 134. The comparison unit 170 calculates the distances between the sampling points AFi and the operation trajectories A1, A2, and A3 along the normal AFNi. These distances are referred to as distances DA1i, DA2i, DA3i. For example, in FIG. 6, the intersection of the normal AFNi about the point AFi on the input locus and the operation locus A1 is A1Ni. The distance DA1i is a distance between the point AFi and the point A1Ni. Similarly, the distances DA2i and DA3i are defined as distances along the normal line at the point AFi.

  The comparison unit 170 calculates DA1i, DA2i, and DA3i for all sampling points (i = 1,..., M; m = natural number) of the input trajectory AF. The comparison unit 170 obtains distance sums SDA1, SDA2, and SDA3 by summing the distances DA1i, DA2i, and DA3i with respect to i = 1,..., M (m = natural number). The comparison unit 170 determines a minimum value among the distance sums SDA1, SDA2, and SDA3, and selects a subject Tj corresponding to the minimum value. The comparison unit 170 outputs the position information 172 of the selected subject Tj to the control unit 180.

(Embodiment 4)
The system 100 described with reference to FIGS. 1 and 7 selects a desired subject according to the trace mode. In the trace mode, the user selects a target subject by tracing the motion locus of the subject on the input unit. The description of the system 100 with reference to FIG. 1 also applies to the fourth embodiment.

  In the trace mode shown in FIG. 7, the user selects a subject by tracing the movement locus of the target subject as in the second embodiment. FIG. 7 shows such a trace AF AF by the user. The locus AF is typically composed of a plurality of sampling points AFi (i = 1,..., M; m = natural number). The input data generation unit 160 calculates the input trajectory AF. The comparison unit 170 calculates the input locus band AFB by adding a width to the input locus AF. The width of the input locus band AFB may be a predetermined fixed value. Alternatively, such a width may be a value set by means for accepting a user designation, for example.

  The comparison unit 170 determines whether or not the sampling points Ajk (j = 1, 2, 3; k = 1,..., N) (j, k: natural numbers) are included in the region AFB for each of the subjects T1, T2, and T3. And the number of sampling points included in the region AFB is calculated. The comparison unit 170 selects the subject Tj corresponding to the locus Aj having the largest number of sampling points Ajk.

(Embodiment 5)
The system 100 described with reference to FIGS. 1 and 8 selects a desired subject according to the trace mode. In the trace mode, the user selects a target subject by tracing the motion locus of the subject on the input unit. The description of the system 100 with reference to FIG. 1 also applies to the fifth embodiment.

  In the trace mode shown in FIG. 8, the user selects a subject by tracing the motion trajectory of the target subject as in the second embodiment. FIG. 8 shows a trace AF AF by such a user. The locus AF is typically composed of a plurality of sampling points AFi (i = 1,..., M; m = natural number). The input data generation unit 160 calculates the input trajectory AF. The comparison unit 170 calculates an input direction ID that represents the direction from the start point to the end point of the input trajectory AF.

  The comparison unit 170 also operates the movement directions MDj (representing directions from the start point or any past point to the current point of the movement trajectories A1, A2, and A3 corresponding to the plurality of subjects T1, T2, and T3. j = 1, 2, 3) is calculated. The input direction ID and the motion direction MDj can be expressed by, for example, a two-dimensional vector. The system 100 may define and calculate such a two-dimensional vector as, for example, a pair of two variables (x, y).

  The comparison unit 170 obtains a difference angle formed by the input direction ID and each operation direction MDj. In general, in order to obtain a difference angle between two vectors, an inner product of the vectors may be obtained. The comparison unit 170 selects the subject Tj corresponding to the movement direction MDj having the smallest difference angle. The comparison unit 170 outputs the position information 172 of the selected subject Tj to the control unit 180.

(Embodiment 6)
The system 100 described with reference to FIGS. 1 and 9 selects a desired subject according to the trace mode. In the trace mode, the user selects a target subject by tracing the motion locus of the subject on the input unit. The description of the system 100 with reference to FIG. 1 also applies to the sixth embodiment.

  In the trace mode shown in FIG. 9, the user selects a subject by tracing the movement locus of the target subject as in the second embodiment. FIG. 9 shows such a trace locus AF by the user. The locus AF is typically composed of a plurality of sampling points AFi (i = 1,..., M; m = natural number). The input data generation unit 160 calculates the input trajectory AF.

  The comparison unit 170 performs n samplings on the motion trajectory Aj, and calculates the average position Unj (j = 1, 2, 3) of the sampling points Ajk (j = 1, 2, 3; k = 1,..., N). calculate. The comparison unit 170 calculates a vector having the average position Unj as the start point and the sampling point Ajk as the end point in each motion trajectory Aj. The comparison unit 170 obtains a vector string TAj in which the vectors are arranged in the order of sampling in each motion trajectory, normalizes the size thereof, and obtains STAj.

  As with each motion trajectory, the comparison unit 170 also performs n samplings on the input trajectory, and obtains the average position of the sampling points AFk as Wn. The average positions Unj and Wn are positions in a virtual two-dimensional plane handled by the system 100. The average position in this specification is, for example, a point having an arithmetic average of the x component and y component of each point as the x component and y component.

  The comparison unit 170 calculates a vector starting from Wn and ending at each sampling point AFk at each sampling point. The comparison unit 170 obtains a vector sequence TAF in which the vectors are arranged in the sampling order in the input trajectory, normalizes the size, and sets it as STAF. The comparison unit 170 sequentially compares pairs of the vector sequence STAF and the vector sequence STAj to select a subject Tj corresponding to the vector sequence TAj having a small vector angle difference and a close size. The comparison unit 170 outputs the position information 172 of the selected subject Tj to the control unit 180.

  In the present embodiment, all the trajectories input by moving the touched point while being pressed are used as the input trajectory. However, a part of the input trajectory, for example, the first half of the input or the second half of the input may be used as the input trajectory.

  In the present embodiment, no distinction is made regarding the movement direction of the motion locus of the subject and the movement direction of the input locus. However, by considering the movement direction of the movement locus Aj of the subject and the movement direction of the input locus AF, the subject Tj corresponding to the movement locus Aj that is generally in the reverse direction can be ignored (ie, not selected). Good. In order to evaluate the relationship between the direction of the motion trajectory of the subject and the direction of the input trajectory, the comparison unit 170, for example, relates the relationship between the start position of the input trajectory and the sampling position in time-series order of the motion trajectory of the subject. Can be compared.

  10 and 11 show a method for evaluating the relationship between the direction of the motion trajectory of the subject and the direction of the input trajectory. The comparison unit 170 sequentially compares the distances of the sampling positions (AS1, AS2, AS3) in the time series order of the motion trajectory A of the subject T from the start point position AFS of the input trajectory AF. In the case shown in FIG. 10, this distance tends to be longer. Therefore, the direction of the input trajectory AF and the direction of the motion trajectory A of the subject T can be determined to be substantially the same direction. Therefore, since the direction of the operation locus A in FIG. 10 is substantially the same as that of the input locus AF, the corresponding subject T is selected.

  On the other hand, in the case shown in FIG. 11, this distance tends to be shorter. Therefore, the direction of the input trajectory AF and the direction of the motion trajectory A of the subject T can be determined to be generally opposite directions. Therefore, since the motion trajectory A in FIG. 11 is substantially opposite in direction to the input trajectory AF, the corresponding subject T is not selected.

  As mentioned above, Embodiment 1-6 of this invention was described in detail with reference to drawings.

  In the above-described first to sixth embodiments, any appropriate point of the motion trajectory can be selected as the start point of the target motion trajectory (for example, A11, A21, A31 in FIG. 2). However, the position of the subject in a frame that is temporally retroactive from the current frame may be selected as the start point of the motion trajectory. Alternatively, a point that is separated from the current subject position along the motion trajectory by a predetermined length can be selected as the start point of the motion trajectory. Furthermore, among points on the motion trajectory that can be displayed in the display unit 140, a point that is farthest from the current subject position along the motion trajectory can be selected as the start point of the motion trajectory. In either case, the current position of the subject may be used as the end point of the motion trajectory.

  In the above, the natural numbers i, j, k, m, n, and p can take arbitrary values as appropriate according to the embodiment. For example, in the motion trajectory Aj, the natural number j has been described as 1 to 3, but the present invention is not limited to this. When only one motion locus Aj exists in the frame, j = 1 holds.

  The controller used to perform the processes of embodiments 1-6 is typically a microprocessor. Alternatively, the controller can be implemented by other hardware or by any suitable combination of hardware and software. For example, the function of the controller can be realized in the form of a semiconductor chip such as an IC, LSI, or ASIC. Alternatively, these functions can also be realized by software running on a microprocessor.

  FIG. 12 is a block diagram of a controller 1200 that implements functional blocks of the system 100. The controller 1200 includes a CPU (Central Processing Unit) 1210, ROM (Read Only Memory) 1220, RAM (Random Access Memory) 1230, Image Memory 1240, HDD (Hard Disk Drive) 1250, Image Input I / F (Interface) 1260, Image It has an output I / F 1270, a position input I / F 1280, and an I / F 1290. These elements of the controller 1200 can be coupled by a system bus 1205 to exchange data in one or both directions. The arrow in the figure showing the direction of data is an example, and the present invention is not limited to this. For example, when the HDD 1250 is set to read-only, data is transferred only in one direction.

  The CPU 1210 can be typically realized by an IC, an LSI, an ASIC, or the like. ROM 1220 permanently stores a software program that achieves the necessary functional blocks. The RAM 1230 temporarily stores various programs or data as a main storage device of the CPU 1210. The image memory 1240 temporarily stores an image that the system 100 displays on the display unit 140. The HDD 1250 stores programs or data stored in the system 100 for a long period of time. The function of the controller 1200 may be realized as a sequencer regardless of the CPU and memory.

  The image input I / F 1260 is coupled to the imaging unit 110 and receives a captured image. The image output I / F 1270 is coupled to the display unit 140 and outputs, for example, an operation locus 132. The position input I / F 1280 is coupled to the input unit 150 and receives data 152 corresponding to the pressed position by the user. The I / F 1290 exchanges various data with an external device. For example, the I / F 1290 may be a network I / F that exchanges data via a network.

  The controller 1200 may be realized as a one-chip IC in whole or in part. Alternatively, the controller 1200 may be composed of a plurality of semiconductor chips.

  The apparatus according to the present invention can select a subject by comparing the motion trajectory of the subject and the pressed position input by the user, and can be applied to uses such as a subject selection device.

It is a block diagram which shows the structure of the system by example embodiment. It is a figure which shows the several to-be-photographed object and the corresponding operation | movement locus | trajectory imaged by the imaging part. It is a figure which shows the operation flow of the process which a controller performs. It is a figure which shows the locus | trajectory of the trace by a user. It is a figure which shows the operation flow of the process which a controller performs. It is a figure which shows the locus | trajectory of the trace by a user. It is a figure which shows the locus | trajectory of the trace by a user. It is a figure which shows the locus | trajectory of the trace by a user. It is a figure which shows the locus | trajectory of the trace by a user. It is a figure which shows the method of evaluating the relationship between the direction of the motion locus | trajectory of a to-be-photographed object, and the direction of an input locus | trajectory. It is a figure which shows the method of evaluating the relationship between the direction of the motion locus | trajectory of a to-be-photographed object, and the direction of an input locus | trajectory. It is a block diagram of the controller which implement | achieves the functional block of a system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 110 Image pick-up part 120 Subject position detection part 130 Motion locus calculation part 140 Display part 150 Input part 160 Input data generation part 170 Comparison part 180 Control part

Claims (4)

An imaging unit for imaging a subject in a continuous frame group;
A detection unit for detecting the position of the subject in each of the frame groups;
An operation trajectory calculation unit for calculating an operation trajectory including the position of the subject;
A display unit for displaying the movement locus;
An input unit for detecting a press by the user on the display unit;
An input data generation unit that calculates a pressing position based on the pressing;
A subject selection device comprising: a comparison unit that selects the subject by comparing the movement locus and the pressed position.   The subject selection device according to claim 1, wherein the comparison unit selects the subject by comparing a distance between the motion locus and the pressed position.   The subject selection apparatus according to claim 1, wherein the comparison unit selects the subject by comparing the direction of the motion locus with the direction of the locus of the pressed position.   The subject selection device according to claim 1, wherein the comparison unit selects the subject by comparing the shape of the motion locus with the shape of the locus of the pressed position.

Images