JP2008005208A - Camera automatic control system for athletics, camera automatic control method, camera automatic control unit, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To securely track and photograph attention drawing events during athletics by a camera automatic control unit used to photograph sports athletics etc. <P>SOLUTION: An image matching unit 14 of a controller 12 performs image matching between a face image 22 as an object of gaze detection photographed by a camera 11 for gaze detection and a face image model 25 in a face image storage section 15 to find a gaze vector 31 corresponding to the recognized face image model 25. A photographed object position estimating unit 16 finds a gaze 32 from the gaze vector 31 and finds a photographed object estimated position 36 from gazes 32 of a plurality of objects of gaze detection. A camera control unit 17 for athletic photography controls a camera 13 for athletic photography so that the camera 13 for athletic photography can track and photograph an object 52 to be photographed at the photographed object estimated position 36. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a camera automatic control system, a camera automatic control method, and a camera automatic control device, and more particularly to a camera automatic control system, a camera automatic control method, a camera automatic control device, and a program used for shooting sports sports.

  In video shooting of sports competitions such as figure skating, a technique has been used in which a cameraman manually moves the camera to shoot a shooting target during the competition. However, in order to always track and shoot the shooting target during the competition, an excellent technique is required. Also, since it is necessary to concentrate on shooting without taking a break while the competition continues, the burden on the photographer becomes very large. Furthermore, since the camera is operated by a human, there is a problem that the place where the camera can be installed is limited. In order to solve such a problem, a technique in which a camera automatically tracks a shooting target has been disclosed.

  For example, in Patent Document 1, the position of the shooting target is specified by performing image recognition using the characteristics of the shooting target on the image shot by the position detection camera attached to the position overlooking the competition area, A technique for performing tracking processing of a shooting camera with respect to a direction is disclosed.

  Similarly to Patent Document 1, Patent Document 2 specifies the position of an object to be captured from an image captured by a camera and performs tracking. However, the camera for position detection and the camera for capturing are shared, A technique for reducing the total number is disclosed.

  In Patent Document 3, a shooting target holds a transmitter that transmits a position information signal, and the position signal is received by a receiver installed near the video camera to identify the position of the shooting target, A technique for controlling the direction is disclosed.

Japanese Patent Laid-Open No. 10-276351 JP 2001-285849 A JP 2003-274257 A

  However, the above-described conventional technique has a problem in that there is a possibility that an object to be photographed may be missed in a highlight scene that is attracting attention by many spectators in the competition. This is because a highlight scene in a sports competition may be composed of a plurality of persons or objects depending on the competition, and the person or object to be photographed may change from moment to moment as the competition progresses. However, in the conventional technique described above, the shooting target tracked by the shooting camera is fixed to an object such as a specific person or a ball.

(Object of the present invention)
An object of the present invention is to provide an automatic camera control device capable of reliably (tracking) photographing an event (target to be photographed) in a highlight scene that has been noticed by many spectators in the above-described conventional technology. Is to provide.

  A first camera automatic control system according to the present invention includes a line-of-sight detection camera having a line-of-sight detection target photographing unit for photographing a line-of-sight detection target, each of face images obtained by photographing the line-of-sight detection target from a plurality of directions, and each face image. Face image storage means for storing a face image model table associated with a line-of-sight vector, and a line-of-sight vector corresponding to the face image of the line-of-sight detection target photographed by the line-of-sight detection target photographing means are extracted from the face image model table. Image collating means, photographing object position estimating means for obtaining a photographing object estimated position from the line-of-sight vector extracted by the image collating means, and competition shooting for obtaining photographing control information for photographing a photographing object at the photographing object estimated position And a camera automatic control device having camera control means.

  In the second camera automatic control system of the present invention, in the first camera automatic control system, the photographing target position estimating unit obtains the photographing target estimated position from a plurality of line-of-sight vectors of face images of the line-of-sight detection target. It is characterized by.

  According to a third camera automatic control system of the present invention, in the first or second camera automatic control system, the shooting control information is received from the competition shooting camera control means, and the shooting target is shot according to the shooting control information. It is characterized by comprising a competition shooting camera having shooting target tracking means.

  A fourth camera automatic control system of the present invention is the third camera automatic control system, wherein one of the output images of the plurality of game shooting cameras arranged at a predetermined position with respect to the shooting target is obtained. When the output video is switched so as to be selected, if there is another competitive shooting camera between the competitive shooting camera before switching and the switched competitive shooting camera, It is characterized by comprising a competition video switching device having output video switching means for sequentially switching output video from the side of the competition shooting camera before switching.

  A fifth camera automatic control system according to the present invention is the third camera automatic control system, wherein the competition shooting camera control means moves the competition shooting camera to move based on the shooting target estimation position. And obtaining movement control information for moving the game shooting camera.

  The sixth camera automatic control system of the present invention is the fifth camera automatic control system, wherein the camera position moving means for moving the competition shooting camera and the camera position moving means according to the movement control information are used to control the camera position moving means. It is characterized by comprising a moving camera device having a movement control means for moving a game shooting camera.

  In a seventh camera automatic control system of the present invention, in any one of the first to sixth camera automatic control systems, the photographing target position estimating unit obtains a line of sight from the line-of-sight vector, and the plurality of lines of sight are placed on a horizontal plane. The center of gravity of a plurality of horizontal plane intersections formed when unfolded is set as a horizontal direction estimated position of the imaging target estimation position, and the vertical heights of the plurality of intersections formed by the perpendicular and the line of sight on the plurality of horizontal plane intersections Is an estimated position in the height direction of the estimated position to be photographed.

  According to the first automatic camera control method of the present invention, the automatic camera control apparatus captures a line-of-sight vector corresponding to a face image of the line-of-sight detection target photographed by the line-of-sight detection camera, and a face image obtained by photographing the line-of-sight detection target from a plurality of directions. An image matching step for extracting each of the face image and a line-of-sight vector of each face image from a face image model table, and a shooting target position estimating step for obtaining a shooting target estimated position from the line-of-sight vector extracted in the image matching step; And a competition shooting camera control step for obtaining shooting control information for shooting the shooting target at the shooting target estimated position.

  According to a second automatic camera control method of the present invention, in the first automatic camera control method, the photographing target position estimating step obtains the photographing target estimated position from a plurality of line-of-sight vectors of face images of the line-of-sight detection target. It is characterized by.

  The third camera automatic control method of the present invention is the first or second camera automatic control method, wherein the competition shooting camera receives the shooting control information obtained in the competition shooting camera control step, and A shooting target tracking step of shooting the shooting target according to shooting control information is included.

  According to a fourth camera automatic control method of the present invention, in the third camera automatic control method, an output image of a plurality of the competition shooting cameras in which the competition video switching device is arranged at a predetermined position with respect to the shooting target. When switching the output video so as to select one of them, if there is another game shooting camera between the game shooting camera before switching and the game shooting camera after switching, An output video switching step of sequentially switching the output video of each game shooting camera from the side of the game shooting camera before switching is included.

  The fifth camera automatic control method of the present invention is the third camera automatic control method, wherein the competitive shooting camera control step is a movement target position to which the competitive shooting camera should move based on the shooting target estimated position. And obtaining movement control information for moving the game shooting camera.

  According to a sixth camera automatic control method of the present invention, in the fifth camera automatic control method, the mobile camera device includes a movement control step of moving the game shooting camera according to the movement control information.

  According to a seventh camera automatic control method of the present invention, in any one of the first to sixth camera automatic control methods, the photographing target position estimating step obtains a line of sight from the line-of-sight vector, and sets the plurality of lines of sight on a horizontal plane. The center of gravity of a plurality of horizontal plane intersections formed when unfolded is set as a horizontal direction estimated position of the imaging target estimation position, and the vertical heights of the plurality of intersections formed by the perpendicular and the line of sight on the plurality of horizontal plane intersections Is an estimated position in the height direction of the estimated position to be photographed.

  The first camera automatic control apparatus according to the present invention uses a line-of-sight vector corresponding to a face image to be detected by a line-of-sight detection camera, a face image obtained by shooting the line-of-sight detection target from a plurality of directions, and each face image. Image collating means for extracting from the face image model table in which the line-of-sight vector is associated, photographing object position estimating means for obtaining a photographing object estimated position from the line-of-sight vector extracted by the image collating means, and the photographing object estimated position It is characterized by comprising a camera control unit for competition shooting that obtains shooting control information for shooting the shooting object.

  In the second camera automatic control device according to the present invention, in the first camera automatic control device, the photographing target position estimating means obtains the photographing target estimated position from a plurality of line-of-sight vectors of the face image of the line-of-sight detection target. It is characterized by.

  According to a third camera automatic control device of the present invention, in the first or second camera automatic control device, the shooting control information is transmitted to the competition shooting camera, and the shooting target is shot according to the shooting control information. Features.

  According to a fourth automatic camera control apparatus of the present invention, in the third automatic camera control apparatus, the competitive shooting camera control means moves the competitive shooting camera to move based on the shooting target estimated position. And obtaining movement control information for moving the game shooting camera.

  According to a fifth camera automatic control apparatus of the present invention, in any one of the first to fourth camera automatic control apparatuses, the photographing target position estimating unit obtains a line of sight from the line-of-sight vector, and the plurality of lines of sight are placed on a horizontal plane. The center of gravity of a plurality of horizontal plane intersections formed when unfolded is set as a horizontal direction estimated position of the imaging target estimation position, and the vertical heights of the plurality of intersections formed by the perpendicular and the line of sight on the plurality of horizontal plane intersections Is an estimated position in the height direction of the estimated position to be photographed.

  According to a first program of the present invention, a gaze vector corresponding to a face image of a gaze detection target captured by a gaze detection camera, a face image obtained by shooting the gaze detection target from a plurality of directions, and a gaze vector of each face image Image matching processing extracted from the face image model table associated with the image, a shooting target position estimation process for obtaining a shooting target estimated position from the line-of-sight vector extracted in the image matching processing, and a shooting target of the shooting target estimated position The camera automatic control device is made to perform a camera control process for competition shooting for obtaining shooting control information for shooting.

  The second program of the present invention is characterized in that, in the first program, the photographing target position estimating process obtains the photographing target estimated position from a plurality of line-of-sight vectors of face images of the line-of-sight detection target.

  A third program of the present invention is characterized in that, in the first or second program, the shooting control information is transmitted to a competition shooting camera and the shooting target is shot according to the shooting control information.

  According to a fourth program of the present invention, in the third program, the game shooting camera control process is configured such that the game shooting camera is moved to a movement target position to which the game shooting camera should move based on the shooting target estimation position. It is characterized in that movement control information for moving the object is obtained.

  A fifth program of the present invention is formed when the shooting target position estimation process obtains a line of sight from the line-of-sight vector and develops a plurality of lines of sight on a horizontal plane in any one of the first to fourth programs. The center of gravity of a plurality of horizontal plane intersections is defined as a horizontal direction estimated position of the imaging target estimated position, and an average value of vertical heights of the plurality of intersections formed by the perpendicular and the line of sight on the plurality of horizontal plane intersections is captured. It is characterized by the height direction estimated position of the object estimated position.

  An effect of the present invention is that an automatic camera control device that tracks a subject to be photographed can reliably track an event that attracts attention in a sports competition or the like. The reason for this is that in order to detect the position of the object to be photographed, a person who is constantly concentrated in the competition, such as a referee or a record clerk, is photographed with a line-of-sight detection camera as a line-of-sight detection target. Compared with the prepared face image model, the estimated position of the shooting target is calculated from the line of sight corresponding to the recognized face image model, so the shooting target is not fixed to a specific person or object, and during the competition This is because an event focused on by the line-of-sight detection target can be set as a shooting target.

  Next, the best mode for carrying out the present invention will be described in detail with reference to the drawings. In the description of the present embodiment, the sports competition is “figure skating”.

  FIG. 1 is a block diagram showing the entire embodiment of the present invention. Referring to FIG. 1, the embodiment of the present invention includes a line-of-sight detection camera 11, a control device 12, and a competition shooting camera 13. The control device 12 operates by program control, and includes an image matching unit 14, a face image storage unit 15, a shooting target position estimation unit 16, and a competition shooting camera control unit 17.

  The line-of-sight detection camera 11 shoots a plurality of still persons who are gazing at the competition without moving during the competition, such as referees and recording staff, and sends the moving images to the image matching unit 14. It is fixed so that the line-of-sight detection target can always be photographed from the front. In the present embodiment, the line-of-sight detection target is the referee. FIG. 2 shows an example of the arrangement of cameras in the embodiment of the present invention. In FIG. 2, a plurality of referees 51 are concentrated in one place such as a referee booth, and all referees 51 are photographed from the front by one gaze detection camera 11. The case where the monitoring image 20 can be acquired is shown. When the plurality of referees 51 are arranged at positions away from each other, the plurality of line-of-sight detection cameras 11 are arranged so that the respective referees 51 can photograph from the front.

  The image collating unit 14 obtains the line-of-sight vector 31 of each referee 51 from the moving image captured by the line-of-sight detection camera 11, cuts out the face image 22 of each referee 51 from the frame 21 of the moving image, The face image model 25 of the referee 51 registered in the face image model table 23 of the face image storage unit 15 is collated, and a line-of-sight vector 31 corresponding to the recognized face image model 25 is determined as a result of the collation. .

  The face image accumulating unit 15 stores the front face image model 25 of the referee 51 in advance so that the image collating unit 14 can collate the face image 22 of the referee 51. A face image model table 23 stored in the face image storage unit 15 is shown in FIG. The face image model table 23 includes a plurality of predicted face directions 24, a face image model 25 corresponding to the face direction 24, and a line-of-sight vector 31, and is prepared for each judge 51 in advance. As the number of face directions 24 increases, more accurate collation can be performed. However, since the amount of data to be stored and the processing load for collation also increase, the amount of data that can be stored in the face image storage unit 15 and the control device 12 An appropriate value is selected according to the processing capability. In FIG. 3, the number of face directions 24 is nine.

  The shooting target position estimating unit 16 obtains a shooting target estimated position 36 (three-dimensional coordinates) to be shot by the competition shooting camera 13 from the line-of-sight vector 31 of each judge 51 determined by the image collating unit 14. . The shooting target position estimation unit 16 obtains the line of sight 32 of each judge 51 from the line-of-sight vector 31 of each judge 51 and sets the point where the lines of sight 32 concentrate as the shooting target estimated position 36. During the competition, each referee 51 is concentrated in the competition, and the competition is being performed at the point where the line of sight 32 of each referee 51 is concentrated, so that the shooting target 52 at the shooting target estimated position 36 can be shot. By controlling the direction and focus of the competition shooting camera 13, it is possible to shoot the event during the competition that the judges 51 are paying attention to.

  The competitive shooting camera control unit 17 controls the direction and focus of the competitive shooting camera 13 so that the shooting target 52 at the shooting target estimation position 36 (three-dimensional coordinates) obtained by the shooting target position estimation unit 16 can be shot. Is.

  The competition shooting camera 13 shoots a competition video for broadcast and recording of the competition, and is arranged at any place where the competition can be shot. The competitive shooting camera 13 changes the direction and focus in accordance with the shooting control information instructed by the competitive shooting camera control unit 17 and shoots while tracking the shooting target 52 existing at the shooting target estimated position 36.

  Next, the operation of the best mode for carrying out the present invention will be described with reference to the drawings.

  FIG. 4 shows a processing flow of the entire apparatus according to the embodiment of the present invention. The image collation unit 14 extracts the frame 21 from the moving image sent from the line-of-sight detection camera 11 with the start of the competition (S102). Next, the face image 22 of the referee 51 in the frame 21 and the face image model 25 of the referee 51 registered in the face image storage unit 15 are collated, and the face image model recognized as a result of the collation is confirmed. 25, the corresponding line-of-sight vector 31 is obtained and sent to the imaging target position estimation unit 16 (S103). Next, the shooting target position estimating unit 16 calculates the shooting target estimated position 36 (three-dimensional coordinates) from the plurality of line-of-sight vectors 31 obtained by the image collating unit 14 and sends the calculated shooting target estimated position 36 (S104). ). The competition shooting camera control unit 17 obtains shooting control information (horizontal and vertical control angles and distances) of the competition shooting camera 13 and sends them to the shooting camera 13 (S105). The competitive shooting camera 13 changes the direction and focus in accordance with the shooting control information instructed from the competitive shooting camera control unit 17, and takes a picture while tracking the shooting target 52 existing at the shooting target estimated position 36 (S106). Steps S102 to S106 are repeated until the competition ends (S107).

  Next, details of operations of the image collation unit 14, the shooting target position estimation unit 16, and the competition shooting camera control unit 17 will be described.

  FIG. 5 shows a processing flow of the image matching unit 14, and FIG. 6 shows a conceptual diagram of image matching. The image collating unit 14 extracts the frame 21 from the moving image sent from the line-of-sight detection camera 11 (S201, FIG. 6A), and extracts the face image 22 of the referee 51 from the image of the frame 21 by trimming (see FIG. 6). S202, FIG. 6 (b)). Here, when two or more face images 22 cannot be extracted, the processing of the frame 21 is stopped and the processing of the next frame 21 is started (S203). Next, the face image 22 of each referee 51 extracted by trimming is compared with the face image model 25 of each referee 51 registered in the face image storage unit 15 (S204, FIG. 6C). ) As a result of the collation, the line-of-sight vector 31 corresponding to the recognized face image model 25 is obtained (S205) and sent to the photographing target position estimating unit 16 (S206).

  For example, when four judges 51a, b, c, and d are arranged as shown in FIG. 2, the face image model table 23a, b, c, and d shown in FIG. Be prepared. When the face direction 24 of each judge 51a, b, c, d is recognized as the top, the top, the top left, and the top left as a result of the matching with the face image model 25 in the image matching unit 14, the line-of-sight vector of each judge 31a, b, c, and d are Va2, Vb2, Vc1, and Vd1, respectively.

  FIG. 7 shows a processing flow of the shooting target position estimation unit 16, and FIG. 9 shows a method of determining the shooting target estimation position 36 from the line-of-sight vector 31. The competition shooting camera control unit 17 draws a straight line from the face position 37 of the judge 51 in the direction of the line-of-sight vector 31 obtained in the image collation unit 14, and sets this as the line of sight 32 (S301). Next, all horizontal plane intersections 33 that are intersections when the respective lines of sight 32 are projected onto the xy plane are obtained (S302). Next, centroids on the xy plane with respect to all the obtained horizontal plane intersection points 33 (average values of the x-coordinates and y-coordinates of all the horizontal plane intersection points 33) are obtained, and this centroid is set as the horizontal direction estimated position 34 of the imaging target 52. (S303). Further, all the heights that the line of sight 32 can take on the horizontal plane intersection point 33 (the value of the z coordinate of the line-of-sight height calculation point 38 that is the intersection of the perpendicular line and the line of sight 31 on the horizontal plane intersection point 33) are obtained, and this average value is taken. The height direction estimated position 35 of the object 52 is set (S304). A point specified from the horizontal direction estimated position 34 and the height direction estimated position 35 is set as the shooting target estimated position 36 (S305), and is sent to the competition shooting camera control unit 17 (S306).

  For example, in FIG. 9, a straight line is drawn in the direction of the line-of-sight vector 31a, b, c, d from the face positions 37a, b, c, d of each judge 51a, b, c, d, and these lines of sight 32a, b , C, d. When these lines of sight 32a, b, c, d are projected onto the xy plane, four horizontal plane intersection points 33a (xac, yac), 33b (xbc, ybc), 33c (xad, yad), 33d (xbd, ybd) are obtained. can get. The average value of the x-coordinate and y-coordinate of these four horizontal plane intersections 33a, b, c, and d is the horizontal direction estimated position 34 (xe, ye) of the imaging target 52. In addition, there are eight line-of-sight height calculation points 38a to 38h as intersections of perpendicular lines on all horizontal plane intersections 33a, b, c, and d and lines of sight 32a, b, c, d, and these line-of-sight height calculation points. The average value of the z coordinates of 38a to h is the height direction estimated position 35ze. As a result, an imaging target estimated position 36 (xe, ye, ze) is obtained.

  FIG. 8 shows a processing flow of the camera control unit 17 for competition shooting. The competitive shooting camera control unit 17 obtains shooting control information (control angles and distances in the horizontal direction and vertical direction) of the competitive shooting camera 13 from the positional relationship between the competitive shooting camera position 39 and the shooting target estimation position 36 ( (S401), and sent to the competition shooting camera 13 (S402).

  For example, in FIG. 9, the shooting target estimated position 36 is a horizontal and vertical control angle α when (xe, ye, ze) and a competition shooting camera position 39 (xv, yv, zv) are connected by a straight line. , Β, and distance L are sent to the competition shooting camera 13 as shooting control information.

  Thus, the operation of the first exemplary embodiment of the present invention is completed.

  According to the first embodiment of the present invention, in a camera automatic control device that tracks a shooting target, it is possible to reliably track an event that is noticed in a sporting event or the like. The reason is that for detecting the position of the shooting target 52, a person who is constantly concentrated in the competition, such as a referee or a recording staff, is shot with the camera 11 for line-of-sight detection as a line-of-sight detection target, and the face images of these line-of-sight detection targets Are compared with the face image model 25 prepared in advance, and the shooting target estimated position 36 is calculated from the line of sight 32 corresponding to the recognized face image model 25, so that the shooting target 52 is a specific person or object. This is because an event that is not fixed and focused by the line-of-sight detection target during the competition can be set as the shooting target 52.

  Further, according to the first embodiment of the present invention, the number of cameras necessary for detecting the position of the shooting target 52 can be reduced. The reason for this is that the gaze detection camera 11 is set as the gaze detection target 11 so that a stationary person who does not move during the competition, such as a referee or a record clerk, is taken as a gaze detection target for the position detection. This is because the line-of-sight detection camera 11 can always capture the line-of-sight detection target by appropriately performing the above, and it is not necessary to photograph the same line-of-sight detection target from a plurality of directions.

  In addition, according to the first embodiment of the present invention, the tracking process of the photographing target 52 becomes stable. The reason for this is that, as described above, a still person who does not move during the competition is photographed by the gaze detection camera 11 as a gaze detection target. Therefore, in the face image collation process, the gaze detection target to be collated is This is because there is no significant movement between the moving image frames 21.

(Another embodiment of the invention)
Next, a second embodiment of the present invention will be described in detail with reference to the drawings.

  In the second embodiment of the present invention, a first embodiment is provided in that a plurality of competition shooting cameras 13 are arranged and a competition video switching device 18 for selecting a video output of the competition shooting camera 13 is provided. And different.

  FIG. 10 shows a block diagram of the second embodiment of the present invention, and FIG. 11 shows an arrangement example of the game shooting camera 13. Referring to FIGS. 10 and 11, in the second embodiment of the present invention, a plurality of competition shooting cameras 13 are provided, and the shooting targets 52 are shot at equal intervals so that shooting can be performed from different viewpoints. To place. The competition video switching device 18 selects the video output of the competition shooting camera 13 and outputs video from each competition shooting camera 13 in accordance with the competition video switching instruction input 42 input by the operator. Is output to the competition video output 41.

  Next, the operation of the second exemplary embodiment of the present invention will be described. In the second embodiment of the present invention, the procedure for obtaining the photographing target estimated position 36 from the line of sight 32 obtained by the face image comparison of the line of sight detection target is the same as that of the first embodiment. The competitive shooting camera control unit 17 determines the shooting control information of each competitive shooting camera 13 based on the positional relationship between the installation position 39 of each competitive shooting camera 13 and the shooting target estimation position 36 for all the shooting cameras 13. The control angles and distances in the horizontal and vertical directions are determined and sent to each competition shooting camera 13. All the competition shooting cameras 13 track and shoot the shooting target 52 existing at the same shooting target estimation position 36 from different viewpoints. In the competition video switching device 18, when the operator instructs the competition shooting camera 13 to output a video by the competition video switching instruction input 42, the competition video switching camera 13 and the operator are instructed at that time. The competition shooting cameras 13 between the competition shooting cameras 13 are sequentially selected to be the competition video output 41.

  For example, in FIG. 11, all the competition shooting cameras 13 a to 13 p track and shoot the shooting target 52 existing at the shooting target estimated position 36 (xe, ye, ze). When switching to the competitive shooting camera 13f is instructed as the competitive video switching instruction input 42 while the competitive shooting camera 13a is selected, the competitive video switching device 18 causes the competitive shooting camera 13a → 13b → 13c → 13d → The competition shooting cameras 13 are sequentially selected in the order of 13e → 13f, and the output video is set as the competition video output 41.

  According to the second embodiment of the present invention, it is possible to smoothly switch the viewpoint of the video in the automatic camera control apparatus that tracks the shooting target. The reason is that a plurality of competition shooting cameras 13 are arranged so that the same shooting target 52 can be shot from different viewpoints, and all the shooting shooting cameras 13 are controlled to shot the same shooting target 52. This is because when the video is switched between the competitive shooting cameras 13 of the viewpoint, the competitive video switching device 18 also sequentially selects and outputs the video output of the competitive shooting camera 13 between the two competitive shooting cameras 13.

  Next, a third embodiment of the present invention will be described in detail with reference to the drawings.

  In the third embodiment of the present invention, the competition shooting camera 13 is mounted on the moving camera device 61, and the position of the competition shooting camera 13 moves while tracking the shooting target 52 as the shooting target 52 moves. In the point which it did, it differs from 1st Embodiment and 2nd Embodiment.

  FIG. 12 is a block diagram of the third embodiment of the present invention, and FIG. Referring to FIGS. 12 and 13, the moving camera device 61 moves the competition shooting camera 13 to any position in the upper space of the competition area 65 (for example, basket court, figure skating link). The imaging camera 13, the movement control unit 62, and the camera position movement unit 63 are configured.

  The movement control unit 62 controls the camera position movement unit 63 so that the competition shooting camera 13 moves at a constant speed toward the movement target position 71 according to the movement control information instructed by the competition shooting camera control unit 17. To do. In addition, the competition shooting camera position 39 is transmitted to the competition shooting camera control unit 17. The camera position moving unit 63 is a mechanism for changing the position of the competition shooting camera 13 and has an arm structure in which one end is fixed to a ceiling or a wall and has a plurality of movable units 64. The mechanism of the camera position moving unit 63 may be another mechanism such as a crane that moves on a rail as long as the position of the competition shooting camera 13 can be arbitrarily changed.

  The competition shooting camera control unit 17 obtains shooting control information (horizontal and vertical control angles and distances of the competition shooting camera 13) and controls the shooting direction of the competition shooting camera 13. The movement control information including the movement target position 71 is obtained from the shooting target estimated position 36 and sent to the movement control unit 62, thereby controlling the position of the competition shooting camera 13.

  Next, the operation of the third exemplary embodiment of the present invention will be described with reference to the drawings.

  FIG. 14 shows a processing flow of the entire apparatus according to the third embodiment of the present invention, and FIG. 15 shows a processing flow of the game shooting camera control unit 17. In FIG. 14, the procedure from the moving image sent from the line-of-sight detection camera 11 until the shooting target position estimation unit 16 obtains the shooting target estimation position 36 (S501 to S504) is the first embodiment of the present invention. It is the same as the form (FIG. 4, S101 to S104). The competitive shooting camera control unit 17 obtains the competitive shooting camera position 39 from the movement control unit 62 (S505), and the shooting control information of the competitive shooting camera 13 from the shooting position of the competitive shooting camera 39 and the shooting target estimation position 36 ( The control angles and distances in the horizontal and vertical directions are obtained and sent to the competition shooting camera 13 (S506, S601, S602). The competitive shooting camera 13 changes the direction and focus according to the shooting control information instructed from the competitive shooting camera control unit 17, and takes the shooting target 52 existing at the shooting target estimated position 36 (S507).

  Next, the game shooting camera control unit 17 obtains the movement target position 71 (xm, ym, zm) to which the game shooting camera 13 should move from the shooting target estimation position 36, and moves the movement control unit 62 as movement control information. (S508, S603, S604). The movement target position 71 is determined in accordance with the target competition so that the competition shooting camera 13 can shoot the shooting target 52 existing at the shooting target estimation position 36 from an appropriate viewpoint (direction, distance).

  For example, in the competition shown in FIG. 13, when it is required to always view the shooting target 52 at a certain height j from directly above, (xe, ye, j) on the perpendicular to the horizontal plane of the shooting target estimated position 36 is It becomes the movement target position 71.

  In accordance with the movement control information instructed by the game shooting camera control unit 17, the movement control unit 62 moves the game shooting camera 13 toward the movement target position 71 (xm, ym, zm) at a constant speed. The camera position moving unit 63 is controlled (S509).

  Thus, the operation of the third embodiment of the present invention is completed.

  According to the third embodiment of the present invention, in an automatic camera control apparatus that tracks a shooting target, the shooting target can be shot while the shooting distance and direction with respect to the shooting target are kept constant. The reason is that the moving camera device 61 can move the competition shooting camera 13 to an arbitrary position, and the shooting camera 52 captures the shooting target 52 from the viewpoint required in the competition from the shooting target estimated position 36. This is because the movement target position 71 is obtained, and the competition shooting camera 13 is controlled to move to the movement target position 71.

  INDUSTRIAL APPLICABILITY The present invention can be used for automatic control of a camera in shooting a sporting event or shooting a stage such as a play.

It is a block diagram which shows the whole 1st Embodiment of this invention. FIG. 2 is a layout view of cameras in the first embodiment of the present invention. It is a face image model stored in the face image storage unit. It is a flowchart which shows the whole operation | movement of the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of an image collation part. It is a conceptual diagram which shows the method of calculating | requiring a gaze vector in an image collation part. It is a flowchart which shows operation | movement of a imaging | photography target position estimation part. It is a flowchart which shows operation | movement of the camera control part for competition imaging | photography. It is explanatory drawing of the method of determining a to-be-photographed object estimated position from a gaze vector. It is a block diagram which shows the whole 2nd Embodiment of this invention. It is a layout drawing of the camera for competition photography in the 2nd embodiment of the present invention. It is a block diagram which shows the whole 3rd Embodiment of this invention. It is a layout view of the mobile camera device in the third embodiment of the present invention. It is a flowchart which shows the whole operation | movement of the 3rd Embodiment of this invention. It is a flowchart which shows operation | movement of the camera control part for competition imaging | photography in the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Gaze detection camera 12 Control device 13 Competition shooting camera 14 Image collation unit 15 Face image storage unit 16 Shooting target position estimation unit 17 Competition shooting camera control unit 18 Competition video switching device 20 Referee monitoring image 21 Frame 22 Face image 23 face image model table 24 face direction 25 face image model 31 line of sight vector 32 line of sight 33 horizontal plane intersection 34 horizontal direction estimated position 35 height direction estimated position 36 photographing target estimated position 37 face position 38 line of sight height calculation point 39 camera for competition shooting Position 41 Competition video output 42 Competition video switching instruction input 51 Referee 52 Shooting target 61 Mobile camera device 62 Movement control unit 63 Camera position movement unit 64 Movable unit 65 Competition area 71 Target position for movement

Claims (24)

A line-of-sight detection camera having line-of-sight detection target photographing means for photographing a line-of-sight detection target;
Face image storage means for storing a face image model table in which each face image obtained by photographing the line-of-sight detection target from a plurality of directions and the line-of-sight vector of each face image are stored, and the image taken by the line-of-sight detection target photographing means Image collating means for extracting a line-of-sight vector corresponding to the face image of the line-of-sight detection target from the face image model table, shooting target position estimating means for obtaining a shooting target estimated position from the line-of-sight vector extracted by the image matching means, A camera automatic control system comprising: a camera automatic control device having a camera control unit for competition shooting for obtaining shooting control information for shooting a shooting target at a shooting target estimated position. The camera automatic control system according to claim 1, wherein the photographing target position estimating unit obtains the photographing target estimated position from a plurality of line-of-sight vectors of face images of the line-of-sight detection target. 3. A competitive shooting camera comprising shooting target tracking means for receiving the shooting control information from the competitive shooting camera control means and shooting the shooting target according to the shooting control information. The camera automatic control system as described in. When switching the output video so as to select one of the output videos of the plurality of competition shooting cameras arranged at a predetermined position with respect to the shooting target, switching to the competition shooting camera before switching is performed. When there is another game shooting camera between the later game shooting cameras, there is output video switching means for sequentially switching the output video of each game shooting camera from the side of the game shooting camera before switching. 4. The automatic camera control system according to claim 3, further comprising a competition video switching device. The game shooting camera control means obtains movement control information for moving the game shooting camera to a movement target position to which the game shooting camera should move based on the shooting object estimated position. The camera automatic control system according to claim 3. A moving camera device having camera position moving means for moving the game shooting camera and movement control means for moving the game shooting camera by controlling the camera position moving means according to the movement control information; The camera automatic control system according to claim 5, wherein: The photographing target position estimating means obtains a line of sight from the line-of-sight vector, sets the center of gravity of a plurality of horizontal plane intersections formed when a plurality of the lines of sight are developed on a horizontal plane as a horizontal direction estimated position of the shooting target estimated position, The average value of vertical heights of a plurality of intersections formed by perpendicular lines on a plurality of horizontal plane intersections and the line of sight is set as a height direction estimation position of the imaging target estimation position. The camera automatic control system according to any one of the above. The camera automatic control device corresponds to the line-of-sight vector corresponding to the face image of the line-of-sight detection imaged by the line-of-sight detection camera, the face image obtained by photographing the line-of-sight detection object from a plurality of directions, and the line-of-sight vector of each face image An image collating step extracted from the attached face image model table, a photographing target position estimating step for obtaining a photographing target estimated position from the line-of-sight vector extracted in the image collating step, and photographing a photographing target at the photographing target estimated position And a camera control step for competition shooting for obtaining shooting control information for the automatic camera control method. 9. The camera automatic control method according to claim 8, wherein the photographing target position estimating step obtains the photographing target estimated position from a plurality of gaze vectors of face images of the gaze detection target. 9. The competitive shooting camera includes a shooting target tracking step of receiving the shooting control information obtained in the competitive shooting camera control step and shooting the shooting target according to the shooting control information. Or the camera automatic control method of 9. When the competition video switching device switches the output video so as to select one of the output videos of the plurality of competition shooting cameras arranged at a predetermined position with respect to the shooting target, When there is another competitive shooting camera between the competitive shooting camera and the switched competitive shooting camera, the output video of each competitive shooting camera is sequentially switched from the previous competitive shooting camera side. 11. The camera automatic control method according to claim 10, further comprising an output video switching step. In the game shooting camera control step, movement control information for moving the game shooting camera to a movement target position to which the game shooting camera should move based on the shooting target estimation position is obtained. The camera automatic control method according to claim 10. 13. The camera automatic control method according to claim 12, further comprising a movement control step in which the moving camera device moves the game shooting camera according to the movement control information. The shooting target position estimation step obtains a line of sight from the line-of-sight vector, sets the center of gravity of a plurality of horizontal plane intersections formed when a plurality of the lines of sight are developed on a horizontal plane as the horizontal direction estimated position of the shooting target estimated position, 14. An average value of vertical heights of a plurality of intersections formed by perpendicular lines on a plurality of horizontal plane intersections and the line of sight is defined as a height direction estimation position of the imaging target estimation position. The camera automatic control method according to any one of the above. A face image model table in which a line-of-sight vector corresponding to a face image to be detected by the line-of-sight detection camera is associated with each of the face images obtained by shooting the line-of-sight detection target from a plurality of directions. Image collating means extracted from the image capturing apparatus, photographing object position estimating means for obtaining a photographing object estimated position from the line-of-sight vector extracted by the image collating means, and photographing control information for photographing the photographing object at the photographing object estimated position. An automatic camera control apparatus comprising: a camera control means for competition photography to be obtained. 16. The camera automatic control apparatus according to claim 15, wherein the photographing target position estimating means obtains the photographing target estimated position from a plurality of line-of-sight vectors of face images of the line-of-sight detection target. The camera automatic control device according to claim 15 or 16, wherein the shooting control information is transmitted to a competition shooting camera, and the shooting target is shot according to the shooting control information. The game shooting camera control means obtains movement control information for moving the game shooting camera to a movement target position to which the game shooting camera should move based on the shooting object estimated position. The camera automatic control device according to claim 17. The photographing target position estimating means obtains a line of sight from the line-of-sight vector, sets the center of gravity of a plurality of horizontal plane intersections formed when a plurality of the lines of sight are developed on a horizontal plane as a horizontal direction estimated position of the shooting target estimated position, 19. An average value of vertical heights of a plurality of intersections formed by perpendiculars on a plurality of horizontal plane intersections and the line of sight is set as a height direction estimation position of the imaging target estimation position. The camera automatic control apparatus in any one of. A face image model table in which a line-of-sight vector corresponding to a face image to be detected by the line-of-sight detection camera is associated with each of the face images obtained by capturing the line-of-sight detection target from a plurality of directions. An image matching process extracted from the image, a shooting target position estimation process for obtaining a shooting target estimated position from the line-of-sight vector extracted in the image matching process, and shooting control information for shooting a shooting target at the shooting target estimated position. A program for causing a camera automatic control device to perform a desired camera control process for competition shooting. 21. The program according to claim 20, wherein the photographing target position estimating process obtains the photographing target estimated position from a plurality of line-of-sight vectors of face images of the line-of-sight detection target. The program according to claim 20 or 21, wherein the shooting control information is transmitted to a competition shooting camera and the shooting target is shot according to the shooting control information. The game shooting camera control process is characterized in that movement control information for moving the game shooting camera to a movement target position to which the game shooting camera should move is obtained based on the shooting object estimated position. The program according to claim 22. The shooting target position estimation process obtains a line of sight from the line-of-sight vector, and sets the center of gravity of a plurality of horizontal plane intersections formed when a plurality of the lines of sight are developed on a horizontal plane as the horizontal direction estimated position of the shooting target estimated position, 24. An average value of vertical heights of a plurality of intersections formed by perpendicular lines on a plurality of horizontal plane intersections and the line of sight is set as a height direction estimation position of the imaging target estimation position. A program according to any of the above.

Images