JP7276538B2 - VIEWING ASSISTANCE DEVICE, VIEWING ASSISTANCE METHOD, AND PROGRAM - Google Patents

Description

(Description of Related Application)
The present invention is based on the priority claim of Japanese Patent Application: Japanese Patent Application No. 2018-045582 (filed on March 13, 2018), and the entire description of the application is incorporated herein by reference. shall be
The present invention relates to a viewing support device, a viewing support method, and a program.

Patent Literature 1 discloses an example of a moving object identification device that associates a plurality of moving objects from moving images of the moving objects. Further, Patent Literature 2 discloses a configuration including a portable information recognition device that recognizes identification information of a mobile object obtained by photographing the mobile object and searches for the identification information registered in advance. Furthermore, on page 13, line 8 and below of Patent Document 2, in track and field events, imaging devices are arranged at various places on the track, the bib number is read from the still image of the athlete, and the athlete's split is sent to general households along with the video. A configuration for distributing time is disclosed.

In addition, Non-Patent Document 1 discloses a system for estimating the distance between runners in a relay road race such as a long-distance relay race or a marathon. In addition, the distance estimation system in the same document estimates the distance between the camera and the runner from the position of the broadcast van, the camera data, and the coordinates of the runner in the video (the position of the subject in the frame), and from the change in the runner's position between frames It is stated that the speed of the runner can be determined (see FIG. 1).

JP 2018-005474 A WO2003/034278

"Road Race Relay Distance Estimation System", Kenrokukan Publishing, "Broadcasting Technology", Vol.69, May, 2016, pp.101-105

The following analysis is given by the present invention. The technique described in the above background art has a problem that it is not possible to estimate the player's speed at an arbitrary point in time.

In the method of Non-Patent Document 1, the speed of the runner can be estimated at any point in time, but the condition is that the position of the broadcast van and camera data can be obtained in real time.

The present invention provides a viewing support device, a viewing support method, and a viewing support device capable of measuring the moving object and its speed displayed in the relay image even if the above-described information cannot be obtained, and contributing to the enrichment of the configuration that can be provided as supplementary information of the image. The purpose is to provide a program.

According to the first viewpoint, a means is provided for cutting out, at predetermined time intervals, an image in which the moving body is projected from a relay video in which the moving body is moving from a first point to a second point. A viewing support device is provided. This viewing support device further comprises means for estimating the speed of the moving object based on changes in the background image of the image showing the moving object, and means for outputting the estimated speed of the moving object. .

According to the second viewpoint, an image in which the plurality of moving bodies appearing is displayed at predetermined time intervals from a relay video in which a plurality of moving bodies moving from a first point to a second point are captured. means for estimating the distance difference of the moving object based on a change in the background image of the image showing the plurality of moving objects; and means for outputting the distance difference between the plurality of moving objects. A viewing support device is provided.

According to the third viewpoint, a step of cutting out an image showing the moving object at predetermined time intervals from a relay video in which the object is the moving object moving from a first point to a second point; A viewing support method is provided that includes the steps of: estimating the speed of the moving object based on changes in the background image of the image showing the moving object; and outputting the estimated speed of the moving object. The method is tied to a specific machine, a computer that takes at least the relay video as input, performs image analysis and outputs additional information.

According to the fourth aspect, the computer constituting the viewing support device receives, at predetermined time intervals, the moving object moving from the first point to the second point from the relay video in which the object is the moving object. A process of cutting out the projected image, a process of estimating the speed of the moving object based on a change in the background image of the image showing the moving object, and a process of estimating the moving object based on the image of the moving object shown in the image. Then, a program for executing a process of identifying the moving body and a process of outputting information on the identified moving body and its estimated speed is provided. This program can be recorded in a computer-readable (non-transitory) storage medium. That is, the present invention can also be embodied as a computer program product. Also, this program is input to the computer device via an input device or an external communication interface, is stored in a storage device, drives the processor according to predetermined steps or processes, and outputs the processing results including intermediate states as necessary. can be displayed step by step via a display device, or can be communicated with the outside via a communication interface. A computer device for this purpose typically includes, as an example, a processor, a storage device, an input device, a communication interface, and optionally a display device, all of which are connectable to each other via a bus.

According to the present invention, it is possible to estimate the speed of a mobile object at any point in time. That is, the present invention converts the viewing support device described in the background art into one capable of providing speed information of a moving object that is a subject.

It is a figure which shows the structure of one Embodiment of this invention. BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the whole structure of the 1st Embodiment of this invention. 1 is a diagram showing the configuration of a viewing support device according to a first embodiment of the present invention; FIG. 4 is a diagram showing an example of information held in a runner information database of the viewing support device according to the first embodiment of the present invention; FIG. It is a figure which shows an example of the provision form of information by the viewing support apparatus of the 1st Embodiment of this invention. FIG. 5 is a diagram showing another example of the form of providing information by the viewing support device according to the first embodiment of the present invention; FIG. 5 is a diagram showing the configuration of a viewing support device according to a second embodiment of the present invention; It is a figure which shows an example of the provision form of information by the viewing support apparatus of the 2nd Embodiment of this invention. FIG. 11 is a diagram showing the configuration of a viewing support device according to a third embodiment of the present invention; FIG. 12 is a diagram showing an example of information held in a reference background image database of the viewing support device according to the third embodiment of the present invention; FIG. 11 is a diagram schematically showing a relay image cut out by the viewing support device according to the third embodiment of the present invention; FIG. 11 is a diagram for explaining the principle of runner position identification by the viewing support device according to the third embodiment of the present invention; It is a figure which shows the structure of the computer which comprises the viewing assistance apparatus of this invention. FIG. 10 illustrates a modified configuration of one embodiment of the present invention; It is a figure which shows the deformation|transformation structure of the viewing assistance apparatus of the 1st Embodiment of this invention. FIG. 5 is a diagram showing another example of the form of providing information by the viewing support device according to the first embodiment of the present invention; FIG. 5 is a diagram showing another example of the form of providing information by the viewing support device according to the first embodiment of the present invention;

First, an outline of an embodiment of the present invention will be described with reference to the drawings. It should be noted that the drawing reference numerals added to this overview are added to each element for convenience as an example to aid understanding, and are not intended to limit the present invention to the illustrated embodiments. Also, connection lines between blocks in drawings and the like referred to in the following description include both bidirectional and unidirectional connections. The unidirectional arrows schematically show the flow of main signals (data) and do not exclude bidirectionality. Also, although there are ports or interfaces at input/output connection points of each block in the figure, they are omitted from the drawing. Also, the program is executed via a computer device, and the computer device includes, for example, a processor, a storage device, an input device, a communication interface, and, if necessary, a display device. In addition, the computer device is configured to be able to communicate with internal or external devices (including computers) via a communication interface, whether wired or wireless.

In one embodiment of the present invention, as shown in FIG. 1, a viewing support device 10 having an image clipping section 11, a speed estimating section 12, and an output section 14 can be implemented.

More specifically, the image clipping unit 11 extracts, at predetermined time intervals, an image showing the moving object from a relay video in which the object is the moving object moving from a first point to a second point. It functions as a means of cutting out.

The speed estimator 12 functions as means for estimating the speed of the moving object based on changes in the background image of the image showing the moving object.

The output unit 14 functions as means for outputting the estimated speed of the moving object. Incidentally, according to the output unit 14, various forms are conceivable as the output form of the estimated speed of the moving body. For example, the estimated speed of the moving object may be displayed as information superimposed on the relay image. Further, for example, the estimated speed of the moving object may be displayed as information displayed in a sub-region different from the relay image. Also, the estimated speed of the moving object may be displayed as information displayed by a website or application program for knowing the contents of the relay video in more detail.

As described above, according to the present embodiment, it is possible to measure a moving object displayed in a relay image and its speed, and to provide them as incidental information of the image.

Further, in another embodiment, the present invention can be configured as a viewing support device that estimates and outputs distance differences between a plurality of moving objects instead of or in combination with the estimated speeds. (See FIG. 14). In the example of FIG. 14, a distance difference estimator 18 is provided instead of the speed estimator 12 described above. Then, in this embodiment, the image clipping unit 11 extracts the plurality of moving bodies moving from the first point to the second point at predetermined time intervals from the relay video in which the subjects are the moving bodies. is processed to cut out the projected image. Then, the distance difference estimating unit 18 estimates the distance difference between the moving objects based on the change in the background image of the image showing the plurality of moving objects. Then, the output unit 14 outputs the distance difference between the plurality of moving objects.

As described above, according to this embodiment, it is possible to measure the distance difference between a plurality of moving objects displayed in a relay image and provide it as supplementary information of the image.

[First embodiment]
Next, a first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 2 is a diagram showing the overall configuration of the first embodiment of the present invention. Referring to FIG. 2, a configuration in which a camera 200 mounted on a broadcast vehicle that relays a road race or the like, a viewing support device 100, and a television receiver 300 serving as an information output destination of the viewing support device 100 are connected. It is shown.

The camera 200 transmits the captured relay image to the viewing support device 100 . Transfer of data between the camera 200 and the viewing support device 100 may be performed by a wired method, a wireless method, or a physical medium transfer method such as a magnetic tape.

The viewing support device 100 is arranged in a broadcasting station, a video distribution company, or the like in one desirable form. Note that the place where the viewing support device 100 is arranged is not limited to a broadcasting station, a video distribution company, or the like, and may be arranged, for example, in a relay car or on the side of a television receiver. Also, the viewing support device 100 may be arranged on the Internet or on a cloud platform.

The television receiver 300 serves as an output destination of the information obtained by the viewing support device 100 in addition to the relay image. Note that the output destination of the information obtained by the viewing support device 100 may be any device capable of displaying the relay image and its incidental information, and a device other than the television receiver 300 may be used. For example, a PC (personal computer), a tablet terminal, a smart phone, or other device capable of reproducing moving images via the Internet can be used instead of the television receiver 300 .

FIG. 3 is a diagram showing the configuration of the viewing support device according to the first embodiment of the present invention. Referring to FIG. 3, a configuration including an image cropping unit 110, a speed estimating unit 120, a moving body identifying unit 130, an output unit 140, and a runner information DB (database) 150 is shown.

The image clipping unit 110 clips an image for identifying the position of the runner from the relay video, and performs processing for sending the image to the speed estimating unit 120 and the moving object identifying unit 130 . In this embodiment, the image clipping unit 110 clips the image at the timing when the foot of the runner, who is the subject in the relay video, touches the ground. The reason why the image is cut out at the timing when the runner's foot touches the ground is that the positional relationship between the runner and the background image can be appropriately specified, and the speed estimation accuracy is improved. In the present embodiment, the expression "cut out an image" is used, but in practice, a frame in which the runner's feet are on the ground may be selected from the frames forming the relay video.

The moving object identification unit 130 refers to the information stored in the runner information DB 150 , identifies the runner from the face image and bib of the runner who is the subject of the relay video, and outputs the result to the output unit 140 . When a face image is used to identify the runner, a method of using AI (Artificial Intelligence) to identify a runner having a feature amount similar to a pre-registered face image can be used.

FIG. 4 is a diagram showing an example of information held in the runner information DB 150. As shown in FIG. In the example of FIG. 4, runner information is stored in which bibs and athlete names are associated with each other. The moving object identifying unit 130 performs character recognition on the bib from the image sent from the image clipping unit 110 , reads the runner name corresponding to the bib, and sends it to the output unit 140 . In the example of FIG. 4, the information for specifying the runner by the bib is shown. The image is registered. In addition to the example shown in FIG. 4, for example, profile information such as the team to which each runner belongs, age, past winning history, etc. may be registered in the runner information DB 150 .

Speed estimating section 120 estimates the speed of the runner from changes in the background image of the runner, who is the subject of the relay video, and outputs the result to output section 140 .

An example of the speed estimation method is as follows. The speed estimating unit 120 uses the difference information of the continuous frames extracted by the image extracting unit 110 to determine the movement amount of the camera 200 and the runner, the positions of the horizon and the ground surface included in the background image in the image, and the positions of the runner's feet. By estimating the ground contact position, the relative distance between the camera 200 and the runner is estimated. In addition, speed estimating section 120 estimates the movement distance of camera 200 (relay van) by a similar method. Next, the speed estimator 120 calculates the relative speed of the runner (the speed as seen from the camera 200) from the frame rate of the frame cut out by the image cutout unit 110 and the change in the relative distance. In addition, speed estimating section 120 estimates the moving speed of camera 200 (relay van) by a similar method. Finally, the speed estimator 120 calculates the moving speed of the runner by adding the relative speed of the runner and the moving speed of the camera 200 (relay van).

The output unit 140 associates the runner information received from the moving body identifying unit 130 with the estimated speed of the runner received from the speed estimating unit 120 and provides the information to the viewer. FIG. 5 shows an example of how information is provided by the output unit 140. In the example of FIG. 5, runner information (athlete name) and estimated speed are displayed in a form superimposed on the relay video. As a result, viewers can obtain information such as not only how fast the runner is currently running, but also whether the pace is slowing down or, conversely, whether the pitch is increasing.

Note that the output form of the runner information and the estimated speed is not limited to the example shown in FIG. For example, as shown in boxes 400a and 400b in FIG. 6, it is possible to specify runner information for each of the multiple runners displayed in the image, and obtain and display their estimated speeds. This allows the viewer to grasp the difference in speed of each runner. Further, for example, it is possible to employ a mode in which the list of the specified runners and the estimated speed of each runner are collectively displayed in a sub-area different from the display area of the relay image.

In the above-described embodiment, the clipped image is sent from the image clipping unit 110 to the moving body identifying unit 130, but the timing of sending the image to the moving body identifying unit 130 is not limited to this. For example, the moving object identification unit 130 may acquire images at appropriate timings such as scene changes, and identify the runner from relay video or frame images thereof independently of speed estimation. This is because once the information on the runner is specified, there is basically no need to change it thereafter.

For example, even in comparison with Patent Literature 1, the present invention has the advantage that it is not necessary for a player or the like, who is a mobile body, to wear a measuring device such as an acceleration sensor. Also, in comparison with Patent Document 2, the present invention has the advantage that it is not necessary to dispose imaging devices at various locations on a predetermined course.

Also, the viewing support device of the present embodiment can be added with a function of displaying the distance difference between a plurality of moving objects, that is, the distance difference between a plurality of runners. FIG. 15 is a diagram showing a modified configuration of the viewing support device 100c according to the first embodiment of the present invention. A difference from the configuration of FIG. 3 is that a distance difference estimating section 180 is added between the speed estimating section 120 and the output section 140 .

The distance difference estimator 180 estimates the distance difference between a plurality of runners using the position information of the runners calculated in the speed estimation process of the speed estimator 120 . Distance difference estimating section 180 then outputs the estimated distance differences between the plurality of runners to output section 140 . The output unit 140 then provides the viewer with the distance difference between the runners in addition to the estimated speed of the runners.

According to the modified configuration shown in FIG. 15, bidirectional arrows between runners and the estimated distance difference can be displayed as shown in FIG. This allows the viewer to refer to the distance difference and speed difference between the multiple runners and make predictions such as "I will catch up with you in how many minutes" or "I will not be able to catch up with you at this pace."

In a more desirable form, when past record times are recorded, it is also possible to display a virtual runner who achieved that record time using AR (Augmented Reality) technology from the record time. can. The runner represented by the dashed line in front of number 11 in FIG. 17 is the image of the virtual record holder displayed using AR technology. In the example of FIG. 17, the distance difference between this virtual record time holder and the actual runner is displayed. This allows the viewer to refer to the respective distance differences and speed differences and make predictions such as "I am likely to break the record time" or "I cannot break the record time at this pace".

[Second embodiment]
Next, a second embodiment of the present invention, in which a function of judging a runner's facial expression and displaying it as supplementary information is added, will be described in detail with reference to the drawings. FIG. 7 is a diagram showing the configuration of a viewing support device according to the second embodiment of the present invention. The difference from the viewing support device 100 of the first embodiment shown in FIG. Since other configurations and operations are the same as those of the first embodiment, differences will be mainly described below.

Facial expression determination unit 170 receives the relay image from image clipping unit 110 via moving object identification unit 130 , identifies the face image of the runner from the image, determines the facial expression, and outputs the result to output unit 140 . do. Various methods can be adopted as the facial expression determination method in the facial expression determination unit 170 . For example, it is possible to adopt a method of judging the facial expression using changes and movement amounts of each organ of the runner's face. The facial expression may be determined not only from the facial expression but also from the runner's gesture. For example, even if the face of the runner itself is expressionless, if the face continues to sway from side to side, it may be determined that the runner has a "painful expression." Alternatively, teacher data for each facial expression may be registered in advance, and the facial expression may be determined by comparing it with the runner's facial expression.

The output unit 140 provides the viewer with the runner information received from the moving body identification unit 130 , the estimated speed of the runner received from the speed estimation unit 120 , and the runner's facial expression received from the facial expression determination unit 170 . FIG. 8 is an example of a form of providing information by the output unit 140. As shown in FIG. A difference from the first embodiment shown in FIG. 6 is that icons indicating the facial expressions of the runners are added in the boxes 400a and 400b. This allows the viewer to guess the condition of each runner from the facial expressions. For example, in the case of FIG. 8, the facial expression of runner A on the front side of the screen can be grasped from the relay video, but the facial expression of runner B on the back side of the screen is small or the camera is out of focus. For some reason, it may not be possible to directly grasp the situation from the relay video. According to this embodiment, even in such a case, it is possible to convey the expression of the runner to the viewer.

As described above, according to the present embodiment, it is possible to convey the runner's facial expression to the viewer in addition to the runner's information and estimated speed.

[Third embodiment]
Next, a third embodiment of the present invention in which the method of estimating the runner's speed is modified will be described in detail with reference to the drawings. FIG. 9 is a diagram showing the configuration of a viewing support device 100b according to the third embodiment of the present invention. The difference from the viewing support device 100 of the first embodiment shown in FIG. 3 is that the speed estimation unit 120 is replaced with a speed estimation unit 120b and a reference background image database (reference background image DB) 160 is added. is. Since other configurations and operations are the same as those of the first embodiment, differences will be mainly described below.

The speed estimation unit 120b of the present embodiment refers to the information stored in the reference background image DB 160, estimates the speed of the runner from changes in the background image of the runner who is the subject of the relay video, and outputs the result to the output unit 140. output to

FIG. 10 is a diagram showing an example of information held in the reference background image DB 160. As shown in FIG. In the example of FIG. 10, information is stored that associates a reference background image for specifying the position of the runner with a point (distance from the starting point) corresponding to the image. Upon receiving the image (for example, FIG. 11) sent from the image clipping unit 110, the velocity estimating unit 120b extracts the background image (see FIG. 12). Then, from the reference background image DB 160, the speed estimating unit 120b identifies, as the position of the runner, a point associated with a reference background image having a feature amount common to that of the background image. By repeating the above process, for example, the runner's position at time t1 and the runner's position at time t+1 shown in FIG. The image sent from the image clipping unit 110 and the image held in the reference background image DB 160 are preferably clipped at the timing when the runner's feet touch the ground, as in the first embodiment.

As the reference background image, the background image of the model moving body included in the photographed image when the model moving body moves along the movement path of the moving body can be used. Also, as the background image of this model moving body, an image clipped from a video of the same tournament held in the past can be used.

According to this embodiment, there is an advantage that the speed of the runner can be estimated without using the speed of the camera 200 (relay van).

Although each embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiments, and further modifications, replacements, and adjustments can be made without departing from the basic technical idea of the present invention. can be added. For example, the network configuration, the configuration of each element, and the form of expression of messages shown in each drawing are examples for helping understanding of the present invention, and are not limited to the configurations shown in these drawings. Moreover, in the following description, "A and/or B" is used to mean at least one of A and B.

For example, in the above-described embodiment, an example in which relay video is reproduced on a television receiver via the viewing support device 100 has been described, but the form of the viewing support device 100 is not limited to this. For example, the viewing support device 100 may be caused to estimate the speed of a subject and provide the estimated speed for a relay video distributed by a video distributor or the like via the Internet. In this case, the speed information or the like provided by the viewing support device 100 is a PC (personal computer), a tablet terminal, a smartphone, or other device capable of reproducing moving images via the Internet. Of course, these devices also include television receivers equipped with Internet connection functions.

Further, in the above-described embodiment, an image cut out from a relay video is mainly used to identify a subject and estimate the speed of the subject. A configuration that corrects or removes errors can also be adopted. For example, if OB van speed information is available, that information can be used to verify whether the estimated speed of the subject is correct. In addition, depending on the competition, it may be obligatory to attach an RFID tag or chip to the body of the runner or the clothing worn by the runner. Information obtained from these RFID tags and chips can also be used to identify runners and verify their estimated speed.

For example, in the above-described embodiment, an example in which the runner's information and its estimated speed are provided from the relay video of the road race has been described, but the scope of application of the present invention is not limited to this, and the relay video can be obtained. It is applicable to competitions in general. For example, in the training of track and field athletes, it is possible to use it for training by measuring the speed and distance in real time from the video taken by the camera. In addition to track races in general, the present invention can also be applied to the speed estimation of objects in relay images of bicycle races, horse races, automobile races, swimming, and the like.

In addition to competitive sports, for example, for videos taken by drones patrolling event venues for security purposes, the above-mentioned method can be used to identify the subject, estimate its speed, and It can also be applied to the use of notifying a person. For example, by providing a security commander with a specific person to be monitored and its speed information, the present invention can also be applied to command purposes such as dispatching a security officer or widening the security range.

Further, the procedures shown in the above-described first to third embodiments are such that the computer (9000 in FIG. 13) functioning as the viewing support devices 10, 100 to 100c functions as the viewing support devices 10, 100 to 100c. It can be implemented by a program that implements it. Such a computer is exemplified by a configuration including a CPU (Central Processing Unit) 9010, a communication interface 9020, a memory 9030, and an auxiliary storage device 9040 in FIG. That is, the CPU 9010 in FIG. 13 may execute the moving object identification program and the speed estimation program to update each calculation parameter held in the auxiliary storage device 9040 or the like.

That is, each part (processing means, function) of the viewing support devices 10, 100 to 100c shown in each of the above-described embodiments performs each of the above-described processes by using the hardware in the processors installed in these devices. It can be realized by a computer program to be executed.

Finally, preferred forms of the invention are summarized.
[First form]
(Refer to the viewing support device from the first viewpoint above)
[Second form]
The mobile object handled by the viewing support device described above is a runner in a competitive event,
The clipping means can take a configuration that clips the image at the timing when the runner's foot touches the ground.
[Third form]
The means for estimating the speed of the moving object includes:
estimating the relative distance between the camera and the runner from the image of the moving object;
A configuration can be adopted in which a process of estimating the relative speed between the camera and the moving body is performed from the frame rate of the image showing the moving body and the change in the relative distance between the camera and the runner.
[Fourth mode]
The means for estimating the speed of the moving body comprises:
By summing the relative speed of the moving object and the moving speed of the camera, it is possible to adopt a configuration in which the moving speed of the runner is calculated.
[Fifth form]
In the viewing support device described above,
The means for estimating the speed of the moving body can adopt a configuration that estimates the speed of the moving body based on changes in feature amounts calculated from a background image of an image in which the moving body is projected.
[Sixth form]
In the viewing support device described above,
The means for estimating the speed of the moving body comprises:
a feature amount of a background image of the model moving body included in a captured image when the model moving body moves along the moving path of the moving body;
Identifying the position of the moving object by matching the feature amount calculated from the background image of the image in which the moving object is projected,
A configuration can be adopted in which the position of the moving object is estimated using the specified position.
[Seventh form]
means for clipping, at predetermined time intervals, an image showing the plurality of moving bodies from a relay video in which a plurality of moving bodies moving from a first point to a second point are captured;
means for estimating a distance difference between the plurality of moving bodies based on a change in a background image of an image showing the plurality of moving bodies;
and means for outputting a distance difference between the plurality of moving objects.
[Eighth mode]
The viewing support device described above is
Furthermore, means for specifying the moving object based on the image of the moving object displayed in the image,
A configuration can be adopted in which information on the identified moving object and at least one of its estimated speed and distance difference are output.
[Ninth form]
In the viewing support device described above,
The moving object is a runner in a competitive event,
The means for identifying the moving object is
A configuration can be adopted in which the runner is specified based on the features of the face image of the runner or information on the bib.
[Tenth mode]
In the viewing support device described above,
moreover,
comprising means for determining the expression of the runner from the features of the face image of the runner;
The means for outputting can adopt a configuration for outputting the facial expression of the runner in a predetermined manner.
[Eleventh form]
(See the viewing support method from the third viewpoint above.)
[Twelfth Mode]
(Refer to the program from the fourth viewpoint above)
It should be noted that the eleventh to twelfth modes can be developed into the second to tenth modes in the same manner as the first mode.

The disclosures of the above patent documents and non-patent documents are incorporated herein by reference. Within the framework of the full disclosure of the present invention (including the scope of claims), modifications and adjustments of the embodiments and examples are possible based on the basic technical concept thereof. Also, within the framework of the disclosure of the present invention, various combinations or selections (partial (including targeted deletion) is possible. That is, the present invention naturally includes various variations and modifications that can be made by those skilled in the art according to the entire disclosure including claims and technical ideas. In particular, any numerical range recited herein should be construed to specifically recite any numerical value or subrange within that range, even if not otherwise stated.

Reference Signs List 10, 100, 100a, 100b, 100c Viewing support device 11, 110 Image clipping unit 12, 120, 120b Speed estimating unit 14, 140 Output unit 18, 180 Distance difference estimating unit 130 Moving body identifying unit 150 Runner information DB (database)
150a Runner identification engine 160 Reference background image DB (database)
170 facial expression determination unit 200 camera 300 television receiver 400a, 400b box 401a, 401b icon 9000 computer 9010 CPU
9020 Communication interface 9030 Memory 9040 Auxiliary storage device

Claims (10)

identifying means for identifying information about a person based on a face appearing in the image, in which the person moves from a first point to a second point;
means for cutting out an image in which the person is projected from the video at predetermined time intervals;
means for estimating the speed of the person based on changes in the background image of the image in which the person is projected;
An information processing apparatus comprising means for outputting the estimated speed of the person and the information. 2. An information processing apparatus according to claim 1, wherein said clipping means clips the image at the timing when said person's feet touch the ground. The means for estimating the velocity of the person comprises:
estimating the relative distance between the camera and the person from the image in which the person is projected;
3. The information processing apparatus according to claim 1, wherein a process of estimating a relative velocity between the camera and the person is performed based on a frame rate of an image showing the person and a change in relative distance between the camera and the person. The means for estimating the velocity of the person comprises:
4. The information processing apparatus according to claim 3, wherein the moving speed of the person is calculated by adding the relative speed of the person and the moving speed of the camera. 3. The information processing apparatus according to claim 1, wherein the means for estimating the speed of the person estimates the speed of the person based on a change in feature quantity calculated from a background image of an image in which the person is projected. The means for estimating the velocity of the person comprises:
Collating the feature amount of the background image of the model person included in the captured image when the model person moves along the movement route of the person and the feature amount calculated from the background image of the image in which the person is projected. Identifying the position of the person by
4. The information processing apparatus according to any one of claims 1 to 3, wherein the position of the person is estimated using the specified position. means for cutting out, at predetermined time intervals, an image showing the plurality of persons from a video in which a plurality of persons moving from a first point to a second point are taken as subjects;
means for estimating a distance difference between the plurality of persons based on changes in a background image of an image in which the plurality of persons are projected;
7. The information processing apparatus according to claim 6, further comprising means for outputting a distance difference between said plurality of persons. Furthermore, means for specifying the person based on the image of the person displayed in the image,
8. The information processing apparatus according to claim 7, which outputs information about the identified person and at least one of its estimated speed and distance difference. In a video showing a person moving from a first point to a second point, specifying information about the person based on the face shown in the video,
Cut out an image showing the person at a predetermined time interval from the video,
estimating the speed of the person based on changes in the background image of the image in which the person is projected;
outputting the estimated velocity of the person and the information;
Information processing methods. to the computer,
A process of identifying information about the person based on the face appearing in the image in the image showing the person moving from the first point to the second point,
A process of cutting out an image in which the person is projected from the video at predetermined time intervals;
A process of estimating the speed of the person based on changes in the background image of the image in which the person is projected,
a process of outputting the estimated speed of the person and the information;
program to run.

Images