JP7464437B2 - Information processing device - Google Patents

Description

The present invention relates to image editing technology.

In live broadcasts of sports such as soccer, where the score is the most important factor, the status of the scores is displayed. For example, Patent Document 1 discloses a system that displays scoreboard information generated by photographing the scoreboard during the match on a television screen that displays real-time footage of the match.

JP 2001-245289 A

In the system described in Patent Document 1, a dedicated camera must be prepared to photograph the scoreboard. Therefore, the present invention aims to provide a mechanism that can display the score status of a competition without using a dedicated camera to photograph the scoreboard.

To solve the above problems, the present invention provides an information processing device that includes a video data acquisition unit that acquires video data showing images captured by one or more imaging devices used by spectators of a sport where scores are competed for, a judgment unit that judges whether the acquired video data contains an image showing the score status of the sport, a generation unit that analyzes the image showing the score status of the sport contained in the video data and generates score display data for displaying the score status when the video data is played back, and a distribution unit that distributes the generated score display data and the video data to a display device.

The system may include an attached data acquisition unit that acquires attached data that includes text information input by the spectators and that has been attached to the video data, and the generation unit may generate score display data for displaying information specific to the sport according to the text information included in the acquired attached data together with the score situation when the video data is played back.

The imaging device may include an attribute data acquisition unit that acquires attribute data of spectators using the imaging device, and the generation unit may generate score display data for displaying information specific to the sport according to the acquired attribute data together with the scoring situation when the video data is played back.

The apparatus may include a data acquisition unit that acquires data indicating the location and date and time when the image was captured by the imaging device, and the generation unit may generate score display data for displaying information specific to the sport according to the acquired data together with the score status when the video data is played back.

The system may include a metadata acquisition unit that acquires metadata of the video data, and the generation unit may generate score display data for displaying information specific to the sport according to the acquired metadata together with the score status when the video data is played.

The imaging device may be provided with a transmission unit that transmits information to the spectators instructing them to capture the scoring situation.

If the determination unit determines that the video data does not include an image showing the scoring status of the competition, the transmission unit may transmit information to the imaging device instructing the spectators to capture an image of the scoring status.

The apparatus may further include a determination unit that analyzes data added to the video data, the data including text information input by the spectators, to determine whether or not there has been a change in the scoring situation, and the transmission unit may transmit, when the determination unit determines that there has been a change in the scoring situation, information to the imaging device instructing the spectators to capture an image of the scoring situation.

If the determination unit determines that the video data does not include an image showing the scoring status of the competition, the generation unit may generate score display data including information indicating that the scoring status displayed when the video data is played back may not be accurate.

If the determination unit determines that the video data does not include an image showing the scoring status of the competition, the generation unit may analyze the video data to infer the scoring status, and generate score display data for displaying the inferred scoring status.

According to the present invention, it is possible to display the score of a competition without using a dedicated camera to photograph the scoreboard.

FIG. 1 is a diagram illustrating an example of a device configuration of an information processing system 1. FIG. 2 is a diagram illustrating a hardware configuration of the server device 10. FIG. 2 is a diagram illustrating a hardware configuration of a user terminal 20. FIG. 2 is a diagram illustrating an example of a functional configuration of the information processing system 1. 4 is a flowchart showing an example of an operation of the server device 10. 1 is a diagram illustrating an example of an image displayed on a user terminal 20 serving as an imaging device. 4 is a flowchart showing an example of an operation of the server device 10. 1 is a schematic diagram illustrating a relationship between video data and additional data; FIG. 2 is a schematic diagram illustrating a structure of additional data; 1 is a diagram illustrating an example of an image displayed on a user terminal 20 as a display device.

[composition]
1 is a block diagram illustrating an example of a device configuration of an information processing system 1 according to an embodiment of the present invention. The information processing system 1 is a system for capturing video of, for example, soccer being played and playing the captured video. The subject of the video captured by the information processing system 1 is not limited to soccer, and may be any sport in which players compete for points.

This information processing system 1 includes a server device 10 that mainly performs processes related to the storage, editing, and distribution of video data, and a plurality of user terminals 20 that mainly performs processes related to the capture or display of video data. These server devices 10 and user terminals 20 are communicatively connected via a network 30. The network 30 is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network), or a combination of these, and includes wired or wireless sections.

The server device 10 is a computer device that functions as an example of an information processing device according to the present invention. The server device 10 has functions such as storing, editing, and distributing video data. The number of server devices 10 shown in the figure is an example, and may be one or more.

The user terminal 20 is, for example, a smartphone, a digital camera, a tablet, or a computer device such as a wearable device or a personal computer. The user terminal 20 has at least one of a video capture function and a video display function. The user terminal 20 with a video capture function functions as an imaging device that captures videos. The user terminal 20 with a video display function functions as a display device that displays videos. The user terminal 20 with a video capture function and a video display function functions as an imaging device and a display device. The user terminal 20 as an imaging device is used by a user as a spectator who captures videos at a stadium. Video data showing the video captured by the user terminal 20 as an imaging device is transmitted to the server device 10 via the network 30 by wireless or wired communication. The user terminal 20 as a display device is used by a user who is located away from the stadium and views the video. The user terminal 20 as a display device acquires the video data distributed by the server device 10 via the network 30 by wireless or wired communication, and displays the video based on this video data. The number of user terminals 20 shown in the figure is an example, and may be one or more.

Figure 2 is a block diagram illustrating an example of the hardware configuration of server device 10 as an information processing device. This is a diagram showing an example of the hardware configuration of server device 10. Server device 10 is physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, and buses connecting these. In the following description, the word "device" can be interpreted as a circuit, device, unit, etc. The hardware configuration of server device 10 may be configured to include one or more of the devices shown in the figure, or may be configured to exclude some of the devices.

Each function of the server device 10 is realized by loading a specific software (program) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communications via the communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and storage 1003.

The processor 1001, for example, runs an operating system to control the entire computer. The processor 1001 may be configured as a central processing unit (CPU) that includes an interface with peripheral devices, a control device, an arithmetic unit, registers, etc.

The processor 1001 reads out programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. The programs used are those that cause a computer to execute at least some of the operations described below. The functional blocks of the server device 10 may be realized by a control program stored in the memory 1002 and running on the processor 1001. The various processes may be executed by one processor 1001, or may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may be transmitted to the server device 10 via a telecommunications line.

The memory 1002 is a computer-readable recording medium, and may be composed of at least one of, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a RAM (Random Access Memory), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing the method according to this embodiment.

Storage 1003 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device.

The communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, a communication module, etc. The transmitting/receiving unit may be implemented as a transmitting unit and a receiving unit that are physically or logically separated.

The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, etc.) that accepts input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. Note that the input device 1005 and the output device 1006 may be integrated into one configuration (e.g., a touch panel).

Each device, such as the processor 1001 and memory 1002, is connected by a bus for communicating information. The bus may be configured using a single bus, or may be configured using different buses between each device.

The server device 10 may also be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

3 is a diagram showing the hardware configuration of the user terminal 20 that functions as an imaging device and a display device. The user terminal 20 is physically configured as a computer device including a processor 2001, a memory 2002, a storage 2003, a communication device 2004, an input device 2005, an output device 2006, a positioning device 2011, and a bus connecting these. Each function of the user terminal 20 is realized by loading a specific software (program) onto hardware such as the processor 2001 and the memory 2002, causing the processor 2001 to perform calculations, control communication by the communication device 2004, and control at least one of reading and writing data in the memory 2002 and the storage 2003. The processor 2001 also functions as a clock device that measures the date and time based on a clock. The processor 2001, memory 2002, storage 2003, communication device 2004, input device 2005, and output device 2006 are a group of devices similar to the processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, and output device 1006 of the server device 10, but differ in the following respects.

The communication device 2004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc., to realize, for example, at least one of Frequency Division Duplex (FDD) and Time Division Duplex (TDD). For example, the transmitting and receiving antennas, the amplifier unit, the transmitting and receiving unit, the transmission path interface, etc. may be realized by the communication device 1004.

3 illustrates an example of the hardware configuration of the user terminal 20 that functions as an imaging device and a display device. For this reason, the input device 2005 includes, for example, a keyboard, a mouse, a switch, a button, a camera 2007 as an imaging unit, and a microphone 2008 as a sound collection unit. The output device 2006 includes a display 2009 as a display unit and a speaker 2010 as a sound emission unit, and may also include, for example, an LED lamp. In a user terminal 20 that functions only as an imaging device, the display 2009 as a display unit and the speaker 2010 as a sound emission unit are not essential. In a user terminal 20 that functions only as a display device, the camera 2007 as an imaging unit and the microphone 2008 as a sound collection unit are not essential. Note that the hardware configuration of the user terminal 20 may be configured to include one or more of the devices shown in the figure, or may be configured without including some of the devices.

Figure 4 is a block diagram illustrating an example of the functional configuration of the information processing system 1. Figure 4 illustrates an example of the functional configuration of the entire system including a user terminal 20 that functions as an imaging device and a display device. In the user terminal 20, a video data generating unit 21 generates video data consisting of an image captured by a camera 2007 and audio picked up by a microphone 2008.

A user as a spectator can perform an operation to add data called a "marker" while capturing an image using the user terminal 20 or while playing back a captured image on the user terminal 20. For example, a marker containing text information such as "GOAL!" can be added to a scene where a goal is scored in soccer. In response to this operation, the marker adding unit 23 of the user terminal 20 adds a marker containing the text information "GOAL!" input by the user to a position specified by the user in the video data. Here, the position specified by the user in the video data is a part of the video data corresponding to the scene where a goal is scored in the above example. The marker includes the text information input by the user and data indicating the position in the video data where the marker was added (for example, a timestamp indicating the elapsed time from the start of capturing the video data). This marker is used to play back the video data from the position where the marker was added.

In the user terminal 20, the transmission unit 22 transmits various data to the server device 10 via the network 30. For example, in the user terminal 20, the transmission unit 22 transmits the video data and markers generated by the video data generation unit 21 to the server device 10 via the network 30.

In the user terminal 20 functioning as a display device, the user as a viewer can view the video captured by the user terminal 20 functioning as an imaging device. In this case, the user can view the video from the scene he or she desires by selecting one of the multiple markers attached to the video data in the user terminal 20 functioning as a display device. For example, in a soccer video, by selecting a marker containing the text information "goal" from a group of markers containing text information such as "pass," "shoot," and "goal," the video can be viewed from the goal scene to which the marker is attached. In the user terminal 20 functioning as a display device, the marker selection unit 24 identifies the selected marker when the user selects one of the multiple markers attached to the video data. In the user terminal 20 functioning as a display device, the transmission unit 22 transmits a marker identifier that identifies the marker identified by the marker selection unit 24 to the server device 10 via the network 30.

In the user terminal 20, the output unit 25 displays an image corresponding to the video data, etc. on the display 2009, and emits audio corresponding to the video data, etc. from the speaker 2010.

In the server device 10, the acquisition unit 11 acquires various data transmitted from the user terminal 20 via the network 30. For example, the acquisition unit 11 functions as a video data acquisition unit that acquires video data captured by each of the multiple user terminals 20 as imaging devices and markers assigned to the video data via the network 30. The acquisition unit 11 also functions as an assigned data acquisition unit that acquires, via the network 30, the marker identifier of a marker selected by a user from each of the multiple user terminals 20 as display devices.

In the server device 10, the selected marker identification unit 12 identifies the marker identifier of the marker selected by the user on each of the multiple user terminals 20 serving as display devices.

In the server device 10, the judgment unit 13 judges whether or not the video data acquired by the acquisition unit 11 (video data acquisition unit) contains an image showing the score status of the competition. In other words, since the video data captured by the user terminal 20 as an imaging device is captured by spectators, this video data may contain images showing the score status of the competition, such as scoreboards and score tables installed at the stadium. The judgment unit 13 uses image recognition technology, such as pattern matching, to judge whether or not the video data contains an image showing the score status of the competition.

The generation unit 14 analyzes an image showing the score status of a competition, which is included in the video data, and generates score display data for displaying the score status when the video data is played. Specifically, the generation unit 14 identifies the score status by performing character recognition on numbers, characters, etc. in the image showing the score status of a competition, and generates score display data for displaying the score status. At this time, the generation unit 14 generates score display data for displaying information specific to the competition based on the score status together with the score status when the video data is played. Here, the information specific to the competition based on the score status is unique information regarding the score status in each competition, such as set point or match point in tennis.

The distribution unit 15 distributes various data, such as the score display data generated by the generation unit 14 and the video data acquired by the acquisition unit 11 (video data acquisition unit), to a user terminal 20 serving as a display device via the network 30. When a user selects one of the multiple markers assigned to the video data, the distribution unit 15 distributes the video data from the position where the marker identified by the selected marker identification unit 12 is assigned.

The transmission unit 16 transmits instruction information to the user terminal 20 as an imaging device, instructing the user as a spectator to capture an image of the scoring situation. In particular, the transmission unit 16 transmits this instruction information when the determination unit 13 determines that the video data does not include an image showing the scoring situation of the competition.

[motion]
Next, the operation of this embodiment will be described below, firstly how to capture a moving image, and then how to display the moving image.

[Video capture]
The user who is the image taker operates the user terminal 20 to start the imaging process using the camera 2007 and the microphone 2008. In response to this, the video data generating unit 21 of the user terminal 20 generates video data. For example, the user performs an operation to add a marker while continuing imaging using the user terminal 20. In response to this operation, the marker adding unit 23 of the user terminal 20 adds a marker including character information input by the user to a position specified by the user in the video data. The transmitting unit 22 of the user terminal 20 transmits the video data and the marker to the server device 10 via the network 30 according to any protocol such as RTMP (Real-Time Messaging Protocol) or HTTPS (Hypertext Transfer Protocol Secure). The acquiring unit 11 of the server device 10 acquires the video data and the marker transmitted by the transmitting unit 22 of the user terminal 20 via the network 30 and stores them (step S11).

Here, FIG. 8 is a schematic diagram illustrating the relationship between video data and markers. As shown in the figure, multiple markers M1 to M5 are added to different positions on the time axis in the video data. Then, as illustrated in FIG. 9, each marker M includes position data indicating the position at which the marker M is added in the video data (e.g., a timestamp indicating the elapsed time from the start of imaging), a marker identifier that identifies the marker, text information input by the user, and other associated information.

In FIG. 5, the judgment unit 13 of the server device 10 performs image analysis on the video data acquired by the acquisition unit 11 (video data acquisition unit) (step S12) and determines whether the video data includes an image showing the score of the competition (step S13).

If it is determined that the video data contains an image showing the score status of the competition (step S13; YES), the generation unit 14 analyzes the image showing the score status of the competition contained in the video data, and generates score display data for displaying the score status when the video data is played back (step S14). At this time, the generation unit 14 generates data for displaying information specific to the competition.

On the other hand, if it is determined that the image showing the score status of the competition is not included (step S13; NO), the transmission unit 16 transmits instruction information to the user terminal 20 as an imaging device, instructing the user as a spectator to capture the score status. Here, FIG. 6 is a diagram showing an example of an image displayed on the user terminal 20 as an imaging device. In the figure, the captured image is displayed in the captured image display area 105. Furthermore, the instruction information display area 106 displays the message "Please capture the scoreboard", which is the instruction information described above. The video data, markers, score display data, etc. generated in the above manner are stored in the distribution unit 15 of the server device 10.

[Video display]
In FIG. 7, when a certain video data is designated from any of the user terminals 20 and a request to start playing is made (step S21; YES), the distribution unit 15 of the server device 10 reads out the designated video data and transmits it to the user terminal 20 to start playing it (step S22). FIG. 10 is a diagram showing an example of a display when video data is played on the user terminal 20. On the display 2009 of the user terminal 20, a video corresponding to the video data is displayed in the video display area 102, and an image 107 of the score situation corresponding to the score display data and an image 108 corresponding to the data for displaying information specific to the game are displayed so as to be superimposed on the video. In addition, an information group 101 specified from the metadata of the video data and a character information group 103 included in a marker assigned to the video data are also displayed. The character information group 103 on the left side of the figure corresponds to a marker assigned to an earlier position on the time axis of the video data, and the character information group 103 on the right side of the figure corresponds to a marker assigned to a later position on the time axis of the video data. By specifying the control image 104, the user can rewind or fast forward the group of text information corresponding to each marker on the time axis, and tap any piece of text information to select that piece of text information (i.e., a marker).

When the user selects any piece of text information (step S23; YES), the user terminal 20 notifies the server device 10 of the marker identifier of the marker corresponding to the selected text information. The selected marker identification unit 12 of the server device 10 identifies the selected marker using the notified marker identifier. In response, the distribution unit 15 changes the playback position and plays back the video by transmitting video data to the user terminal 20 from the position where the identified marker is added (step S24). This allows the user to view the video from the scene corresponding to the marker selected by the user.

According to the embodiment described above, since an imaging device that is used by an unspecified number of spectators is used, it is possible to display the score situation of the competition without using a dedicated camera for photographing the scoreboard.

[Modification]
The present invention is not limited to the above-described embodiment. The above-described embodiment may be modified as follows. In addition, two or more of the following modifications may be combined and implemented.

[Modification 1]
In the embodiment, the user terminal 20 accepts an operation of adding a marker by a user who is an image capturer, but an imaging device for capturing a video and a marker adding device for adding a marker may be separate devices. Specifically, a user who is an image capturer may operate an imaging device (e.g., a smartphone or a video camera) to capture an image, and may operate a marker adding device (e.g., a remote control for a smartphone, or a wearable device such as a smartwatch connected to a smartphone) to add a marker.

[Modification 2]
It is considered that the sport being imaged can be identified from the text information included in the marker. For example, it is possible to determine that the sport being imaged is soccer based on text information such as "I love soccer" or "Go XXX football team." Therefore, the generating unit 14 may determine the sport being imaged based on the text information included in the marker acquired by the acquiring unit 11 (attachment data acquiring unit) together with the score situation when playing back the video data, and generate score display data for displaying information specific to the sport.

[Modification 3]
It is also possible to determine the sport to be imaged from the attributes of the user as a spectator. For example, it can be inferred that the sport being imaged by a user whose hobby is watching soccer is soccer. In other words, the server device 10 may be provided with an attribute data acquisition unit that acquires attribute data of spectators who use the imaging device from a predetermined user database or the like, and the generation unit 14 may generate score display data for displaying information specific to the sport according to the acquired attribute data together with the score situation when playing back the video data.

[Modification 4]
It is also possible to determine the sport to be imaged from the location and time of the image capture. For example, if the location and time of the image capture are the location and date and time when a soccer match is scheduled to be held at a stadium, it can be inferred that the sport being imaged is soccer. Therefore, the server device 10 may include a data acquisition unit that acquires data indicating the location and date and time when the image capture was performed by the imaging device, and the generation unit 14 may refer to a database that stores the schedule of the sport together with the score status when playing back the video data, and generate score display data for displaying information specific to the sport corresponding to the acquired data.

[Modification 5]
As shown in Fig. 10, the information group 101 may be specified from the metadata of the video data, and it is also possible to determine the sport being filmed from the metadata of such video data. For example, if the metadata contains information such as "XX soccer tournament", it can be inferred that the sport being filmed is soccer. Therefore, the server device 10 may include a metadata acquisition unit that acquires the metadata of the video data, and the generation unit 14 may generate score display data for displaying, together with the score situation, information specific to the sport according to the acquired metadata when the video data is played back.

[Modification 6]
The camera may be instructed to capture an image when there is a possibility that the score situation has changed. For example, when the marker contains text information such as "GOAL!" or "WINNING", there is a high possibility that the score situation has changed. Therefore, the server device 10 stores in advance candidates for text information that may be included in the marker when there is a high possibility that the score situation has changed. The server device 10 includes a determination unit that analyzes the marker, which contains text information input by a spectator and is added to the video data, based on the stored contents, and determines whether there has been a change in the score situation. The transmission unit 16 may transmit information to the imaging device instructing the spectator to capture an image of the score situation when the determination unit determines that there has been a change in the score situation.

[Modification 7]
If it is determined that the scoring situation has not been captured, it may be displayed that the displayed scoring situation may be incorrect. In other words, if the determination unit 13 determines that the video data does not include an image showing the scoring situation of the competition, the generation unit 14 may generate score display data including information meaning that the scoring situation displayed when the video data is played back may be inaccurate (for example, a message such as "The latest scoring situation may differ from what is being displayed").

[Modification 8]
When it is determined that the scoring situation has not been captured, the latest scoring situation may be inferred. For example, in soccer, it is possible to perform image analysis such as pattern matching on the ball and goal included in the video data to infer the scoring situation from the positional relationship between the ball and the goal. When the determination unit 13 determines that the video data does not include an image showing the scoring situation of the game, the generation unit 14 may analyze the video data to infer the scoring situation and generate score display data for displaying the inferred scoring situation.

[Other Modifications]
The block diagrams used in the description of the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and/or software. The means for realizing each functional block is not particularly limited. That is, each functional block may be realized by one device that is physically and/or logically coupled, or may be realized by two or more devices that are physically and/or logically separated and directly and/or indirectly (e.g., wired and/or wirelessly) connected to each other. For example, at least a part of the functions of the server device 10 may be implemented in the user terminal 20.

Each aspect/embodiment described herein may be applied to systems utilizing LTE (Long Term Evolution), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), or other suitable systems and/or next generation systems enhanced based thereon.

The steps, sequences, flow charts, etc. of each aspect/embodiment described herein may be reordered unless inconsistent, for example, the methods described herein present elements of various steps in an example order, and are not limited to the particular order presented.
Each aspect/embodiment described in this specification may be used alone, in combination, or switched depending on the implementation.

As used herein, the terms "system" and "network" are used interchangeably.

As used herein, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. "Determining" and "determining" may also include resolving, selecting, choosing, establishing, comparing, and the like. In other words, "judging" and "deciding" can include regarding some action as having been "judged" or "decided."

The present invention may be provided as an information processing method having processing steps performed in the information processing system 1 or the server device 10. The present invention may also be provided as a program executed in the server device 10. Such a program may be provided in a form recorded on a recording medium such as an optical disk, or in a form in which the program is downloaded to a computer via a network such as the Internet and installed to make it available for use.

Software, instructions, etc. may be transmitted and received over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies, such as coaxial cable, fiber optic cable, twisted pair, and digital subscriber line (DSL), and/or wireless technologies, such as infrared, radio, and microwave, these wired and/or wireless technologies are included within the definition of transmission media.

The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

Terms described in this specification and/or terms necessary for understanding this specification may be replaced with terms having the same or similar meanings. For example, a channel and/or a symbol may be a signal. A signal may be a message. A component carrier (CC) may be called a carrier frequency, a cell, etc.

To the extent that the terms "including," "comprising," and variations thereof are used herein, these terms are intended to be inclusive, similar to the term "comprising." Further, the term "or" as used herein is not intended to be an exclusive or.

Throughout this disclosure, where articles have been added through translation, such as a, an, and the in English, these articles are intended to include the plural unless the context clearly indicates otherwise.

Although the present invention has been described in detail above, it is clear to those skilled in the art that the present invention is not limited to the embodiments described in this specification. The present invention can be implemented in modified and altered forms without departing from the spirit and scope of the present invention as defined by the claims. Therefore, the description in this specification is intended to be illustrative and does not have any limiting meaning on the present invention.

1: Information processing system, 10: Server device, 11: Acquisition unit, 12: Selected marker identification unit, 13: Determination unit, 14: Generation unit, 15: Distribution unit, 16: Transmission unit, 20: User terminal, 21: Video data generation unit, 22: Transmission unit, 23: Marker assignment unit, 24: Marker selection unit, 25: Output unit, 30: Network, 1001: Processor, 1002: Memory, 1003: Storage, 1004: Communication device, 1005: Input device, 1006: Output device, 2001: Processor, 2002: Memory, 2003: Storage, 2004: Communication device, 2005: Input device, 2006: Output device, 2007: Camera, 2008: Microphone, 2009: Display, 2010: Speaker.

Claims (10)

a video data acquisition unit that acquires video data representing images captured by one or more imaging devices used by spectators of a sport in which points are competed for;
a determination unit that determines whether the acquired video data includes an image showing the score status of the game;
a generation unit that analyzes an image showing the score status of the game, which is included in the video data, and generates score display data for displaying the score status when the video data is played;
a distribution unit that distributes the generated score display data and the video data to a display device. an additional data acquisition unit that acquires additional data that includes character information input by the spectator and that is added to the video data;
The information processing device according to claim 1, characterized in that the generation unit generates score display data for displaying, together with the score status, information specific to the sport corresponding to the text information contained in the acquired grant data when the video data is played back. an attribute data acquisition unit that acquires attribute data of spectators who use the imaging device;
The information processing device according to claim 1 or 2, characterized in that the generation unit generates score display data for displaying, together with the score status, information specific to the sport according to the acquired attribute data when the video data is played back. a data acquisition unit that acquires data indicating a place and date when an image was captured by the imaging device,
The information processing device according to claim 1 or 2, characterized in that the generation unit generates score display data for displaying, together with the score status, information specific to the sport according to the acquired data when the video data is played back. a metadata acquisition unit for acquiring metadata of the video data,
The information processing device according to claim 1 or 2, characterized in that the generation unit generates score display data for displaying, together with the score status, information specific to the sport according to the acquired metadata when the video data is played back. The information processing apparatus according to any one of claims 1 to 5, further comprising a transmission unit configured to transmit, to the imaging device, information instructing the spectators to image the scoring situation. When the determination unit determines that the video data does not include an image showing the score status of the competition,
The information processing device according to claim 6 , wherein the transmission section transmits, to the imaging device, information instructing the spectators to image the scoring situation. a determination unit that analyzes added data, which includes character information input by the spectators and is added to the video data, and determines whether or not there is a change in the score situation;
The information processing device according to claim 6, characterized in that the transmission unit, when the determination unit determines that there is a change in the score situation, transmits information to the imaging device instructing the spectators to capture the score situation. When the determination unit determines that the video data does not include an image showing the score status of the game,
The information processing device according to any one of claims 1 to 8, characterized in that the generation unit generates score display data including information indicating that the score status displayed when the video data is played back may not be accurate. When the determination unit determines that the video data does not include an image showing the score status of the game,
The information processing device according to any one of claims 1 to 9, wherein the generation unit analyzes the video data to infer the score situation, and generates score display data for displaying the inferred score situation.

Images