KR102107055B1 - Method and device for recommending sports relay video based on machine learning - Google Patents

Abstract

The present invention relates to a method for recommending a sports relay video based on machine learning. The method for recommending the sports relay video based on machine learning comprises the steps of: receiving a plurality of video data photographed by a plurality of video cameras; performing primary machine learning for calculating a characteristic value related to the preset sports characteristic information with regard to the plurality of received video data; performing secondary machine learning for extracting any one of the plurality of video data as a user preference sports video data by using a result of the primary machine learning; and recommending the extracted user preference sports video data.

Description

METHOD AND DEVICE FOR RECOMMENDING SPORTS RELAY VIDEO BASED ON MACHINE LEARNING

The present invention relates to a method and apparatus for recommending a sports relay video based on machine learning.

The stereoscopic image (3D image) refers to a 3D image that simultaneously provides shape information about depth and space. In the case of a stereo image, images of different viewpoints are provided to the left and right eyes, whereas a stereoscopic image provides an image as viewed from different directions each time the viewer views a different viewpoint. Therefore, in order to generate a stereoscopic image, images captured at various views are required.

In general, according to a multi-view high efficiency video coding (HEVC) process, a real scene is captured using two or more cameras, and a multi-view video (MVV) sequence is encoded, followed by bitstream. Is transmitted to MV-HEVC through the receiver side, and then through a decoding process, a 3D image can be displayed.

In this process, images taken at various viewpoints to generate a stereoscopic image have a large amount of data. Accordingly, considering a network infrastructure for realizing a stereoscopic image, terrestrial bandwidth, and the like, MPEG-2, H.264 / AVC, single-view video coding, or a conventional coding apparatus optimized for MVV compression Even using compression, there are many restrictions on implementation.

Unlike HEVC, MV-HEVC has multiple sequences of information, so motion estimation on the spatial axis is not only necessary, but motion estimation on the spatial axis is essential. Since the motion estimation of the spatio-temporal axis affects the efficiency of compression coding, research / development improves coding efficiency through optimal research of inter-frame prediction methods, and for computational tasks that require large amounts of computation, such as motion estimation and DCT. Research to increase the throughput of the system through parallelization is essential.

In addition, in the related art, despite the fact that the actual scene can be captured by two or more cameras and the captured image data can be secured, due to limitations in network infrastructure, broadcast bandwidth, etc., the user terminal only receives camera images provided unilaterally by the broadcasting station. It was only about receiving and printing. Therefore, the user could be guaranteed a variety of channel selection, but could not be guaranteed the right to select a camera image in the same channel.

In addition, in a live streaming broadcast, a viewer can select and view one of a plurality of images in the same channel. However, in most situations of live broadcasting such as TV, the viewer may miss the most suitable image in the process of selecting a desired image from among multiple images. In particular, a video with a fast flow of content, such as an athletic event, may miss an important moment compared to a static video like a drama.

The background technology of the present application is disclosed in Korean Patent Registration No. 10-1158491 (Registration Date: 2012.06.14).

The present application is for solving the above-mentioned problems of the prior art, and for the purpose of providing a sports image recommendation method and apparatus that can automatically recommend a desired image to a general viewer even if the viewer does not directly select the image based on machine learning. do.

However, the technical problems to be achieved by the embodiments of the present application are not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, a machine learning-based sports relay image recommendation method according to an embodiment of the present application includes receiving a plurality of image data captured by a plurality of image cameras, and receiving the Performing primary machine learning for calculating feature values related to predetermined sports characteristic information on a plurality of image data, and using any one of the plurality of image data using a result of the first machine learning, a user preferred sport And performing secondary machine learning to extract as image data and recommending the extracted user-preferred sports image data.

According to one embodiment of the present application, the step of performing the primary machine learning may include a feature value of the sports image data from the plurality of image data using a learning module learned by using the preset sports characteristic information as an input. Can be calculated.

According to one embodiment of the present application, the method for recommending a sports relay video based on machine learning further includes receiving user feedback regarding the recommended user preferred sports image data, and performing the second machine learning is , Reflecting the user feedback, it is possible to extract user preferred sports image data.

According to one embodiment of the present application, the method for recommending a sports relay video based on machine learning further includes collecting user information, wherein performing the second machine learning comprises: collecting the user information and the 1 Secondary machine learning can be performed to extract the results of secondary machine learning as user preferred sports image data.

According to one embodiment of the present application, the secondary machine learning may use a support vector machine (SVM) algorithm.

According to one embodiment of the present application, the preset sports characteristic information includes information related to the preference of a sports player, information related to possession of a ball, information related to a game watching terminal, information related to a player attempting to score a ball, information related to a foul player, and a player when a score occurs It may include related information and ball-related information in case of scoring.

According to an embodiment of the present application, a machine learning-based sports relay image recommendation apparatus includes: an image data receiving unit that receives a plurality of image data captured by a plurality of image cameras, and presets the received plurality of image data A first machine learning unit performing primary machine learning to calculate feature values related to sports characteristic information, and extracting any one of the plurality of image data as user preferred sports image data using the results of the first machine learning It may include a second machine learning unit for performing the second machine learning and a preferred image recommendation unit for recommending the extracted user preferred sports image data.

According to an embodiment of the present disclosure, the first machine learning unit may calculate a feature value of the sports image data from the plurality of image data using a learning module learned by using the preset sports characteristic information as an input. .

According to one embodiment of the present application, the machine learning-based sports relay image recommendation apparatus further includes a feedback receiving unit receiving user feedback regarding the recommended user-preferred sports image data, and the second machine learning unit comprises the user User preference sports image data may be extracted by reflecting the feedback.

According to an embodiment of the present application, the machine learning-based sports relay image recommendation device further includes a user information collection unit for collecting user information, wherein the second machine learning unit includes the collected user information and the primary machine. Secondary machine learning can be performed to extract the results of learning as user-preferred sports image data.

According to one embodiment of the present application, the sports relay video recommendation system receives a plurality of video data from a plurality of video cameras, and transmits the received plurality of video data, an image encoding device and image data encoded from the video encoding device Receiving, and performing a first machine learning to calculate a feature value related to predetermined sport characteristic information on the received plurality of image data, and using the result of the first machine learning to obtain a plurality of image data. And performing a secondary machine learning for extracting any one of the user preference sports image data, and may include a sports relay image recommendation apparatus recommending the extracted user preference sports image data.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present application. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and detailed description of the invention.

According to the above-described problem solving means of the present application, by using a machine learning to show a scene that is expected to be most desired by a user, a user misses a scene that occurs in a process of directly selecting an image that he or she wants to watch among multiple images. Let's solve the problem.

According to the above-described problem solving means of the present application, it is possible to select a specific value for a plurality of features as the main content of a sports event and recommend a sports scene that is expected to be most desired by a user through machine learning using the feature.

However, the effects obtainable herein are not limited to the effects described above, and other effects may exist.

1 is a schematic diagram of a sports relay video recommendation system according to an embodiment of the present application.
2 is a view for explaining a plurality of image data captured by a plurality of image cameras of a sports relay image recommendation apparatus according to an embodiment of the present application.
3 is a view for explaining the results of the first machine learning performance of the sports relay image recommendation apparatus according to an embodiment of the present application.
4 is a schematic block diagram of a sports relay video recommendation apparatus according to an embodiment of the present application.
5 is an operation flowchart of a method for recommending a sports relay video based on machine learning according to an embodiment of the present application.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present application pertains may easily practice. However, the present application may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present application in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout this specification, when a part is "connected" to another part, it is not only "directly connected", but also "electrically connected" or "indirectly connected" with another element in between. "It also includes the case where it is.

Throughout the present specification, when a member is positioned on another member “on”, “on the top”, “top”, “bottom”, “bottom”, and “bottom”, it means that a member is on another member. This includes cases where there is another member between the two members as well as when in contact.

Throughout this specification, when a part “includes” a certain component, it means that the component may further include other components, not to exclude other components, unless specifically stated to the contrary.

In the existing MV-HEVC environment, the user must directly select the video he or she wants to watch from among multiple videos within the same channel, which may cause problems such as a separate interface for selection or missed scenes in the selection process. there was. Based on machine learning, the present application can solve problems such as missing images by showing a scene that is expected to be most desired by a user in the same channel.

1 is a schematic diagram of a sports relay video recommendation system according to an embodiment of the present application.

Referring to FIG. 1, the sports relay video recommendation system 1000 may include a sports relay video recommendation apparatus 100, a video encoding apparatus 200, a user terminal 300, and a plurality of video cameras 400. In addition, although not shown in FIG. 1, the sports relay video recommendation system 1000 may further include, for example, a broadcast server and a device, a device for receiving a broadcast signal propagated from a broadcast station, and the like. In addition, some of the components of the sports relay video recommendation system 1000 may be provided in one device or a server. For example, the sports relay video recommendation device 100 and the user terminal 300 may be included in one device.

The sports relay image recommendation apparatus 100 uses the plurality of image data obtained by photographing a plurality of areas or people in a wide range of places such as a soccer field, a basketball court, a baseball field, and a volleyball field to utilize the user preferred sports image data 500. Can recommend The sports relay image recommendation apparatus 100 may recommend the user preferred sports image data 500 by using machine learning. The sports game described below has a fast flow of content and needs to capture important points such as scoring and fouling, and may include games such as soccer, basketball, volleyball, and baseball.

According to an embodiment of the present application, the sports relay video recommendation device 100 may be included in a set top box connected to the user terminal 300. Set-top box refers to a receiver for digital satellite broadcasting, and serves as a home communication terminal for interactive multimedia communication, connecting a TV and a network.

In addition, the sports relay video recommendation apparatus 100 may receive encoded image data from the video encoding apparatus 200. The video encoding apparatus 200 may transmit a bit stream of image data of a plurality of images captured by the plurality of image cameras 400 to the sports relay image recommendation apparatus 100. The image encoding apparatus 200 may convert image data of a plurality of images received from the plurality of image cameras 400 into a bit stream format. The plurality of video data is compressed into at least one bit-stream, and the sports relay video recommendation apparatus 100 may receive a compressed bit-stream through a network. The sports relay video recommendation apparatus 100 may decode and output some or all of the image data among a plurality of compressed images encoded by the image encoding apparatus 200.

The image encoding apparatus 200 may receive a plurality of image data respectively captured from each of the plurality of image cameras 400. The image encoding apparatus 200 includes characteristics of image data from a plurality of image cameras 400, the degree of zoom-in / zoom-out, the view of the image camera, and the position of the camera A plurality of image data, each of which is different from each other, may be received. For example, characteristics of image data may include resolution, color, similarity of background, number of pixels, number of image frames, and the like.

The plurality of cameras 400 may be image cameras that capture images from a plurality of locations. For example, the plurality of video cameras 400 may be video cameras that photograph a plurality of areas or people in a wide range of places, such as a soccer field, a basketball court, and a baseball field. For example, the arrangement of the plurality of cameras 400 may be one-dimensional parallel, two-dimensional parallel, one-dimensional array, or the like, and may be an image camera that photographs an image at a predetermined interval. The video camera may include a camera capable of video recording, such as a binocular camera, a camera using a horizontal rig, and a camera using an orthogonal rig. For example, referring to FIG. 1, the plurality of cameras 400 may include first cameras to twentieth cameras 1 to 20. Each of the first to 20th cameras 1 to 20 may provide a plurality of image data 1 'to 20'.

The image encoding apparatus 200 may encode image data received from a plurality of image cameras 400. In addition, the video encoding apparatus 200 may display the received video data according to the characteristics of the received video data, the viewpoint of the video camera, the degree of zoom-in / zoom-out, and the position of the camera. The encoding method can be determined.

Referring to FIG. 1 as an example, the plurality of video cameras 400 may include first cameras to twentieth cameras 1 to 20. Each of the plurality of video cameras 400 may be disposed at a predetermined interval within a stadium in which a current sporting event is in progress. A plurality of video cameras 400 are provided in different areas to photograph a situation in a stadium in which a sports event is currently in progress. The sports relay video recommendation apparatus 100 may receive the first video data to the 20th video data (1 'to 20') captured from each of the first camera to the 20th camera (1 to 20).

2 is a view for explaining a plurality of image data captured by a plurality of image cameras of a sports relay image recommendation apparatus according to an embodiment of the present application.

For example, referring to FIG. 2, the first image data 1 ′ illustrated in FIG. 2A may be image data captured from the first camera. Also, the second image data 2 'illustrated in FIG. 2B may be image data captured from the second camera. Further, the third image data 3 'illustrated in FIG. 2C may be image data captured from a third camera. Also, the fourth image data 5 'illustrated in FIG. 2D may be image data captured from the fourth camera. The first camera is located in the first area of the soccer field, and may capture the player located in the first area to obtain the first image data 1 '. In addition, the second camera is located in the second area of the soccer stadium, and may capture the player located in the second area to obtain the second image data 2 '. In addition, the third camera is located in the third area of the soccer stadium, and it is possible to acquire the third image data 3 'by photographing the player located in the third area. In addition, the fourth camera is located in the fourth area of the soccer stadium, and it is possible to acquire the fourth image data 4 'by photographing the audience seat. The first to fourth image data may include different sports characteristic information.

According to one embodiment of the present application, the sports relay image recommendation apparatus 100 may perform primary machine learning for calculating a feature value related to predetermined sports characteristic information on a plurality of received image data. The first machine learning can use the artificial neural network algorithm. In addition, the artificial neural network may be a deep learning neural network, but is not limited thereto, and various neural network systems developed or developed in the future may be applied.

In the image data analysis described below, an algorithm for image recognition, image analysis, and object tracking may be used, and the algorithm used at this time is a convolutional network, which is a type of neural network mainly used in speech recognition or image recognition. Neural Networks) may be used, but is not limited thereto, and various embodiments may exist.

According to one embodiment of the present application, the sports relay image recommendation apparatus 100 may calculate the feature values of the sports image data from the plurality of image data using a learning module learned by inputting predetermined sports characteristic information as an input. In other words, the sports relay image recommendation device 100 constructs a learned learning module by inputting a large amount of image data classified as including predetermined sports characteristic information (first sports characteristic information to seventh sports characteristic information). By doing so, the feature values of the sports image data may be calculated from the plurality of image data captured by the plurality of cameras 400. The accuracy of the learning module may increase as a large amount of preset sports characteristic information is acquired.

The sports relay video recommendation apparatus 100 may calculate a feature value of 1 for image data including preset sport characteristic information and a feature value of 0 for image data not including preset sport characteristic information. . In one embodiment of the present application, the feature values are calculated by dividing them by 1 or 0 for convenience of description, but are not limited to the above-described method. The characteristic value may be expressed as a value according to the degree of inclusion, similarity, etc. of the preset sports characteristic information.

The predetermined sports characteristic information may include a plurality of features selected as judged to be major in a sports event. For example, the sports characteristic information may be arbitrarily selected by the user. In addition, the sports characteristic information may extract characteristics that are broadcast through every major event (for example, a foul, goal scoring, etc.) in a sports game through big data analysis. The sports characteristic information described below is exemplified in seven types, but is not limited thereto, and various embodiments may exist.

According to one embodiment of the present application, the preset sports characteristic information includes information related to the preference of a sports player (first sports characteristic information), information related to occupying a ball (second sports characteristic information), and information related to a game viewing terminal (third sports) Characteristic information), information related to the player who attempted to score the ball (4th sports characteristic information), information related to the foul player (5th sports characteristic information), information related to the player when scoring occurred (6th sports characteristic information), and information related to the ball when scoring occurred (7th sports characteristic information).

According to an embodiment of the present application, the learning module includes a sports star's club, issues, game performance, transfer fee, search ranking, sports star's face image, and uniform image included in sports player preference-related information (first sports characteristic information) It can be learned by inputting information such as. The learning module may be learned by inputting face information and uniform information of a plurality of sports players obtained from various angles. The learning module may be learned using a plurality of image information of sports players collected online. In addition, the learning module can be learned about a plurality of uniform information worn by a sports player. Further, the learning module may be learned using information obtained from news, posts, etc. posted online at a preset reference time (eg, within 7 days).

The sports relay image recommendation apparatus 100 uses the learning module learned by inputting various information related to the preference information (first sports characteristic information) of the sports player, and the information related to the preference of the sports player from the received plurality of image data A characteristic value related to (first sports characteristic information) can be calculated.

For example, the sports relay image recommendation apparatus 100 performs face analysis of each player through image analysis on the received plurality of image data, and includes a player whose preference is greater than or equal to a preset value after learning using a learning module The feature value can be calculated as 1 in the image data. In other words, the sports relay video recommendation device 100 may analyze that the first player is included in the first image data and the second player is included in the second image data. The sports relay image recommendation apparatus 100 may calculate the feature value as 1 in the first image data including the first athlete using the analyzed result using the learning module. In other words, since the first player has a higher preference than the second player, the sports relay image recommendation device 100 may calculate a feature value as 1 in the first image data.

In addition, the learning module may be learned by inputting information of a player who occupies a ball in a game such as soccer, volleyball, basketball included in the information related to the possession of the ball (second sports characteristic information). The learning module may be learned using various ball-shaped image data including ball-shaped images used in ball games such as soccer, volleyball, and basketball. The learning module can be learned by using the ball occupancy-related information (second sports characteristic information) in which the ball is included within a certain distance from the player. In addition, in the case of a soccer game, the learning module uses the image data of the ball located within 10 cm from the player's body (eg, foot, head, chest), for information related to the possession of the ball (second sports characteristic information). Information about can be learned.

The sports relay image recommendation apparatus 100 uses the learning module learned by inputting various information related to the ball occupancy-related information (second sports characteristic information), and then receives the ball occupancy-related information (second sports) from the received plurality of image data. Characteristic information).

For example, the sports relay image recommendation apparatus 100 performs image analysis on a player's body part, a ball, etc. through image analysis on the received plurality of image data, learns using a learning module, and then learns about ball possession The feature value may be calculated as 1 in the image data including (second sports characteristic information). For example, the ball tracking method can be applied to various tracking methods that have been developed or are developed in the future. The sports relay image recommendation apparatus 100 may calculate a feature value of the first image data as 1 with respect to a result of performing machine learning of the first image data in which the ball is located within 10 cm from the foot of the athlete among the plurality of image data. On the other hand, the sports relay image recommendation device 100 may calculate the feature value of the second image data as 0 for the result of performing the first machine learning of the second image data including the ball and the player at a position 20 cm from the athlete's foot. have.

In addition, the learning module may be learned by inputting information related to a game viewing terminal (third sports characteristic information) among preset sports characteristic information. The learning module may be learned by inputting a sports player's face, national flag, and sports player's uniform information. The learning module may be learned by inputting information related to a country from a plurality of image data including sports events.

The sports relay video recommendation apparatus 100 uses the learning module learned by inputting various information related to the game viewing terminal-related information (third sports characteristic information), and then receives the game watching terminal-related information (refer to 3 sports characteristic information).

For example, the apparatus 100 for recommending a sports relay video may obtain country information (eg, Korea) of the competition watching terminal from the IP address included in the user terminal 300 currently watching the sports competition. The sports relay video recommendation device 100 is a player of a country associated with an IP address (for example, Korea) included in the user terminal 300 through video analysis from the received plurality of video data (for example, Korea) In the case of image data that includes, a feature value may be calculated as 1. On the other hand, the sports relay video recommendation device 100 does not include a player from a country associated with an IP address (for example, Korea) included in the user terminal 300 (for example, video data including a Japanese player) ) In the case of image data, the feature value can be calculated as 0.

In addition, the learning module may be learned by inputting information related to a player who attempts to score a ball (fourth sports characteristic information). The learning module may be learned by inputting image data including a player who is attempting to score during a ball game such as soccer, volleyball, basketball, and the like. For example, in the case of soccer, the information of the player who is attempting to score may be information including an operation in which the player who occupies the ball attempts to shoot toward the goal. In addition, in the case of basketball, the information of the player who is attempting to score may be information including an operation in which the player who occupies the basketball attempts to shoot toward the goal by raising his hand over his head. The learning module may be learned by inputting image data of a player who is trying various points included in a sports game (for example, information of a player who has shot a goal toward a goal in the case of soccer).

The sports relay image recommendation device 100 uses the learning module learned by inputting various information related to the player attempting to score the ball (fourth sports characteristic information), and the information related to the player attempting to score the ball from the received plurality of image data A characteristic value related to (4th sports characteristic information) can be calculated.

For example, the sports relay image recommendation apparatus 100 may calculate a feature value of the first image data including an operation of a player who occupies a ball among a plurality of received image data attempts to shoot toward the goal 1 have.

In addition, the learning module may be learned by inputting a plurality of foul information included in the foul player related information (the fifth sports characteristic information). The learning module may be learned by inputting information determined as a foul. For example, in the case of soccer, the information determined as a foul may include the case where the ball touches one hand, the foot of the first player falls, and the second player falls. In other words, the learning module can be learned by inputting various foul images.

The sports relay video recommendation apparatus 100 uses the learning module learned by inputting various information related to the foul player-related information (the fifth sports characteristic information), and the foul player-related information (five sports) from the received plurality of image data Characteristic information).

For example, the sports relay image recommendation apparatus 100 may calculate a feature value of the first image data including an operation in which the first player and the second player collide among the received plurality of image data as 1. On the other hand, the sports relay video recommendation device 100 may calculate a feature value for a plurality of video data that does not include information related to a foul player (the fifth sports characteristic information).

In addition, the learning module may be learned by inputting player-related information (sixth sports characteristic information) when scoring occurs. The learning module may be trained using image data in which scoring has occurred. The learning module may be learned using photo information of a plurality of faces obtained from various angles of players currently playing. The learning module may be learned using various ball-shaped image data including ball-shaped images used in ball games such as soccer, volleyball, and basketball. The learning module may be learned by inputting information such as a club, an issue, game score, transfer fee, and search ranking of a sports star included in information related to preferences of a sports player (first sports characteristic information). The learning module may be learned by inputting face information of a plurality of sports players obtained from multiple angles.

The sports relay video recommendation apparatus 100 uses the learning module learned by inputting various information related to player-related information (sixth sports characteristic information) when a score occurs, and the player-related information when a score occurs in a plurality of received image data A characteristic value related to (sixth sports characteristic information) can be calculated. For example, in the case of soccer, the sports relay video recommendation apparatus 100 may be a star player (for example, the player with the highest preference score, the transfer fee) in a situation in which a goal is scored in a goal from a plurality of received video data. The feature value can be calculated on the image data including the location of the highest player, etc., and information on the player who shot the player.

For example, the sports relay image recommendation apparatus 100 may calculate a feature value of the first image data including an operation in which a first player scores a goal among a plurality of received image data. In other words, when the first image data includes the operation of scoring the goal by the first player and the image in which the goal is located in the goal, the sports relay image recommendation device 100 calculates a feature value of the first image data as 1 can do.

In addition, the learning module may be learned by inputting ball-related information (seventh sports characteristic information) when scoring occurs. The learning module may be learned using goal-related information included in an area inside the goalpost. The learning module may be learned using various ball-shaped image data including ball-shaped images used in ball games such as soccer, volleyball, and basketball.

 The sports relay video recommendation device 100 uses the learning module learned by inputting various information related to the ball-related information (seventh sports characteristic information) when a score occurs, and the ball-related information when a score occurs in a plurality of received image data A characteristic value related to (seventh sports characteristic information) can be calculated.

For example, the sports relay image recommendation apparatus 100 may calculate a feature value of 1 related to the ball-related information (seventh sports characteristic information) when a score is generated for the first image data among the received plurality of image data. . In this case, when ball-related information (seventh sports characteristic information) is included in the first image data when a score is generated, the feature value may be calculated as 1.

3 is a view for explaining the results of the first machine learning performance of the sports relay image recommendation apparatus according to an embodiment of the present application.

Referring to FIG. 3, a description will be given of a process of performing a first machine learning for calculating a feature value of predetermined sports characteristic information for a plurality of image data received from the sports relay image recommendation apparatus 100.

For example, referring to FIG. 3, the sports relay video recommendation device 100 may include first image data to 20th image data 1 'to 20' captured from each of the first camera to the 20th camera 1 to 20 ), The characteristic value may be calculated by performing the first machine learning on the predetermined sports characteristic information (first sports characteristic information to seventh sports characteristic information). For example, the sports relay image recommendation apparatus 100 calculates feature values of the first to seventh sports characteristic information described above with respect to the first to fourth video data illustrated in FIG. 2. Can perform primary machine learning.

For example, the sports relay image recommendation apparatus 100 may set predetermined sports characteristic information (first sports characteristic information to seventh sports characteristic information) with respect to the first image data 1 'shown in FIG. 2A. ). The sports relay video recommendation apparatus 100 calculates a feature value as 1 when a feature related to preset sports characteristic information (first sports characteristic information to seventh sports characteristic information) is included, and preset sports characteristic information (second If the feature related to 1 sports characteristic information to the 7th sports characteristic information) is not included, the characteristic value may be calculated as 0.

2 and 3, the sports relay video recommendation device 100, for the first image data (1 ') captured by the first camera (1), information related to the foul player (5th sports characteristics It can be determined that it contains all the sports characteristic information except information), and the characteristic value of all the sports characteristic information except the foul player-related information (the fifth sports characteristic information) can be calculated as 1.

In addition, as illustrated in b of FIGS. 2 and 3, the apparatus 100 for recommending a sports relay video, regarding the second image data 2 ′ captured by the second camera 2, information related to the preference of the sports player ( 1st sports characteristic information), match viewing terminal related information (3rd sports characteristic information), foul player related information (5th sports characteristic information), player related information when scoring occurs (6th sports characteristic information) and ball when scoring occurs The characteristic value of the related information (seventh sports characteristic information) can be calculated as 1.

In addition, as shown in c shown in FIGS. 2 and 3, the sports relay image recommendation device 100 performs sports characteristic information (first sports) for the third image data 3 'captured by the third camera 3 The characteristic values of the characteristic information to the seventh sports characteristic information) may be calculated as 1.

In addition, as shown in d of FIGS. 2 and 3, the sports relay video recommendation device 100 performs sports characteristic information (first sports) with respect to the fourth video data 4 ′ captured by the fourth camera 4 It does not include all of the characteristic information to the seventh sports characteristic information), and the characteristic value can be calculated as 0. As an example, the third image data 3 'is data obtained by photographing a large area of a stadium compared to the first image data 1' and the second image data 2 ', and various sports compared to other image data The characteristic value can be calculated by including the characteristic information. On the other hand, the fourth image data 4 'is data obtained by photographing an audience seat, and since it does not include image information related to a sports player and a ball, it may be that a feature value related to predetermined sports characteristic information is not calculated. .

According to one embodiment of the present application, the sports relay image recommendation apparatus 100 uses the result of the first machine learning to perform secondary machine learning to extract any one of a plurality of image data as the user preferred sports image data 500 You can do The result of the first machine learning may be a result of calculating feature values related to predetermined sport characteristic information. According to one embodiment of the present application, the sports relay image recommendation device 100 prefers any one of the plurality of image data through secondary machine learning to input sports characteristic information included in each of the input multiple image data It can be determined as sports image data. The sports relay video recommendation apparatus 100 may use any one of a plurality of video data as a user-preferred sports video data through black box-based secondary machine learning that inputs sports characteristic information included in each of the inputted plurality of video data. Can decide. Secondary machine learning can use SVM (Support Vector Machine) algorithm. The SVM (Support Vector Machine) algorithm is one of the fields of machine learning, and is a supervised learning model for pattern recognition and data analysis, and may be an algorithm mainly used for classification and regression analysis.

The sports relay image recommendation apparatus 100 extracts any one of a plurality of image data (1 'to 20') into the user's preferred sport image data 500 using the result of the primary machine learning as shown in FIG. Secondary machine learning can be performed. In other words, the sports relay image recommendation apparatus 100 may perform training by attaching each label to the second machine learning using the result of the first machine learning.

The sports relay image recommendation apparatus 100 may recommend the user preferred sports image data 500 extracted as a result of the second machine learning. The user-preferred sports image data 500 may be a result of the secondary machine learning extracted by predicting that the user wants the most in a specific situation.

The sports relay image recommendation apparatus 100 may provide the user preferred sports image data 500 extracted as a result of the second machine learning to the user terminal 300. The user terminal 300 may receive user-preferred sports image data transmitted from the sports relay image recommendation apparatus 100 and display (output) the display on the display.

According to one embodiment of the present application, the sports relay image recommendation apparatus 100 may receive user feedback regarding the recommended user preferred sports image data 500. The user feedback may be feedback regarding whether or not to maintain the corresponding image received from the user terminal 300, after providing the user preferred sports image data 500 to the user terminal 300. For example, the sports relay image recommendation apparatus 100 provides the recommended user preferred sports image data 500 to the user terminal 300, but the user terminal 300 is different from the user preferred sports image data 500. It is possible to receive input information for viewing sports video data.

As an example, referring to FIG. 2, the sports relay video recommendation apparatus 100 recommends the user preferred sports video data as the first video data 1 ′, but the user wants to watch the second video data 2 ′. Input information can be received. In addition, the sports relay video recommendation device 100 provides the recommended user preferred sports video data 500 to the user terminal 300, but the user within the preset time (for example, 5 seconds) in the user terminal 300 It is possible to receive input information for viewing sports video data other than the preferred sports video data 500. When receiving the user feedback described above, the sports relay image recommendation apparatus 100 may determine that the user does not prefer the recommended user preferred sports image data. At this time, the sports relay image recommendation apparatus 100 may perform secondary machine learning by reflecting user feedback and re-extract the user preferred sports image data.

According to one embodiment of the present application, the sports relay video recommendation device 100 may collect user information. The user information may be information including gender, age, age, country, and the like. In order to provide more accurate user-preferred sports video data, the sports relay video recommendation device 100 performs secondary machine learning by inputting the collected user information and the results of the first machine learning and extracts user-preferred sports image data can do. The sports relay video recommendation apparatus 100 may extract user-preferred sports video data by inputting user information and results of primary machine learning as inputs, thereby providing accurate user-preferred sports video data to each user.

According to one embodiment of the present application, The sports relay video recommendation device 100 may decode all of the encoded video data received from the video encoding device 200 and transmit only the video data extracted as the user's preferred sports video data to the user terminal. Also, the apparatus 100 for recommending a sports relay image may decode only the extracted image data as the user preferred sports image data.

4 is a schematic block diagram of a sports relay video recommendation apparatus according to an embodiment of the present application.

Referring to FIG. 4, the sports relay image recommendation apparatus 100 includes an image data receiving unit 110, a first machine learning unit 120, a second machine learning unit 130, a preferred image recommendation unit 140, and a feedback receiving unit 150 and a user information collection unit 160.

According to an embodiment of the present application, the image data receiving unit 110 may receive a plurality of image data photographed by the plurality of image cameras 400. A plurality of image data captured by the plurality of image cameras 400 may be received. The image data receiving unit 110 may receive a plurality of image data captured by each of the plurality of image cameras 400. A plurality of image data The characteristics of image data, a viewpoint, a location of the image camera, and the like may be different image data. As an example, the image data receiving unit 110 may receive a plurality of image data encoded by the image encoding apparatus 200.

Also, the image data receiving unit 110 may receive a bit stream of image data of a plurality of images photographed by the plurality of image cameras 400. As an example, the image data receiving unit 110 may receive a bit stream related to image data of a plurality of images photographed by the plurality of image cameras 400 from an encoding device (not shown) of multi-view image information. You can. The image encoding apparatus 200 may convert image data of a plurality of images received from the plurality of image cameras 400 into a bit stream format. The plurality of images may be images obtained from a plurality of cameras 400 provided at different locations. In other words, a plurality of images are taken by one object, but may be images taken at different locations. That is, a plurality of images may be images of the same object at the same time.

The bit stream is stored according to an encoding type that satisfies the constraints of the encoder. MPEG requires a syntax (syntax, hereinafter referred to as 'syntax') and semantics (hereinafter referred to as 'semantics') as constraints of a bit stream. The syntax indicates the structure, format, and length of data, and indicates in what order the data is expressed. That is, syntax is for matching the grammar for encoding / decoding, and defines the order of each element included in the bit stream, the length of each element, and the data format. Semantics indicates the meaning of each bit constituting data. In other words, Semantics indicates what the meaning of each element in the bit stream is. Accordingly, various types of bit streams may be generated according to an encoding condition of an encoder or an applied standard (or codec). In general, each standard (eg, MPEG-1, MPEG-2, MPEG-4, MPEG-4 AVC, etc.) has a different bit stream syntax. Therefore, it can be said that the bit streams encoded according to each standard or encoding condition have different formats (ie, syntax and semantics), and a decoder corresponding to the encoder must be used to decode the bit stream.

The first machine learning unit 120 may perform the first machine learning for calculating a feature value related to predetermined sport characteristic information on the received plurality of image data. The first machine learning unit 120 may calculate a feature value of sports image data from a plurality of image data using a learning module learned by inputting predetermined sports characteristic information as an input. The learning module can be continuously learned through a plurality of received image data. The learning module can be learned using an artificial neural network.

According to one embodiment of the present application, the preset sports characteristic information includes information related to the preference of a sports player (first sports characteristic information), information related to occupying a ball (second sports characteristic information), and information related to a game viewing terminal (third sports) Characteristic information), information related to the player who attempted to score the ball (4th sports characteristic information), information related to the foul player (5th sports characteristic information), information related to the player when scoring occurred (6th sports characteristic information), and information related to the ball when scoring occurred (7th sports characteristic information).

In other words, the learning module may be constructed through learning using the first sports characteristic information to the seventh sports characteristic information as inputs. The first machine learning unit 120 may calculate a feature value of sports image data by using a plurality of image data as inputs of a learning module.

The preset sports characteristic information is the same as the sports characteristic information described above, and may be applied in the same manner even if omitted.

The first machine learning unit 120 uses a plurality of image data including a sports match in relation to information related to the preference of the sports player (first sports characteristic information), and uses the image data including the player with the highest preference. Learn about preference-related scenes. Then, based on the learning result, it is possible to calculate a feature value related to preference information of a sports player from the received plurality of image data. The first machine learning unit 120 may calculate the feature value of the image data as 1 when the first image includes information related to the preference of the sports player (first sports characteristic information). On the other hand, if the first machine learning unit 120 does not include information related to the preference of the sports player (first sports characteristic information) in the first image, the feature value of the image data may be calculated as 0.

In addition, the first machine learning unit 120 uses the image data including the player who occupies the ball from a plurality of image data including a sports event in relation to the ball occupancy-related information (second sports characteristic information). Learn about scenes related to player preferences. Subsequently, based on the learning result, it is possible to calculate a feature value related to the ball occupancy related information of a sports player from a plurality of received image data. The first machine learning unit 120 may calculate the feature value of the image data as 1 when the first image includes the information related to occupancy (second sports characteristic information). On the other hand, the first machine learning unit 120 may calculate the feature value of the image data as zero occupancy related information (second sports characteristic information) is not included in the first image.

In addition, the first machine learning unit 120 uses the image data including the nationality of the athlete from a plurality of image data including a sports event in relation to the game viewing terminal related information (third sports characteristic information). Learn about nationality-related scenes. Subsequently, based on the learning result, a feature value may be calculated from the video data including the player corresponding to the IP address included in the user terminal 300 from the received plurality of video data. The first machine learning unit 120 may calculate the feature value of the video data as 1 when the first video includes information related to the game viewing terminal (third sports characteristic information). On the other hand, the first machine learning unit 120 may calculate the feature value of the image data as 0 when the information related to the game viewing terminal (third sports characteristic information) is not included in the first image.

In addition, the first machine learning unit 120 attempts to score a ball by using image data determined as a score from a plurality of image data including a sports match in relation to information related to a player attempting to score a ball (fourth sports characteristic information) Learn about. Then, based on the learning result, it is possible to calculate a feature value related to information related to a player who attempts to score a ball from a plurality of received image data. The first machine learning unit 120 may calculate the feature value of the image data as 1 when the first image includes information related to the player attempting to score the ball (fourth sports characteristic information). On the other hand, the first machine learning unit 120 may calculate the feature value of the image data when the first image does not include information related to the player attempting to score the ball (fourth sports characteristic information).

In addition, the first machine learning unit 120 learns about the scene of the foul using the image data determined to be a foul from a plurality of image data including a sports match in relation to the information related to the foul player (the fifth sports characteristic information). . Subsequently, based on the learning result, feature values related to foul-related information may be calculated from the received plurality of image data. The first machine learning unit 120 may calculate the feature value of the image data as 1 when the first image includes information related to the foul player (the fifth sports characteristic information). On the other hand, if the first machine learning unit 120 does not include the information related to the foul player (the fifth sports characteristic information) in the first image, the feature value of the image data may be calculated as 0.

In addition, when the first machine learning unit 120 scores from a plurality of image data including a sports event in relation to player-related information (sixth sports characteristic information) when a score occurs, the image data including the position of the players Use to learn about the player-related scenes when scoring occurs. Then, based on the learning result, when a score occurs in the received plurality of image data, it is possible to calculate a feature value associated with information about the players of both teams and the player who shot the information. The first machine learning unit 120 may calculate a feature value of image data when the first image includes player-related information (sixth sports characteristic information) when scoring occurs. On the other hand, if the first machine learning unit 120 does not include player-related information (sixth sports characteristic information) when a score is generated in the first image, the feature value of the image data may be calculated as zero.

In addition, the first machine learning unit 120 uses the image data including the position of the ball when scoring occurs from a plurality of image data including a sports event in relation to the ball-related information (seventh sports characteristic information) when the scoring occurs. So, learn about the ball-related scenes when scoring occurs. Thereafter, based on the learning result, a feature value associated with the ball may be calculated when a score is generated from the received plurality of image data. The first machine learning unit 120 may calculate the feature value of the image data as 1 when ball-related information (seventh sports characteristic information) is included in the first image. On the other hand, when the first machine learning unit 120 does not include ball-related information (seventh sports characteristic information) when scoring occurs, the feature value of the image data may be calculated as zero.

According to one embodiment of the present application, the first machine learning unit 120 may be learned using an artificial neural network. For example, the artificial neural network may be a deep learning neural network, but is not limited thereto, and various neural network systems developed or developed in the future may be applied.

The second machine learning unit 130 may perform secondary machine learning to extract any one of a plurality of image data as the user preferred sports image data 500 by using the result of the first machine learning. Secondary machine learning can use SVM (Support Vector Machine) algorithm. The user-preferred sports image data 500 may be a result of secondary machine learning extracted by the user expecting the most desired.

The preferred image recommendation unit 140 may recommend the extracted user preferred sports image data. The preferred image recommendation unit 140 may provide the extracted user preferred sports image data 500 to the user terminal 300. The user terminal 300 may output the provided user preferred sports image data 500 through a display.

For example, the user terminal 300 is a device that is linked to the sports relay video recommendation device 100 through a network, for example, a smart phone, a smart pad, a tablet PC, a wearable device, and the like PCS. (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication) -2000, CDMA (Code Division Multiple) Access) -2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet), and all kinds of wireless communication devices and desktop computers, fixed terminals such as smart TVs.

The feedback receiving unit 150 may receive user feedback regarding recommended user preferred sports image data. The feedback receiving unit 150 may receive user feedback regarding the user preferred sports image data recommended by the user terminal 300. The feedback receiver 150 may determine that the user does not prefer the video when the video output by the user preferred sports video data provided to the user terminal 300 is changed to another video within a preset time. When the second machine learning unit 130 receives a user selection input for at least one of a plurality of image data other than the recommended user preferred sports image data, the feedback receiving unit 150 reflects the information as feedback 2 Secondary machine learning can be performed to re-extract user preferred sports image data.

Referring to FIG. 2 as an example, when an image related to goal scoring is provided while a soccer game is in progress, the secondary machine learning unit 130 extracts the first image data 1 'as user preferred sport image data. By doing so, it is possible to recommend the first image data 1 'to the user terminal 300. At this time, when the feedback receiving unit 150 receives an input for viewing the third image data 3 'instead of the first image data 1' from the user terminal 300, the secondary machine learning unit 130 Secondary machine learning can be performed again by reflecting the input information as user feedback information. The secondary machine learning unit 130 reflects the user feedback received from the feedback receiving unit 150 to perform secondary machine learning, so that the user can grasp which image is preferred in a specific situation.

The user information collection unit 160 may collect user information. The user information may be information including age, age, gender, country, preferred sports, preferred sports players, preferred sports teams, and the like. The user information collection unit 160 may receive user information from the user terminal 300.

According to an embodiment of the present application, the user information collection unit 160 may provide a user information acquisition menu to the user terminal 300. For example, the user terminal 300 downloads and installs the application program provided by the user information collection unit 160, and a user information acquisition menu may be provided through the installed application. The user information collection unit 160 may transmit and receive data, contents, and various communication signals to and from the user terminal 300 through a network.

The secondary machine learning unit 130 may perform secondary machine learning by inputting the user information collected by the user information collection unit 160 and the result of the primary machine learning. The secondary machine learning unit 130 performs secondary machine learning by inputting the collected user information and the result of the primary machine learning as input, rather than the result of the secondary machine learning performed using only the results of the primary machine learning. More accurate user preference sports image data can be extracted.

 Hereinafter, based on the details described above, the operation flow of the present application will be briefly described.

5 is an operation flowchart of a method for recommending a sports relay video based on machine learning according to an embodiment of the present application.

The method for recommending a sports relay image based on machine learning shown in FIG. 5 may be performed by the apparatus 100 for recommending a sports relay image based on machine learning described above. Accordingly, even if omitted, the description of the apparatus 100 for recommending a sports relay video based on machine learning may be applied to the description of the method for recommending a sports relay video based on machine learning.

In step S501, the sports relay image recommendation apparatus 100 may receive a plurality of image data captured by a plurality of image cameras.

In step S502, the sports relay image recommendation apparatus 100 may perform primary machine learning for calculating a feature value related to predetermined sports characteristic information, with respect to the received plurality of image data.

In step S503, the sports relay image recommendation apparatus 100 may perform secondary machine learning for extracting any one of a plurality of image data as user-preferred sports image data using the result of the first machine learning.

In step S504, the sports relay video recommendation device 100 may recommend the extracted user preferred sports video data.

In the above description, steps S501 to S504 may be further divided into additional steps, or combined into fewer steps, according to embodiments herein. In addition, some steps may be omitted if necessary, and the order between the steps may be changed.

The method for recommending a sports relay image based on machine learning according to an embodiment of the present application may be implemented in a form of program instructions that can be executed through various computer means and may be recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

In addition, the above-described method for recommending a sports relay image based on machine learning may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The above description of the present application is for illustrative purposes, and those skilled in the art to which the present application pertains will understand that it is possible to easily modify to other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the claims below, rather than the detailed description, and it should be interpreted that all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present application.

100: sports relay video recommendation device
110: image data receiving unit
120: first machine learning department
130: second machine learning department
140: preferred video recommendation unit
150: feedback receiver
160: user information receiving unit
200: video encoding device
300: user terminal
400: multiple cameras

Claims (12)

In the method of recommending a sports relay video based on machine learning,
Receiving a plurality of image data photographed by a plurality of image cameras arranged in a stadium in which a sports event is in progress;
Performing primary machine learning on the received plurality of image data to calculate feature values related to predetermined sport characteristic information;
Collecting user information;
Performing secondary machine learning to extract any one of the plurality of image data as user-preferred sports image data using the collected user information and the results of the primary machine learning; And
Recommending the extracted user preferred sports image data;
Including,
The step of performing the first machine learning,
Shooting by each of the plurality of video cameras by applying the plurality of video data captured and received by each of the plurality of video cameras to a learning module learned based on a plurality of video data including preset sport characteristic information Performing the first machine learning to calculate the feature value for determining whether or not the sports characteristic information is included in each received image data,
The step of performing the second machine learning,
Performing a second machine learning using the collected user information and the result of the first machine learning as input, and any of a plurality of image data captured and received by each of the plurality of image cameras through the second machine learning One is extracted as user preferred sports video data,
The predetermined sports characteristic information,
Sports that include information related to the preferences of sports players, information related to possession of a ball, information related to a game watching terminal, information related to a player attempting to score a ball, information related to a player attempting a foul, information related to a player when a goal occurs, and information related to a ball when a goal occurs How to recommend relay video. According to claim 1,
The step of performing the first machine learning,
A method for recommending a sports relay image, wherein the feature value of the sports image data is calculated from the plurality of image data using a learning module learned by using the preset sports characteristic information as an input. According to claim 1,
Receiving user feedback regarding the recommended user preferred sports video data,
Further comprising,
The step of performing the second machine learning is to extract user preferred sports image data by reflecting the user feedback, a method for recommending sports relay images. delete According to claim 1,
The second machine learning is using a support vector machine (SVM) algorithm, a method for recommending a sports relay video. delete In the machine learning-based sports relay video recommendation device,
An image data receiving unit configured to receive a plurality of image data photographed by a plurality of image cameras disposed in a stadium in which a sports event is in progress;
A first machine learning unit for performing a first machine learning for calculating a feature value of predetermined sports characteristic information on the received plurality of image data;
A user information collection unit for collecting user information;
A second machine learning unit performing secondary machine learning to extract any one of the plurality of image data as user preferred sport image data using the collected user information and the results of the first machine learning; And
A preference image recommendation unit for recommending the extracted user preference sports image data;
Including,
The first machine learning unit,
Shooting by each of the plurality of video cameras by applying the plurality of video data captured and received by each of the plurality of video cameras to a learning module learned based on a plurality of video data including preset sport characteristic information Performing the first machine learning to calculate the feature value for determining whether or not the sports characteristic information is included in each received image data,
The second machine learning unit,
Performing a second machine learning using the collected user information and the result of the first machine learning as input, and any one of a plurality of image data captured and received by each of the plurality of image cameras through the second machine learning Is extracted as user preferred sports video data,
The predetermined sports characteristic information,
Sports that include information related to the preferences of sports players, information related to possession of a ball, information related to a player watching a game, information related to a player attempting to score a ball, information related to a foul player, information related to a player when a score occurs, and information related to a ball when a score occurs Relay video recommendation device. The method of claim 7,
The first machine learning unit,
A sports relay image recommendation apparatus for calculating a feature value of the sports image data from the plurality of image data using a learning module learned by inputting the predetermined sports characteristic information as an input. The method of claim 7,
A feedback receiver configured to receive user feedback on the recommended user preferred sports video data,
Further comprising,
The second machine learning unit extracts user preferred sports image data by reflecting the user feedback. delete A video encoding apparatus for receiving a plurality of video data from a plurality of video cameras arranged in a stadium in which a sports event is in progress, and transmitting the received plurality of video data; And
Performs first machine learning to receive encoded image data from the image encoding device and calculate feature values related to predetermined sports characteristic information, and collects and collects user information on the received plurality of image data Using the extracted user information and the result of the first machine learning, performing the second machine learning to extract any one of the plurality of image data as user preferred sports image data, and recommending the extracted user preferred sports image data Sports relay video recommendation device,
Including,
The sports relay video recommendation device,
Shooting by each of the plurality of video cameras by applying the plurality of video data captured and received by each of the plurality of video cameras to a learning module learned based on a plurality of video data including preset sport characteristic information Secondary machine learning, which performs primary machine learning to calculate feature values for determining whether sports characteristic information is included in each received image data, and inputs the collected user information and the primary machine learning results as input. And extracting any one of a plurality of image data captured and received by each of the plurality of image cameras through the second machine learning as user preferred sports image data,
The predetermined sports characteristic information,
Sports that include information related to the preferences of sports players, information related to possession of a ball, information related to a player watching a game, information related to a player attempting to score a ball, information related to a foul player, information related to a player when a score occurs, and information related to a ball when a score occurs Relay video recommendation system. A computer-readable recording medium recording a program for executing the method of any one of claims 1 to 3 and 5 in a computer.

Images