CN109101911B - Visual analysis method for football match formation change and personnel flow - Google Patents
Visual analysis method for football match formation change and personnel flow Download PDFInfo
- Publication number
- CN109101911B CN109101911B CN201810858815.1A CN201810858815A CN109101911B CN 109101911 B CN109101911 B CN 109101911B CN 201810858815 A CN201810858815 A CN 201810858815A CN 109101911 B CN109101911 B CN 109101911B
- Authority
- CN
- China
- Prior art keywords
- data
- court
- formation
- player
- players
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
Abstract
The invention provides a visual analysis method for the formation change and the personnel flow of a football match, which comprises the following steps: (1) acquiring data, wherein the data comprises player position data, football position data, court occurrence data and ball control information; (2) establishing a two-dimensional court model, projecting the player position data in the step (1) into the court, and smoothing the position data; (3) clustering the player position data in the step (1) to obtain overall array information; (4) visualizing the data obtained in the steps (1), (2) and (3), wherein the visualized images comprise formation visualization, match real-time data visualization and match event visualization; the invention provides a football data acquisition mode and a preprocessing method, provides analysis data support for relevant experts of football analysis, and visualizes the formation data, thereby being convenient for finding the characteristics of formation change.
Description
Technical Field
The invention relates to a football match data analysis method, in particular to a visual analysis method for football match formation change and personnel flow.
Background
Computer vision technology has been widely used in sports games over the last two decades, such as eagle eye technology in tennis games, to quickly reconstruct the drop point of a tennis ball moving at high speed and to show it with three-dimensional animation. Likewise, many computer vision studies have also been discussed for soccer games. However, because the football match has particularity and the background color of the scene is monotonous, the traditional method firstly cuts out the green background of the court and then detects the players according to the rest part. Common video data are from live television, so some researches focus on detecting the positions of players who live television and projecting the positions to a two-dimensional court through a homography matrix. The main problem is that the live television always focuses on the players near the ball, and the live video can lose a large amount of credits. Much of the computer vision work in recent years has focused on deep learning, such as pedestrian detection. For the players in the football game, the detection method is feasible, but the data is special, the proportion of the players in the panoramic video is very small, and a neural network loses a large number of player characteristics in the process of down-sampling, so that the detection effect is poor.
Visualization techniques have been widely used in the past years in sports analysis, such as table tennis, basketball, tennis, hockey, etc. In the aspect of soccer, visualization technology is also applied to various visual analysis systems. For example, there are visualization tasks that integrate the player's motion trajectory into the game video, there are visualization systems that use novel views to view the football game at different game stages, there are also visualization tasks directed at the ball or the amount of compression the player is subjected to, and so on. These efforts have all improved the effectiveness and efficiency of soccer analysis from different aspects.
However, in the visualization work of these soccer, there is little concern about the team's formation, the most important part of the team's tactics.
Disclosure of Invention
The invention provides a visual analysis method for the formation change and the personnel flow of a football match, provides a data acquisition mode and a preprocessing method, provides analytical data support for experts related to football analysis, visualizes the formation data, and can quickly find the characteristics of the formation change.
A visual analysis method for the formation change and the personnel flow of a football match comprises the following steps:
(1) acquiring data, wherein the data comprises player position data, football position data, court occurrence data and ball control information;
(2) establishing a two-dimensional court model, projecting the player position data in the step (1) into the court, and smoothing the position data;
(3) clustering the player position data in the step (1) to obtain overall array information;
(4) and (3) visualizing the data obtained in the steps (1), (2) and (3), wherein the visualized images comprise formation visualization, match real-time data visualization and match event visualization.
Preferably, in the step (1), the position data of two players on the football field is collected by using a man-machine interaction mode: selecting a player on the scene by using a method of combining particle filtering with a color histogram, and automatically tracking the selected player; when positions of a plurality of players are overlapped and staggered, tracking errors are generated, and the positions of the players on the field are collected again by combining an artificial correction method.
Preferably, in the step (2), the two-dimensional court data is based on the size of a real court, a coordinate system is established by taking one corner of the court as an origin, a mapping relation between the court in the panoramic video and the two-dimensional court model is established, and a series of key points p on the court are defined1，…，pkAnd target point d1，…，dkThe X and y coordinates of which are X ═ X1，x2，…，xk) And Y ═ Y1，y2，…，yk) The homography matrix mapping is established as follows:
in order to optimize the visualization effect, preferably, in step (2), the smoothing process uses an efficient linear filter to perform an average filtering process on the time-series position information to remove the position jitter between frames due to random errors.
Preferably, in step (3), the specific process of obtaining the overall array information is as follows:
the positions of each player on the court can be clustered by applying a Gaussian mixture model and combining a k-means algorithm, and the specific algorithm is as follows:
defining the position information of the players of each frame of a certain team as p0，1，p0，2，…，p0，11Centralizing the player information, and subtracting the average position of all players in the frame from each position information to obtain the player information p of each frame in a certain team after centralization1，p2，…，p11；
Assuming that the formation of the team has stability, even if a player moves to other positions, the player usually carries out position compensation by the team friends, so that the movement range of the player at one position in the formation conforms to the Gaussian distribution, and the specific algorithm is as follows:
3-1, initializing Gaussian distribution, taking the positions of the players in the first frame as the mean value of the initial distribution, and adopting a random number generation method for the variance of the distribution;
3-2, traversing the next frame, and calculating the cost c of the ith player distributed to the jth positioni，j＝-log(pi，j) Wherein p isi，jThe size of the probability density function of the ith player under the condition that the jth player belongs to the jth position; then if a player belongs to a location, the greater its probability density function in the distribution, the smaller the value of the cost function. The problem then translates into a distribution problem, distributing 11 players, so that the total cost is minimized. A Hungarian algorithm is applied in the calculation process, and the result is calculated efficiently;
3-3, updating distribution information, calculating the mean and variance of the positions of the players belonging to a certain distribution, and using the mean and variance to represent the expectation and variance of the distribution as the basis of the next frame iteration;
3-4, traversing each frame, and classifying the players of each frame respectively to obtain a distribution result of the first iteration, wherein each distribution comprises expectation and variance;
3-5, taking the result of the previous iteration as the initial value of the second iteration, and traversing each frame to obtain a new iteration result;
and 3-6, carrying out one round of iteration until the distribution is stable, wherein the difference is smaller than a threshold value.
Preferably, the formation visualization takes the form of a sang-based chart to represent the variation of the formation and the people flow information of the players throughout the game.
Preferably, the matrix visualization: the stream view is used for visualizing the change of the formation in the normal match and can show the flowing condition of the personnel in the formation. Also included in the flow view is a timeline that shows key events in a normal game, such as a change, yellow, red, goal.
Preferably, real-time match data is visualized, and data changes along a time line field are displayed, wherein the data changes comprise the running distance, the array coverage area, the pass success rate, the ball control time and the like of team members on the field; the real-time match data visualization can triangulate the formation structure of the player and re-cover the original video in real time, so that the video analysis by a coach is facilitated.
Preferably, the game event is visualized, the key event is displayed on the time line, the goal event is displayed in the event list, and the goal video can be played by clicking the goal event; the mouse is used for moving on the time line, so that the corresponding time point of the video can be jumped to in real time and played.
The invention has the beneficial effects that:
the visual analysis method for the formation change and the personnel flow of the football match provides a collection mode and a pretreatment method aiming at football data, provides analytical data support for experts related to football analysis, and visualizes the formation data, thereby being convenient for finding the characteristics of the formation change.
Drawings
Fig. 1 is a flow chart of the visual analysis method for the formation change and the personnel flow of the football game according to the invention.
Fig. 2 is a format diagram of an original video before processing in part a of fig. 1.
Fig. 3 is a schematic diagram of the effect of the player tracking algorithm in section B of fig. 1.
Fig. 4 is a schematic diagram of a two-dimensional mapping of the player positions obtained in fig. 3.
FIG. 5 is a schematic diagram of iterative convergence of clustering the matrix in the method of the present invention.
FIG. 6 is a schematic diagram of the correlation between the array overview detected by the algorithm and the actual array in the method of the present invention.
Detailed Description
As shown in FIGS. 1-6, the visual analysis method for football match formation change and personnel flow of the embodiment provides data support for analysis of relevant experts of football analysis aiming at the visual analysis work of football match data, particularly formation change, and simultaneously visualizes the formation data, thereby being convenient for finding the characteristics of formation change.
The visual analysis method for the football match formation change and the personnel flow comprises the following steps:
(1) and acquiring data, wherein the data comprises player position data, football position data, court occurrence data and ball control information. The player position data and the football position data have x, V and t attributes and respectively represent two-dimensional coordinates and time information of the players in the video. The ball control information is information of players of each frame of ball control, and the players start to calculate after obtaining the ball until the ball is obtained by another player;
the position data of 22 players on the football field are collected by using a man-machine interaction mode: selecting a player on the scene by using a method of combining particle filtering with a color histogram, and automatically tracking the selected player by a machine; when the positions of a plurality of players are overlapped and staggered, the machine generates tracking errors, and the positions of the players on the field are collected again by combining an artificial correction method. In addition, a data acquisition unit is redesigned, so that the ball right and court data can be acquired in the real-time competition process, and the data acquisition efficiency is improved;
(2) establishing a two-dimensional court model, projecting the match position data of players into the court, and smoothing the position data;
the two-dimensional court data is based on the size of a real court, a coordinate system is established by taking one corner of the court as an original point, a mapping relation between the court and a two-dimensional court model in the panoramic video is established, and a series of key points p on the court are defined firstly1，…，pkAnd target point d1，…，dkThe X and y coordinates of which are X ═ X1，x2，…，xk) And Y ═ Y1，y2，…，yk). The homography matrix mapping is established as follows:
and smoothing the two-dimensional position information obtained after mapping, wherein an algorithm applies a high-efficiency linear filter to perform mean filtering on the position information of the time sequence, so that position jitter generated by random errors between frames is eliminated, and the visualization effect is optimized.
(3) A formation data generation algorithm, which is used for clustering the player position information to obtain overall formation information;
the array type data generation algorithm utilizes a Gaussian mixture model and a k-means algorithm to cluster the position of each player on the court;
firstly, defining the position information of each player in a certain team as p0，1，p0，2，…，p0，11Centralizing the player information, and subtracting the average position of all players in the frame from each position information to obtain the player information p of each frame in a certain team after centralization1，p2，…，p11. Assuming that the formation of the team has stability, even if a player moves to other positions, the player usually carries out position compensation by the team friends, so that the movement range of the player at one position in the formation conforms to the Gaussian distribution, and the specific algorithm is as follows:
and 3-1, initializing Gaussian distribution, taking the position of the player in the first frame as the mean value of the initial distribution, and adopting a random number generation method for the variance of the distribution.
3-2, traversing the next frame, and calculating the cost c of the ith player distributed to the jth positioni，j＝-log(Pi，j) Wherein p isi，jThe size of the probability density function for the case where the player belongs to the jth position. Then if a player belongs to a location, the greater its probability density function in the distribution, the smaller the value of the cost function. The problem then translates into a distribution problem, distributing 11 players, so that the total cost is minimized. A Hungarian algorithm is applied in the calculation process, and the result is calculated efficiently;
and 3-3, updating the distribution information. Calculating the mean and variance of the positions of the players belonging to a certain distribution to represent the expectation and variance of the distribution, and using the expectation and variance as the basis of the iteration of the next frame;
3-4, traversing each frame, and classifying the players of each frame respectively to obtain a distribution result of the first iteration, wherein each distribution comprises expectation and variance;
3-5, taking the result of the previous iteration as the initial value of the second iteration, and traversing each frame to obtain a new iteration result;
and 3-6, carrying out one round of iteration until the distribution is stable, wherein the difference is smaller than a threshold value.
(4) And (3) visualizing the data obtained in the steps (1), (2) and (3), wherein the visualized images mainly comprise formation visualization, match real-time data visualization and match event visualization. The formation visualization mainly utilizes the form of the sang-based chart to represent the variation of the formation and the staff flow information of the players in the whole game. The design of the visualization system contains three views:
visualization of the array type: the stream view is used for visualizing the change of the formation in the normal match and can show the flowing condition of the personnel in the formation. Also included in the flow view is a timeline that shows key events in a normal game, such as a change, yellow, red, goal.
Visualization of real-time data of the match: showing data changes along a time line field, including running distance, formation coverage area, pass success rate, ball control time and the like of team members on the field;
visualization of game events: displaying the key events on the time line, displaying the goal events in the event list, and clicking the goal events to play a goal video; the mouse is used for moving on a time line, so that the corresponding time point of the video can be skipped to in real time and played;
the real-time match data visualization method can triangulate the formation structure of the player and re-cover the formation structure on the original video in real time, and is more beneficial for a coach to analyze the video.
Claims (4)
1. A visual analysis method for the formation change and the personnel flow of a football match is characterized by comprising the following steps:
(1) acquiring data, wherein the data comprises player position data, football position data, court occurrence data and ball control information;
(2) establishing a two-dimensional court model, projecting the player position data in the step (1) into the court, and smoothing the position data;
(3) clustering the player position data in the step (1) to obtain overall array information;
(4) visualizing the data obtained in the steps (1), (2) and (3), wherein the visualized images comprise formation visualization, match real-time data visualization and match event visualization;
in the step (1), the position data of two players on the football field is collected by using a man-machine interaction mode: selecting a player on the scene by using a method of combining particle filtering with a color histogram, and automatically tracking the selected player; when positions of a plurality of players are overlapped and staggered, tracking errors are generated, and the positions of the players on the field are collected again by combining an artificial correction method;
in the step (2), the two-dimensional court data is based on the size of a real court, a coordinate system is established by taking one corner of the court as an original point, a mapping relation between the court in the panoramic video and the two-dimensional court model is established, and a series of key points p on the court are defined1,…,pkAnd target point d1,…,dkThe X and y coordinates of which are X ═ X1,x2,…,xk) And Y ═ Y1,y2,…,yk) The homography matrix mapping is established as follows:
in the step (3), the specific process of obtaining the overall array information is as follows:
defining the position information of the players of each frame of a certain team as p0,1,p0,2,…,p0,11Centering itAnd (4) performing conversion processing, namely subtracting the average position of all the players in the frame from each position information to obtain the information p of the players in each frame of a certain centralized team1,p2,…,p11；
The activity range of a player at one position in the matrix accords with Gaussian distribution, and the specific algorithm is as follows:
3-1, initializing Gaussian distribution, taking the positions of the players in the first frame as the mean value of the initial distribution, and adopting a random number generation method for the variance of the distribution;
3-2, traversing the next frame, and calculating the cost c of the ith player distributed to the jth positioni,j＝-log(pi,j) Wherein p isi,jThe size of the probability density function of the ith player under the condition that the jth player belongs to the jth position;
3-3, updating distribution information, calculating the mean and variance of the positions of the players belonging to a certain distribution, and using the mean and variance to represent the expectation and variance of the distribution as the basis of the next frame iteration;
3-4, traversing each frame, and classifying the players of each frame respectively to obtain a distribution result of the first iteration, wherein each distribution comprises expectation and variance;
3-5, taking the result of the previous iteration as the initial value of the second iteration, and traversing each frame to obtain a new iteration result;
3-6, performing one round of iteration until the distribution is stable, wherein the difference is smaller than a threshold value;
the form visualization adopts a form of a sang-based chart to express the form change in the whole match and the personnel flow information of the players;
the visualization of the formation specifically is to use the stream view to visualize the change of the formation in the normal match and to show the flowing condition of the personnel in the formation.
2. The visual analysis method for the formation change and the human flow of the football match as claimed in claim 1, wherein in the step (2), the smoothing process uses a linear filter to perform a mean filtering process on the time series position information.
3. A visual analysis method of the formation change and the staff movement of the football match as claimed in claim 1, characterized in that the match real-time data visualization is embodied to show the data change along the time line field, including the running distance, the formation coverage area, the pass success rate and the ball control time of the team members on the field.
4. The visual analysis method for the formation change and the staff movement of the football match as claimed in claim 1, wherein the match event visualization is to show the key events on the time line, show the goal events in the event list, click the goal events, and play the goal video; and moving on the time line by using a mouse, jumping to a corresponding time point of the video in real time and playing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810858815.1A CN109101911B (en) | 2018-07-31 | 2018-07-31 | Visual analysis method for football match formation change and personnel flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810858815.1A CN109101911B (en) | 2018-07-31 | 2018-07-31 | Visual analysis method for football match formation change and personnel flow |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109101911A CN109101911A (en) | 2018-12-28 |
| CN109101911B true CN109101911B (en) | 2021-03-05 |
Family
ID=64848009
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810858815.1A Active CN109101911B (en) | 2018-07-31 | 2018-07-31 | Visual analysis method for football match formation change and personnel flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109101911B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110727826B (en) * | 2019-09-30 | 2022-03-25 | 浙江大学 | Visual analysis method for tactical excavation of table tennis |
| CN110968733B (en) * | 2019-12-05 | 2022-08-09 | 浙江大学 | Icon-based multi-scale table tennis tactical analysis visualization system |
| CN110826539B (en) * | 2019-12-09 | 2022-04-19 | 浙江大学 | Visual analytic system of football pass based on football match video |
| CN111001149B (en) * | 2019-12-10 | 2021-06-04 | 苏宁智能终端有限公司 | Position positioning method and system based on video playback |
| CN113625910B (en) * | 2021-08-06 | 2023-10-27 | 浙江大学 | Visual analysis system for beat motion polytropic simulation |
| CN115414655A (en) * | 2022-08-15 | 2022-12-02 | 浙江大学 | Visual analysis method and system for basketball-oriented sports of basketball-free players |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102819749A (en) * | 2012-07-23 | 2012-12-12 | 西安体育学院 | Automatic identification system and method for offside of football based on video analysis |
| CN104274969A (en) * | 2013-07-09 | 2015-01-14 | 恩海恩娱乐公司 | Simulation method and system for real-time broadcasting |
| CN104881882A (en) * | 2015-04-17 | 2015-09-02 | 广西科技大学 | Moving target tracking and detection method |
| CN106310660A (en) * | 2016-09-18 | 2017-01-11 | 三峡大学 | Mechanics-based visual virtual football control system |
| CN106512400A (en) * | 2016-12-08 | 2017-03-22 | 上海时年信息科技有限公司 | Football game simulation method based on process |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101834842B (en) * | 2010-03-16 | 2013-01-30 | 清华大学 | Intelligent control method for RoboCup platform players under embedded environment and system thereof |
| US20130203494A1 (en) * | 2012-02-03 | 2013-08-08 | Charles Edward Coiner, JR. | Electronic football playbook |
| JP6368965B2 (en) * | 2014-07-11 | 2018-08-08 | 株式会社コナミデジタルエンタテインメント | GAME SYSTEM, GAME CONTROL DEVICE, AND PROGRAM |
| CN105344086B (en) * | 2015-08-19 | 2018-06-26 | 刘庆斌 | A kind of football attack and defensive technique intelligent tutoring, the setting method and its equipment of training and competition system |
| CN107992464A (en) * | 2017-12-08 | 2018-05-04 | 浙江大学 | The method for visualizing of single game Basketball Match data |
-
2018
- 2018-07-31 CN CN201810858815.1A patent/CN109101911B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102819749A (en) * | 2012-07-23 | 2012-12-12 | 西安体育学院 | Automatic identification system and method for offside of football based on video analysis |
| CN104274969A (en) * | 2013-07-09 | 2015-01-14 | 恩海恩娱乐公司 | Simulation method and system for real-time broadcasting |
| CN104881882A (en) * | 2015-04-17 | 2015-09-02 | 广西科技大学 | Moving target tracking and detection method |
| CN106310660A (en) * | 2016-09-18 | 2017-01-11 | 三峡大学 | Mechanics-based visual virtual football control system |
| CN106512400A (en) * | 2016-12-08 | 2017-03-22 | 上海时年信息科技有限公司 | Football game simulation method based on process |
Non-Patent Citations (3)
| Title |
|---|
| Large-Scale Analysis of Soccer Matches using Spatiotemporal Tracking Data;Alina Bialkowski 等;《International conference on data mining》;20141231;第725-730页第1-2节，图4 * |
| Visual soccer match analysis using spatiotemporal positions of players;Vinicius Machado 等;《Computers & Graphics》;20171130;第68卷;第84-95页图1，第1-4节 * |
| 球场数据可视化分析;卢挺;《中国优秀硕士学位论文全文数据库-信息科技辑》;20150515(第5期);第I138-1139页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109101911A (en) | 2018-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109101911B (en) | Visual analysis method for football match formation change and personnel flow | |
| Figueroa et al. | Tracking soccer players aiming their kinematical motion analysis | |
| Barris et al. | A review of vision-based motion analysis in sport | |
| Perše et al. | A trajectory-based analysis of coordinated team activity in a basketball game | |
| Sheng et al. | GreenSea: visual soccer analysis using broad learning system | |
| CN101639354B (en) | Method and apparatus for object tracking | |
| Tora et al. | Classification of puck possession events in ice hockey | |
| Negin et al. | A decision forest based feature selection framework for action recognition from rgb-depth cameras | |
| US20180137363A1 (en) | System for the automated analisys of a sporting match | |
| CN106131469A (en) | Ball intelligent robot based on machine vision coach and judgment system | |
| CN105912991B (en) | Activity recognition based on 3D point cloud and crucial bone node | |
| CN102819749A (en) | Automatic identification system and method for offside of football based on video analysis | |
| CN111444890A (en) | Sports data analysis system and method based on machine learning | |
| Pingali et al. | Instantly indexed multimedia databases of real world events | |
| Sternig et al. | Multi-camera multi-object tracking by robust hough-based homography projections | |
| Beetz et al. | Camera-based observation of football games for analyzing multi-agent activities | |
| Setterwall | Computerised video analysis of football-technical and commercial possibilities for football coaching | |
| Hsu et al. | Coachai: A project for microscopic badminton match data collection and tactical analysis | |
| Ong et al. | Tracking of moving athlete from video sequences using flower pollination algorithm | |
| US20230377336A1 (en) | Method of operating server providing sports video-based platform service | |
| Demaj | Geovisualizing spatio-temporal patterns in tennis: An alternative approach to post-match analysis | |
| Li et al. | Deep learning algorithm-based target detection and fine localization of technical features in basketball | |
| Chu et al. | Extraction of baseball trajectory and physics-based validation for single-view baseball video sequences | |
| Lee et al. | A study on sports player tracking based on video using deep learning | |
| Waltner et al. | Indoor activity detection and recognition for sport games analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |