KR102045439B1 - Virtual tennis simulation system, sensing device and method for sensing process using the same - Google Patents

Abstract

The present invention is to provide a virtual tennis simulation system for allowing a player to experience the dynamic sports characteristics of tennis and to be provided with a variety of information, the sensing of the ball hit by the player as well as the movement of the player Through sensing and analyzing various information such as sensing and analyzing the data, it provides dynamic and realistic virtual tennis play that could not be realized in conventional screen tennis, and provides players with various analysis information about virtual tennis play. It is to provide a virtual tennis simulation system, a sensing device and a sensing method used therein.

Description

VIRTUAL TENNIS SIMULATION SYSTEM, SENSING DEVICE AND METHOD FOR SENSING PROCESS USING THE SAME

According to the present invention, a screen for implementing a tennis play image by a virtual opponent in a predetermined indoor space is installed, and a tennis ball or a tennis game can be played in such a manner that a user hits a tennis ball fired from a ball machine with a racket through the screen. It relates to a virtual tennis simulation system, a sensing device and a sensing method used therein.

Recently, sports such as golf and baseball, such as golf and baseball, which provide virtual contents for practice or play, such as playing golf or baseball on a real field, even in a narrow space in the room, such as screen golf and screen baseball, etc. Popular popularity and the resulting technological developments have led to the emergence of so-called screen tennis systems for tennis, the most representative sport that uses balls.

Unlike golf or baseball, tennis is a sport in which the user does not hit the ball continuously in one place but rather rallys the player with the racquet and moves the ball with the other player continuously. By properly implementing the features in the screen tennis system, users who use screen tennis can hit the ball from various positions and make continuous rally with the opponent in the video realizing the reality of tennis sports, but it is difficult to grasp easily in the actual tennis sports. The key is to provide a variety of analysis information.

As a prior patent document related to the conventional screen tennis system and the sensing device used therein, Patent Application No. 10-2015-0010077, Patent Application No. 10-2015-0138876, US Patent No. 6,776,732, US Patent No. 3,989,246 The issue is disclosed.

However, the conventional screen tennis system is limited to the sensing of the hit ball when the player hits the ball, and even the method widely used in the conventional screen golf system, that is, the start speed and the start speed when the ball starts Since the sensing range is very limited and the accuracy is low in that the entire trajectory is predicted using only the angle and direction information, various information is provided to match the characteristics of the sport of tennis that is played dynamically while moving around the court. There is a problem that the conventional screen tennis system is quite insufficient in terms of sensing and analyzing and providing the user with various information according to the characteristics of the tennis.

Furthermore, the conventional screen tennis system has a problem that the screen tennis system does not sufficiently capture the characteristics of the sport of tennis because the play is very monotonous, boring, and simple repetition due to the problems described above.

The present invention is to provide a virtual tennis simulation system for allowing a player to experience the dynamic sports characteristics of tennis and to be provided with a variety of information, the sensing of the ball hit by the player as well as the movement of the player Through sensing and analyzing various information such as sensing and analyzing the data, it provides dynamic and realistic virtual tennis play that could not be realized in conventional screen tennis, and provides players with various analysis information about virtual tennis play. It is to provide a virtual tennis simulation system, a sensing device and a sensing method used therein.

The virtual tennis simulation system according to an embodiment of the present invention, the player is provided in front of the floor of the court to play tennis, the screen on which the virtual tennis simulation image is projected; A ball machine provided at the rear of the screen to provide a ball to the player; A camera unit for continuously acquiring an image for a shooting range including a preset area; A sensing processing unit configured to calculate a ball motion model for the ball by extracting and analyzing an object corresponding to the ball from the acquired image; And implementing an image of a virtual counterpart player that is the counterpart that the player plays, controlling the virtual counterpart player to play according to a predetermined artificial intelligence, and using the ball motion model calculated by the sensing processing unit. A control device for generating and processing a virtual tennis simulation image and controlling the ball machine;

The control device is a virtual ball on the image to correspond to the ball motion model by using the sensing processing unit calculates a ball motion model through analysis of the image acquired by the camera unit according to the ball hit of the player A simulation image of moving to the virtual court where the virtual opponent player is located, and the simulated image of the virtual ball being moved as the virtual opponent player hits the virtual ball according to preset artificial intelligence. And the virtual opponent player to the virtual player against the player hitting the ball in such a way that the ball machine provides the ball on the extension line that the virtual ball is moving so that the player can hit the ball. Repeatedly hitting the ball and the video on the player The virtual machine allows a player party to implement the tennis rally.

Also, preferably, the control device determines whether the virtual ball simulated on the image is in-play corresponding to the ball sensed by the camera unit and the sensing processing unit according to the hit of the player. Determine the exercise condition of the virtual ball hit by the virtual opponent player according to,

The ball machine sets the direction and speed of the ball to be provided to the player according to the determined motion condition of the virtual ball, and the ball machine moves the ball on an extension line of the trajectory of the virtual ball hit by the virtual opponent. To provide the direction and speed of the set ball so that the player hits the ball fired from the ball machine, so that the player and the virtual counterpart player on the image can implement a tennis rally. do.

Also preferably, the sensing processing unit is configured to detect the position of the player by extracting and analyzing an object corresponding to the player from the image acquired by the camera unit, and the control device is configured to detect the sensing processing unit. It is characterized in that the artificial intelligence to determine the motion conditions of the virtual ball when the virtual opponent player is hit based on the position of the player detected by the.

On the other hand, the virtual tennis simulation system according to another embodiment of the present invention, the screen is provided in front of the court floor portion where the player plays the tennis is a virtual tennis simulation image is projected; A ball machine provided at the rear of the screen to provide a ball to the player; A camera unit for continuously acquiring an image for a shooting range including a preset area; Sensing to detect the position of the player by extracting and analyzing the object corresponding to the ball from the acquired image to calculate the ball motion model for the ball, and extracting and analyzing the object corresponding to the player from the acquired image Processing unit; And generating and processing the virtual tennis simulation image and controlling the ball machine by using the ball motion model calculated by the sensing processing unit, and based on the position of the player detected by the sensing processing unit. And a controller configured to determine a direction of a ball provided to the user and to correspondingly process the virtual tennis simulation image and control the ball machine.

Also preferably, the sensing processing unit may predefine a function of the coordinate system of the three-dimensional space played by the player and the motion of the ball in each coordinate axis direction, and analyze the image photographed by the camera unit. Detecting coordinate data in three-dimensional space on each image for the ball to be applied, and applying the detected coordinate data of the ball to a predefined function for each coordinate axis direction for the ball hit by the player And a ball movement model, wherein the controller implements an image related to a virtual opponent's player and a virtual court played by the virtual opponent's player in correspondence with a court floor portion played by the player. The virtual counterpart player based on the ball motion model calculated by the sensing processing unit It characterized by configured to implement a simulation image of the virtual ball in exercising followed.

Also preferably, the sensing processing unit may calculate the average height of the balls in the calculated ball movement model when the ball extracted from the acquired image is out of a predetermined range from the calculated ball movement model. If the ball is located below the average height of the calculated ball, the average speed of the calculated ball motion model is calculated to determine whether it is at least one of the cases having a speed lower than the average speed of the calculated ball motion model. It is configured to classify the corresponding ball as the floor ball, not the ball used in the player's play, the control device is to determine the number of the ball classified as the floor ball by the sensing processing unit and to the floor ball If the number of balls classified exceeds the preset number, the player is provided with an alarm for danger warning or It characterized by configured to provide an alarm for requesting to fill the ball to the ball of the ball machine box.

Also preferably, the sensing processing unit may track the position of the player by detecting the player from each of the acquired frames of images, and sense the moving distance of the player and the play time of the player by the tracking. And calculate the exercise amount of the player from the sensed movement distance and play time, and wherein the control device is configured to provide the player exercise amount information calculated by the sensing processing unit to the player through video or audio. It is characterized in that the configuration.

Also preferably, the sensing processing unit presets a dangerous area as a preset area adjacent to the screen, and detects an object in the area corresponding to the dangerous area on the image of every frame to be acquired. It is configured to detect a human body from the object, the control device, characterized in that the control to stop the operation of the ball machine, when the human body is detected in the dangerous area by the sensing processing unit.

Meanwhile, a sensing device according to another embodiment of the present invention is a sensing device used in a virtual tennis simulation system that allows a player to play tennis on a court floor based on a virtual tennis simulation image. A camera unit for continuously acquiring an image for a shooting range including a; Sensing to detect the position of the player by extracting and analyzing the object corresponding to the ball from the acquired image to calculate the ball motion model for the ball, and extracting and analyzing the object corresponding to the player from the acquired image And a processing unit.

Also preferably, the camera unit may be configured in a stereo manner including a first camera and a second camera which acquire images at the same angle of view at different positions, wherein the first camera and the second camera are centers of the angle of view. 1 axis tilting in the front-rear direction or the left-right direction or 2-axis tilting in the front-rear direction and the left-right direction on the basis of the reference point, and according to the position of the player detected in the images acquired by the first camera and the second camera. And a camera controller for controlling the tilting of the first camera and the second camera to allow the player to be detected in the acquired image.

Also, preferably, the sensing processing unit selects the reference object based on a preset item for the size and shape of the object with respect to the plurality of objects detected by the extraction of the difference image and the detection of an outline. Setting a region of interest including a reference object and setting objects existing in the region of interest as one pixel group to unify surrounding objects based on the reference object to identify uniqueness of the united object; And detect the unitary object as an object corresponding to the player.

Also preferably, the sensing processing unit may preset a dangerous area as a preset area, detect an object within the area corresponding to the dangerous area on the image of each frame to be acquired, and detect the size of the detected object and By analyzing the shape to determine whether it corresponds to the human body is characterized in that it is configured to detect whether a third person, not the player appeared in the angle of view of the camera unit.

Also preferably, the sensing processing unit may be configured to perform the camera unit through a high speed difference image between an image acquired in real time by the camera unit and an image selected as a difference operation image from a previously acquired image of the acquired image. Extracts an object moving above the image acquisition speed as a racket object, analyzes the racket object extracted from each of the plurality of difference images extracted through the high speed vehicle image, and detects the position and movement direction of the racket And detect whether the swing is forehand or backhand.

On the other hand, the control method of the virtual tennis simulation system according to an embodiment of the present invention, the court floor portion that the player plays the tennis, a screen provided in front of the court floor portion to project a virtual tennis simulation image, the screen A ball machine provided at the rear of the ball providing the ball to the player, a sensing device for sensing the player and the ball played by the player, generating and processing the virtual tennis simulation image, and controlling the ball machine. A control method of a virtual tennis simulation system including a control device, the control method comprising: continuously obtaining an image of a shooting range including a preset area; Calculating sensing data on the motion of the ball by extracting and analyzing an object corresponding to the ball from the acquired image; Implementing a simulation image in which a virtual ball moves toward a virtual court where a virtual opponent is located on the image so as to correspond to the ball movement by using the calculated sensing data for the ball movement; Implementing a simulation image in which the hit virtual ball moves as the virtual opponent player strikes the virtual ball according to a preset artificial intelligence; And the virtual opponent player hits the virtual ball in response to the player hitting the ball in such a manner that the ball machine provides the ball so that the player can hit the extension line on which the virtual ball moves. Repeating the step of implementing the tennis rally by the player and the virtual counterpart player on the image.

On the other hand, the control method of the virtual tennis simulation system according to another embodiment of the present invention, the court floor for the player to play the tennis, the screen is provided in front of the court floor and the virtual tennis simulation image is projected, A ball machine provided at the rear of the screen to provide a ball to the player, a sensing device for sensing the player and the ball played by the player, generation and processing of the virtual tennis simulation image, and control of the ball machine. A control method of a virtual tennis simulation system including a control device, the control method comprising: continuously acquiring an image of a shooting range including a preset area; Detecting a position of the player by extracting and analyzing an object corresponding to the player from the acquired image; Calculating sensing data on the motion of the ball by extracting and analyzing an object corresponding to the ball from the acquired image; Generating and processing the virtual tennis simulation image using the calculated ball motion sensing data; And determining a direction of a ball provided to the player based on the detected position of the player, processing the virtual tennis simulation image accordingly, and controlling the ball machine.

Also preferably, the controller is configured to store in advance information on at least one of skill level, difficulty level, and play style of the player, and to control a virtual opponent player on the virtual tennis simulation image by artificial intelligence, The controlling of the ball machine by processing the virtual tennis simulation image may include: detecting the position of the player according to information on at least one of skill level, difficulty level, and play style of the player and artificial intelligence of the virtual opponent player; Determining the direction of the ball to be provided to the player according to the virtual ball hit by the virtual opponent player, and the virtual ball hit by the virtual opponent player according to the determined direction of the ball. Processing a simulation image of the trajectory; Setting the ball firing direction of the ball machine to correspond to the determined direction of the ball, and the ball machine moves the ball in the direction of the determined ball on an extension line to the trajectory of the virtual ball hit by the virtual opponent player. Controlling to provide.

Also preferably, the controller pre-stores information on at least one of skill level, difficulty level, and play style of the player, and processes the virtual tennis simulation image and controls the ball machine. Determining a direction of a ball to be provided from the ball machine to the player in response to the detected player position according to information on at least one of skill level, difficulty level, and play style; and according to the determined direction of the ball. Processing a simulation image of the trajectory of the virtual ball hit by the virtual opponent player on the virtual tennis simulation image, setting a ball firing direction of the ball machine to correspond to the determined ball direction; A description of the trajectory of an imaginary ball hit by an imaginary opponent. On board it characterized in that it comprises a step of controlling so as to provide that the ball machine seen in the direction of the determined view.

Also preferably, the image acquisition speed of the camera unit may be higher than or equal to the image acquisition speed of the camera unit through a high speed difference image between the image acquired in real time by the camera unit and the image selected as an image to be computed from a previously acquired image of the acquired image. Extracting a moving object as a racket object, analyzing the racket object extracted from each of a plurality of difference images extracted through the high-speed vehicle image, and detecting a position and a moving direction of the racket, and swinging the player And detecting whether the hand is a forehand or a backhand.

Meanwhile, a sensing method according to an embodiment of the present invention is a sensing method of a sensing device used in a virtual tennis simulation system that allows a player to play tennis on a court floor based on a virtual tennis simulation image. Acquiring in real time an image of a shooting range including an area; Extracting a difference image between a background image prepared in advance and each of the images acquired in real time, and detecting an outline of each object detected from the extracted difference image; Detecting the position of the player by extracting an object corresponding to the player by unifying objects within a preset range among the plurality of objects whose contours are detected; And calculating a ball motion model for the ball by extracting and analyzing an object corresponding to the ball from the acquired image.

Also preferably, the detecting of the position of the player may include selecting a reference object according to a preset item for the size and shape of the object with respect to the plurality of objects detected by extraction of the difference image and detection of an outline. Setting a region of interest of a size including the selected reference object, and unifying neighboring objects based on the reference object by setting objects existing in the set region of interest as one pixel group. And detecting the unity of the unified object and detecting the unified object as an object corresponding to the player.

Also preferably, the detecting of the position of the player may include calculating a center of gravity of an object corresponding to the extracted player as an estimated position of the player, initial position of the calculated ball movement model and the calculation. And if the estimated position of the player is less than or equal to a predetermined distance, determining the calculated estimated position of the player as the position of the player.

The virtual tennis simulation system, the sensing device and the sensing method used according to the present invention, to provide a virtual tennis simulation system for allowing a player to experience the characteristics of the dynamic sport of tennis and to receive various information Through sensing and analyzing various information such as sensing and analyzing the movement of the player as well as sensing the ball hit by the player, it provides a dynamic and realistic virtual tennis play that could not be realized in conventional screen tennis. By providing the user with various analysis information about the virtual tennis play, the player who enjoys the virtual tennis play can improve the interest.

1 shows an example in which a screen tennis system is implemented as a virtual tennis simulation system according to an embodiment of the present invention.
FIG. 2 is a diagram showing the configuration of the screen and the surroundings of the virtual tennis simulation system shown in FIG. 1.
3 is a diagram illustrating an example of image content provided by the virtual tennis simulation system shown in FIG. 1.
4 is a block diagram showing the configuration of the virtual tennis simulation system shown in FIG.
5 and 6 are flowcharts illustrating a process relating to a sensing method of a sensing device used in a virtual tennis simulation system according to an embodiment of the present invention.
7 and 8 are diagrams illustrating an example of detecting a player and another third party in the captured image, respectively.
9 is a flowchart illustrating a process for detecting a racket of a sensing device used in a virtual tennis simulation system according to an embodiment of the present invention.
FIG. 10 is a diagram illustrating a process of detecting a racket held by a user from an image captured according to the flowchart shown in FIG. 9.
FIG. 11 is a time series diagram illustrating an operation of each object as an object corresponding to a racket detected according to the process illustrated in FIG.
12 is a view for explaining the play between the player (P) and the virtual opponent player (VP) on the image and the driving of the ball machine in the virtual tennis simulation system according to an embodiment of the present invention.

A detailed description of the virtual tennis simulation system according to the present invention, a sensing apparatus and a sensing method used therein will be described with reference to the drawings.

First, a configuration of a virtual tennis simulation system according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4.

1 is a view showing an example of a screen tennis system implemented as a virtual tennis simulation system according to an embodiment of the present invention, Figure 2 is a view showing the configuration of the screen and the surroundings of the virtual tennis simulation system shown in FIG. 3 is a diagram illustrating an example of image content provided by the virtual tennis simulation system illustrated in FIG. 1, and FIG. 4 is a block diagram illustrating a configuration of the virtual tennis simulation system illustrated in FIG. 1.

As shown in FIG. 1, the virtual tennis simulation system according to an embodiment of the present invention provides a screen 20 in front of the player floor with the court floor 30 and the net 32 in a predetermined space. It is provided with a ball machine 100 is provided on the back of the screen 20 to provide a ball for the player (P) to hit the player (P) holding a racket and play tennis, on one side of the space Sensing device for collecting the sensing data for the image output unit 420 for projecting the image on the screen 20, the ball hit by the player (P) and the movement of the player (P), the racket used by the player (P) 200 may be installed and configured.

Devices such as the ball machine 100, the sensing device 200, and the image output unit 420 are connected to the control device 300, and the control device 300 transmits sensing data of the sensing device 200. It receives and processes and generates the image content for the user's virtual tennis practice and game to deliver to the image output unit 420, the image output unit 420 projects the received image content on the screen 20, The ball firing of the ball machine 100 and the control of the ball firing parameters, such as the direction, the speed of the ball to be fired.

The ball machine 100 is installed at the rear of the screen 20 to shoot the ball 1 to a pre-computed position on the court bottom 30 through the ball firing hole 22 formed in the screen 20. As shown in FIG. 1, the ball machine 100 is configured to enable firing angle control and firing speed control so as to launch balls 1 of various speeds at various positions.

2 shows a photograph taken of the configuration around the screen 20 by actually implementing a virtual tennis simulation system according to an embodiment of the present invention.

As shown in FIG. 2, the ball firing hole 22 may be provided at an approximately intermediate position on the screen 20. Of course, it can be installed at any position on the screen 20, but in the implementation of receiving (relative player's) to the ball of various trajectories, an approximately intermediate position is preferable as shown in FIG.

The front of the screen 20 is inclined with the net 32 so that the ball hit by the user towards the screen 20 can be recovered back toward the ball machine 100 without hitting the screen 20 and then rolling back to the user. The bottom net 32a, etc., which are installed as a structure, may be installed, and safety nets 33 and 34 may be installed on both sides thereof so that balls do not protrude to the outside.

The balls 2 gathered by the net 32 and the bottom net 32a may be configured to be collected toward the recovery port 40 at the bottom of the screen 20 and then transferred back to the ball machine.

3 illustrates an example of image content provided by a virtual tennis simulation system according to an embodiment of the present invention.

As shown in FIG. 3, the entire image IM includes a main image 500 and a minimap image 600.

The main image 500 may implement a virtual environment for the opposite side of the court floor that the user plays, and may include a content image of the virtual opponent player VP, the referee, and the like.

In the main video 500, as if the opponent is playing in a real tennis match, the virtual opponent player VP receives the virtual ball VB according to a predetermined artificial intelligence (hereinafter referred to as "AI"). In this case, as shown in FIG. 3, the trajectory Tb of the virtual ball VB hit by the virtual opponent player VP may or may not be displayed. Regardless of whether or not the trajectory is displayed, the trajectory of the virtual ball VB should be calculated, and the control of the image content and the ball firing parameter in the ball machine may be calculated and implemented according to the trajectory. Detailed description thereof will be described later.

On the other hand, the mini-map image 600 is a part that is displayed by reducing the size of the court floor and the opposite part of the virtual environment that the user plays as one entire tennis court to a predetermined size as a plan view from above.

The minimap image 600 may display various information such as the current location of the user, a set area based on the user, the location and trajectory of the virtual player, and the location of the ball.

On the other hand, with reference to Figure 4 will be described with respect to the control system of the components constituting the virtual tennis simulation system according to an embodiment of the present invention as shown in FIG.

As shown in FIG. 4, a virtual tennis simulation system according to an embodiment of the present invention includes a ball machine 100, a sensing device 200, a control device 300, and an image output unit 420. Can be.

The control device 300 may include a data storage 320, an image processor 330, a controller 310, and the image output unit 420 is processed by the image processor 330. The image may be implemented by a projection device or the like for projecting the image on the screen 20.

The data storage unit 320 is a part for storing data for processing a background image of a tennis practice or a game, a video for a virtual player, a simulation video for a hit ball, and the like, which are performed in a virtual tennis simulation system. The data storage unit 320 may be configured to serve as a storage for temporarily storing data received from a server (not shown).

The image processor 330 may include a background image of a virtual tennis court, a background image of a player, a referee, an audience, and the like, an image of a virtual counterpart player hitting the ball, a simulation image of the trajectory of the ball hit by the player, and the like. Generates various simulation related images.

Meanwhile, the sensing device 200 continuously photographs an image of the player hitting the racket and analyzes the captured images, as well as the motion information of the hitting ball, and the hitting ball reaches a part of the screen. It can be configured as a device of the camera sensing method that senses correctly.

The sensing device 300 may include a camera unit 210, a sensing processing unit 220, and a camera controller 230, and the camera unit 210 may be in a predetermined shooting range including a preset area. Image is continuously obtained, and the sensing processing unit 220 receives the image from the camera unit 210 and performs an image analysis according to a preset matter, which is required for tennis practice or tennis match according to the virtual tennis simulation. Calculate the information.

The camera unit 210 of the sensing device 200 photographs the same photographing range at different locations, respectively, to calculate three-dimensional position information of the object on the photographed image. By configuring the stereoscopic method including the camera 212, the sensing processing unit 220 to obtain the three-dimensional coordinate data for the ball through the analysis of the image taken by the camera unit 210 This is preferred.

Although not shown in the drawings, the camera unit 210 of the sensing device 200 provided in the system according to the present invention may be configured to be capable of tilting back, forth, left, and right, respectively, based on the center of the angle of view, and the first camera. The 211 and the second camera 212 may be configured to automatically zoom in-zoom out, respectively.

The camera controller 230 may tilt the front, rear, left, and right sides of the first camera 211 and the second camera 212 in advance according to the processing result of the image or the position where the player is detected in the captured image. It can be configured to control according to the set program.

The first camera 211 and the second camera 212 may be configured to tilt in one direction in the front-rear direction or the left-right direction, respectively.

By referring to the location information of each camera and the angle of view of the camera in a three-dimensional space, a data table having pan and zoom values of the camera for each specific area in a three-dimensional space is produced and stored in advance. Is detected, it is possible for the camera controller 230 to control each camera to target the corresponding position.

Therefore, it is possible to accurately capture all the hits of the ball fired at various positions while the user dynamically moves on the bottom of the court to acquire an image for sensing data collection.

The sensing processing unit 220 receives and collects images captured from each of the first camera 211 and the second camera 212 of the camera unit 210 and analyzes each of the collected images to find a ball. The 3D coordinate data of the ball is extracted and the extracted 3D coordinate data is used to determine the motion model of the ball with respect to the ball being hit and moved (by the racket of the player). Using the virtual tennis simulation to calculate the various information necessary for the tennis practice and tennis game.

Here, the motion model of the ball means that it is represented by the equation of motion relating to the trajectory on the three-dimensional space of the ball to be hit and move, as shown in FIG. By defining the three-dimensional coordinate system of the x-axis, y-axis and z-axis with respect to space, the motion model of the ball can be determined according to the defined coordinate system.

That is, the motion model of the ball may be defined as a motion equation in the x-axis direction, a motion equation in the y-axis direction, and a motion equation in the z-axis direction.

The calculation of the motion model of the ball as described above is completely different from that in the conventional screen golf system or the like that the sensor senses the motion of the ball hit by the golf club to calculate the sensing data.

Although a camera-type sensing device is widely used in a conventional screen golf system, the sensing device calculates initial motion conditions such as starting speed, starting direction angle, and height angle when the ball is hit, and thus, virtual space, that is, virtual In the golf course, the ball trajectories simulated by the physics engine when the ball starts according to the calculated initial motion conditions were calculated.

However, the motion model of the ball calculated by the sensing device of the virtual tennis simulation system according to the present invention does not calculate the initial motion condition when the ball is hit, but the angle in the space where the actual player plays, that is, the actual three-dimensional space. It is to find a model in which a ball actually moves as a motion model of a ball calculated based on a motion equation in the coordinate axis direction.

By calculating the motion model of the ball, it is possible to provide more diverse and more information than simply calculating the initial motion condition of the ball, and in particular, the real world and the virtual world frequently cross each other more closely, such as virtual tennis simulation. It is a more suitable sensing method when it is necessary to be connected.

For example, since the above-described motion model calculation of the ball is a ball motion model in a three-dimensional space in which actual play is performed, the position at which the ball reaches the actual screen can be easily calculated. After reaching the screen, the virtual ball can be simulated from the location, so that the real world and the virtual world can be connected naturally.

Of course, it is also possible to sense the position where the ball moving to a separate device reaches the screen, the present invention is that it is possible to do so only by the sensing device for detecting the movement of the ball.

More specific details of the calculation of the above-described motion model of the ball will be described later.

On the other hand, the ball machine 100 is installed on the back of the screen 20 is a device for firing the ball through the ball launch hole 22 on the screen 20, as shown in Figure 4 ball supply unit 110 It may be configured to include a launch driver 120 and the launch control unit 130.

The ball supply unit 110 is a component that transfers the balls one by one from the storage box in which a plurality of balls are stored and supplies them to the launch driver 120, that is, supplies the balls to a position for ball launch.

The launch driver 120 is a component for launching a ball supplied by the ball supply unit 110, and rotates two or more wheels and positions the ball between the wheels to launch the ball by the rotational force of the wheels. It can be implemented in a variety of ways, such as shooting a ball like a bullet.

The firing control unit 130 controls the firing driving unit 120 according to a predetermined firing condition or a blow condition determined according to a predetermined AI for the virtual opponent player on the virtual tennis and the trajectory of the hit ball. The speed, pitch, and the like of the conditional ball can be controlled to allow the ball to be fired.

In addition, the ball machine 100 is configured to tilt the firing drive unit 120 in the vertical direction and the left and right directions, respectively, so that the firing control unit 130 to the firing drive unit 120 in accordance with the hitting conditions described above. By controlling the tilting drive, the ball may be fired in various directions within a predetermined angle range through the ball firing hole on the screen.

On the other hand, with reference to Figure 5 will be described in detail with respect to the calculation of the exercise model of the ball as a sensing method of the sensing device used in the virtual tennis simulation system according to an embodiment of the present invention and an example of a method of using the same.

As the virtual tennis play by the player starts, the sensing device is operated so that each camera of the sensing device continuously acquires an image of the shooting range (S100).

The sensing processing unit detects an object corresponding to the ball in each image continuously acquired as described above (S110), checks the feature corresponding to the ball with respect to the detected object, and confirms that the ball is a ball. Various noises, such as not being removed (S120).

Here, detecting an object in an image may use techniques such as difference image and edge detection, and checking a feature corresponding to a ball with respect to the detected object may include, for example, an object size. , Presetting each of the corresponding items with information corresponding to the ball in the aspect ratio, the roundness of the object, the brightness of the object, etc., and then checking each detected object for each preset item to identify the ball-in object. It is.

The difference image obtains a difference image by an initial reference image, an image acquired later in real time, and a difference calculation technique, and the moving object remains in the difference image.

The image of the moving object is converted into a binarized image based on a specific threshold value, and the contour is detected to extract features of the object extracted from the binarized image. The shape of the object is identified by the information on the detected contour, and it is determined that the circle (the tennis ball is output as the original image in the 2D image). The check items preset as corresponding to the characteristics of the ball are the size of the object, the aspect ratio of the object, the circumference ratio of the object (roundness), and the brightness of the object.

Looking at the size of the object, it is checked whether the BlobSize (width) of the recognized object is out of the threshold that can be displayed on the image based on the actual ball size. In general, when playing tennis, it predicts the size of the ball that enters the angle of view and outputs the image. It designates a wide range of absolute thresholds to reduce the computation speed.

This can speed up computation by excluding elements that can be quickly excluded by size, such as salt noise or people.

Looking at the aspect ratio of the object, considering the expected maximum speed of the moving ball (less than 60m / s) and the image capture speed (350fps), it is determined that there is no distortion in the image of the ball, It is expected that the width and height will be similar, so the higher the similarity, the closer to the circle.

Looking at the ratio of the circumference of the object, the circumference and the area value of the object extracted by the contour detection can be extracted, and it is determined whether the object is close to the circle based on the width and the circumference of the object.

The closer the ratio is to 1, the closer the shape of the circle.

S = area of the circle, r = radius of the circle, l = circumference of the circle

,

,

비율 circumference ratio (0 ~ 1) = 4 πS / (l ^ 2)

In addition, since the ball comes out above a certain brightness in the image, the brightness value of the object can also be used as a criterion for determining the ball.

On the other hand, it is necessary to appropriately remove portions of the image that are considered noise in the detection of the ball. First, the ball portion outside the region of the 3D booth can be easily identified using the region information (prestored information) of the 3D space to be played. This can be judged as noise and eliminated.

Using the actual size of the ball and the position of the ball in three-dimensional space, you can calculate the blob size of the ball expected in the image, remove 3D ball data that is outside the threshold as noise, and determine the actual size information of the ball and the Since the position is known, the distance between the position information of the ball calculated in the three-dimensional space and the camera position can be estimated to estimate the width of the ball expected in the image, and thus the excessively deviated data can be determined and removed as noise.

When the ball is detected from the image as described above, it may be necessary to determine in which direction the detected ball moves. This is because the sensing device wants to sense the motion model of the ball when the ball hit by the player moves to the screen, so it may not be necessary to calculate the motion model of the ball when the ball moves from the screen to the player.

Therefore, when the ball is detected in step S120, it is determined whether the ball moves in the screen direction (S130). If the ball moves in the screen direction, a trigger signal is generated to start the calculation process of the ball motion model described above (S140).

If the detected ball does not move in the screen direction, the process starts again from the step S110 for the detection of the ball.

When the calculation trigger of the ball motion model is generated in step S140, a predetermined number of images before and after the trigger generation are extracted from the continuously acquired images, and the respective images are analyzed. First, the object corresponding to the ball in each image (S142), and verifies whether the detected object is viewed (S144). This is substantially the same as the above-described steps S110 and S120.

When the ball is detected and verified in each image, three-dimensional coordinate information of each object verified as the ball is calculated (S146), and the ball is moved from the hitting point of the player to the screen using the calculated ball coordinate information. A ball motion model for the ball to be determined is determined (S148).

The RANSAC algorithm can be used to determine such a motion model of the ball.

For example, the final model of the motion of the ball can be selected by sampling two of the three-dimensional ball coordinates and selecting a model supported by the most data while trying a virtual model.

The coordinates of the extracted virtual model and the ball count the number of data adjacent to the model below a certain threshold. At this time, the data out of the threshold becomes outlier (Noise).

Here, when setting a threshold that distinguishes inlier (True Data) and outlier, the larger the threshold value, the higher the probability of extracting the model, but also increases the probability of extracting an incorrect model. Conversely, the smaller the threshold, the higher the probability of failing model extraction, but the higher the probability of extracting the correct model.

If the model extraction fails even though there is the coordinate data of the ball, it may be because the error of the distortion of the camera or the lens or the precision of the data of the three-dimensional coordinates extracted by the stereo camera technique may be low.

Therefore, it is important to give appropriate thresholds, for example, by running RANSAC once with a strict threshold (small threshold) and twice with a wide range of thresholds (large threshold), increasing the probability of picking the right model. For example, it is possible to increase the probability of failing to extract the model.

In addition, the function of each triaxial direction in determining the motion model of a ball is as follows.

[function in x, y axis direction]

x = a_x * t + b_x

Here, t is a time value, x is an x-direction coordinate value, a_x is an amount of increase of the x-direction coordinates (slope of the function) with respect to time, and b_x means an x-direction coordinate value (intercept) when t is zero.

y = a_y * t + b_y

Here, t denotes a time value, y denotes a y-direction coordinate value, a_y denotes an increase amount of the y-direction coordinates with respect to time (the slope of the function), and b_y denotes a y-direction coordinate value (intercept) when t is zero.

Since the x and y axis movements of the moving ball can be assumed to be constant velocity motions, they can be expressed as linear functions.

[function in z-axis direction]

z = a_z * t + b_z-0.5 * g * t ²

Where g is gravity acceleration, t is time value, z is z-direction coordinate value, a_z is z-direction velocity when t is 0, and b_z is z-direction coordinate value (intercept) when t is 0.

If there are many balls out of the motion model after checking whether there is a ball within a certain distance close to the ball motion model in the interval of expected time t from the calculated time t and x, y, z coordinate values of the ball motion model (In case of exceeding preset number) It is judged that the model is incorrect.

It is easy to distinguish between models that hit the ball in general and models that are irregularly extracted due to other reflections.

On the other hand, if the motion model of the ball is determined as a specific function using the equation of motion in each of the coordinate axis direction as described above, by using the determined ball motion model to distinguish the floor ball rather than the ball used in the player's play Can be classified (S150).

That is, when the ball extracted from the acquired image is out of a predetermined range from the ball motion model, when the average height of the ball in the ball motion model is calculated and positioned below the average height of the calculated ball, the ball motion model The average speed of the ball may be classified as at least one of the cases having a speed lower than the average speed of the calculated ball motion model, and the corresponding ball may be classified as the above-described floor ball.

At this time, the controller checks the number of balls classified as the floor balls by a sensing processing unit, and if the number of balls classified as the floor balls exceeds a preset number, the controller provides an alarm for warning of danger to the player or An alarm requesting to fill the ball into the ball box of the ball machine may be provided (S160).

In the case of tennis, unlike baseball, the predicted hitting position is not determined, and the play area is wider than that of baseball. In addition, the ball machine also has a wide range of vertical and horizontal angles for firing the ball, it is difficult to determine the correct pitching model and the heating model based on a specific position. Therefore, in order to remove the floor ball such as the ball rolling on the floor, the average height information of the data inside the ball motion model is taken as the heating model, and the model of the ball rolling on the floor is removed.

In addition, after checking the average speed of the ball motion model, it may be determined that the heating model is not less than a specific speed.

Since tennis has a high probability of having a predetermined speed or more in the case of a hit ball movement model, tennis can be determined as a heating model when the speed is higher than a specific speed.

Meanwhile, the present invention not only calculates the movement of the ball hit by the player as a ball movement model as described above, but also calculates the position of the player by sensing the movement of the player to continuously calculate the position of the player. By enabling virtual tennis play based on both movements of the player, the player can enjoy dynamic tennis play according to various situations and play styles.

The control device is a ball calculated by the sensing device and the position of the player detected by the sensing device according to the artificial intelligence of the virtual counterpart player implemented with information and at least one of the player's skill level, difficulty, and play style. It is possible to determine the direction and speed of the ball to be provided to the player according to the virtual ball hit by the virtual opponent player in response to the motion model of and allow the ball machine to provide the ball accordingly.

For example, if the player's skill level is high, a virtual play that allows the ball machine to hit the ball in a difficult receive situation by specifying a location far from the player's position detected by the sensing device to provide the ball to the location. Can be implemented.

As described above, the sensing device detects the player and calculates the position thereof. 7 and 8 illustrate an example of detecting the player and other third parties in the captured images, respectively.

Referring to FIG. 6, a background is first extracted from an acquired image (S200). KNN (K-Nearest Neighbor) algorithm may be used for background detection.

The KNN is a kind of clustering algorithm. By using the algorithm, the KNN checks the data value coming from the designated area in the image by frame unit and determines that the data value most received from the designated location is the background of the designated area.

After detecting the background and removing it from the image, an object corresponding to the player is extracted from the image (S210).

An example of detecting an object corresponding to a player is illustrated in FIG. 7. (A) of FIG. 7 is a captured image and (b) corresponds to an image detected by the player.

The difference image is obtained by the extracted background image, an image acquired in real time, and a difference calculation technique. In the difference image, the moving object remains. The image shown in FIG. 7A is obtained from another reference image and a difference image.

When the difference image is obtained as described above, as shown in FIG. 7A, the moving object MO is detected, and a predetermined area including a portion corresponding to the moving object is set as the ROI and set as the region of interest Ra. Since the region of interest Ra can be extracted and analyzed, it can be quickly processed.

Meanwhile, an image in which an object moving through the difference image remains is converted into a binarized image based on a specific threshold value. The contour is detected to extract the features of the extracted objects from the binarized image. The shape of the object is identified by the information on the detected contour.

The object Nb binarized in this manner and detected as an outline is shown in FIG.

However, in general, when the difference image is obtained from the background image and the acquired image, there is no difference in the brightness of the background due to the clothing or shadow of the object (player) to be detected, and thus it is not well distinguished or recognized as a single object. There is a case.

To compensate for this, it is actually the only player, but separates the input objects into a single object. This is to group them into one object if they exist within a preset distance based on the reference object.

As such, the size and shape of the object detected through the unification work may be analyzed to determine that the player is larger than the preset size. At this time, since there is a limit to the area where the player in the image can exist, it can be determined as noise when it is out of the area.

6 again, as described above, the object corresponding to the player is extracted, but another human body may be detected, and thus it is determined (S220). In other words, it is necessary to confirm the uniqueness of the detected player.

For example, in the case of a tennis lesson or the like, a coach or the like for the lesson may enter the camera's angle of view in addition to the player.

If more than one human body is detected in the image, the position of each detected human body is detected (S221), and it is determined whether the human body is within a dangerous area (S222).

This will be described in more detail with reference to FIG. 8.

The sensing device may preset the dangerous area SZ as a preset area adjacent to the screen. In the figure, C30 denotes the court bottom portion on the image and CP denotes the player on the image, respectively.

The sensing device detects an object in the danger zone SZ on the image of every frame acquired and detects a human body from the detected object to inform the controller or the ball machine that a person is in the danger zone. That is, when the sensing device detects a human body in the dangerous area, the controller may request the operation of the ball machine to be stopped by the control device or the ball machine (S223, see FIG. 6).

Through the above process, when a coach who teaches tennis stands at a certain position (eg, in front of the ball launching hole where the ball is fired from the ball machine), the ball machine stops immediately so that the lesson can be smoothly performed without stopping the game. You can proceed and warn you of danger if more than one user appears during play.

6, if no object corresponding to the human body other than the object corresponding to the player is detected in step S220, the sensing apparatus detects the center of gravity of the object corresponding to the extracted player as the expected position of the player. It may be (S230).

The sensing device determines whether the initial position (initial position when the ball is heated) and the estimated position of the calculated player are close to the previously calculated ball movement model (S240), specifically, the initial position of the ball movement model. And when the estimated position of the player is less than or equal to a preset distance, the calculated estimated position of the player may be determined as the position of the player (S251).

As described above, the sensing device can detect not only the ball movement model but also the position of the player and use it for play.

The sensing apparatus may further detect the player's moving distance and play time data by using the position information of the player detected as described above (S252), and calculate the amount of exercise of the corresponding player during the play time using the same (S252). S253).

Through the process as described above, it is possible to adjust the difficulty of the game by measuring the total amount of movement of the player by the cumulative value of the player position change amount, and by setting the direction of the ball provided from the ball machine based on the position of the player.

For example, it is possible to further enhance the fun of virtual tennis play by requesting a mission to designate the player's hitting position when playing virtual tennis based on the player's position, and request a mission to specify the player's position during a specific play such as serve. This can further enhance the fun of virtual tennis play.

In addition, for example, in the case of a female player or a child player, the play time and the like may be appropriately adjusted by measuring the exercise amount of the player as described above.

On the other hand, the present invention calculates the movement of the ball hit by the player as described above as a ball movement model, and also calculates the position of the player by sensing the movement of the player, as well as even the racket held by the player By detecting, the player can play the virtual tennis based on the ball movement hit by the player, the movement of the player, and posture information of the player's play through the racket detection. It is characterized by enabling to enjoy play.

The control device is a player's position detected by the sensing device according to the artificial intelligence of the virtual counterpart player, which is realized as information and at least one of the player's skill level, difficulty, and play style, and the ball calculated by the sensing device. The direction and speed of the ball to be provided to the player according to the virtual ball hit by the virtual opponent player in response to the swing posture information on the motion model of the player, the player swinging in the backhand position or the forehand position, etc. It is possible to determine and to make the ball machine provide the ball accordingly.

For example, if the player's skill level is high, the ball machine will provide the ball to the location by specifying the location far from the player's location detected by the sensing device, and determine in advance whether the player is right-handed or left-handed. By providing the ball in a position that will lead to the player, he can implement virtual play that forces the player to hit the ball in a difficult receive situation.

As described above, the sensing device calculates information on the swing posture by detecting the racket of the player is described with reference to FIGS. 9 to 11.

FIG. 9 is a flowchart illustrating a process for detecting a racket described above, and FIG. 10 is a diagram illustrating a process of detecting a racket held by a user in an image captured according to the flowchart shown in FIG. 9. FIG. 11 is a time series diagram of an operation of each object as an object corresponding to a racket detected according to the process illustrated in FIG. 10.

As shown in FIG. 9, first, an object moving at high speed is detected through a high speed difference image between a real-time acquired image and a previous image (S300).

That is, an object moving above the image acquisition speed of the camera unit is detected through a high-speed image of an image acquired in real time by the camera unit and an image selected as a difference operation image among previously acquired images of the acquired image. It is.

In the case of golf and baseball, golf clubs and baseball bats are roughly similar to the long rod shape, so it is easy to detect golf clubs or baseball bats by detecting lines in the image, but in the case of tennis rackets, In case of real-time acquisition image and difference image, the slow moving object and the fast moving object are both detected, so the racket must be detected through separate detection. There is a problem in that the processing time becomes long.

Therefore, as described above, when the difference image is obtained for the image acquired in real time and the image before it (for example, the image immediately before the image), the time between the two images is very short, such as ms, so that the slow moving There is almost no change in the two images, so that they are removed by the difference operation and moved at such a high speed that they move even during the extremely short time, that is, the racket can be detected by the high speed vehicle image.

Since there may be a high speed moving in addition to the racket, the object detected through the high speed vehicle image may be extracted as a racket object based on the position of the player detected earlier (ie, if it is within a predetermined distance). It may be (S310).

The racquet can be detected more quickly by setting a preset area close to the position of the player as the ROI and finding an object with a high-speed image within the area.

(A), (b) and (c) of FIG. 10 show an example of a result of detecting only the racket object Rb through the high-speed image as described above, and FIGS. 10 (d) and (e) ) And (f) show the detection of the racket object Rb through the high-speed image as described above using the image of another frame.

9 again, as described above, the racket object extracted from each of the plurality of difference images extracted through the high speed vehicle image may be analyzed to detect the position and the moving direction of the racket (S320). Information on the player's posture, such as whether the player's swing is forehand or backhand and whether he is swinging, may be calculated using the detected position of the racket and the direction of movement (S330).

This is illustrated in FIG. 11, which illustrates an example of the images PM1 to PM5 of the racket object detected for the image of each frame as shown in (c) and (f) of FIG. 10. have.

It can be seen that the detected racket is moving with a specific orientation. In FIG. 11, the center of gravity of the racket object moves in the + direction in the y axis (in the order of PM1-> PM2-> PM3-> PM4-> PM5 in FIG. 11), and when the player information is collected, the player is moved to the forehand. It can be judged to be a hit (similar to a left handed player).

Through the above process, the player's posture (backhand / forhand) can be inferred based on the player's striking position and the racket's position and direction, and whether the swing is detected by detecting the racket movement and detecting the ball motion model. You can judge.

Therefore, by using the player information, the detected player position information, the calculated player posture information, the simulation information according to the ball motion model, etc., the direction of the ball to be provided to the player by the ball machine is determined and the ball is provided accordingly. This can be done (S340). That is, the virtual tennis simulation system according to an embodiment of the present invention may be implemented in various play situations and difficulty levels by making good use of the characteristics of tennis, such as dynamics and rally.

In FIG. 12, the play between the player P and the virtual counterpart player VP on the image and driving of the ball machine in the virtual tennis simulation system according to an embodiment of the present invention will be described.

12 illustrates a virtual area provided as image content as if it is a physical area.

That is, it is assumed that the same virtual area VW as the image content implemented on the screen 20 exists behind the screen 20 installed in front of the court bottom 30 where the player P is located. It is shown.

As shown in FIG. 12, when the virtual player VP strikes the ball in the virtual court VC of the virtual area VW, the controller of the controller calculates the trajectory Tb and calculates the trajectory Tb. According to the Tb and the movement conditions, the trajectory Tb is connected to the screen 20 with the ball launching unit 22, and the ball 1 from the ball launching unit 22 corresponds to the trajectory Tb and the moving condition. (It can be fired with the same trajectory and motion conditions, or it can be fired a bit weaker to take into account the real space by applying appropriate weights.)

The controller is configured to store in advance information on at least one of skill level, difficulty level, and play style of the player P, and to control the virtual opponent player VP on the virtual tennis simulation image by artificial intelligence. .

The control device according to the information on at least one of the skill level, difficulty, play style of the player P and the artificial intelligence of the virtual opponent player VP, the position of the player detected by the sensing device (and according to the racket detection The direction, speed, etc. of the ball to be provided to the player P (the ball to be fired by the ball machine as the actual ball) to be provided to the player P according to the virtual ball hit by the virtual opponent player VP corresponding to the swing posture of the player. The ball firing parameter is determined, a simulation image of the trajectory of the virtual ball hit by the virtual opponent player VP is processed according to the determined direction of the ball (actual ball), and the ball firing of the ball machine 100 is performed. Set the direction to correspond to the determined direction of the ball, and as shown in Figure 12 the ball machine 100 on the extension line to the trajectory of the virtual ball hit by the virtual opponent player (VP) The ball (1) in such a manner as to control so as to provide the direction of the determined ball can be carried out, the virtual play tennis.

In this manner, the player P, who plays tennis toward the screen 20 on the actual court floor 30, is the virtual opponent player VP on the virtual court VC on the image projected on the screen 20. You can play a rally of tennis across real and virtual reality.

In this case, the sensing device may be configured to sense only the movement of the ball, or may be configured to sense the position of the player together with the sensing of the movement of the ball, and may reflect the sensing result in the implementation of the rally.

As described above, the rally may be performed by the controller using the sensing data of the sensing apparatus, and the controller may control the ball machine so that the actual ball is fired in response to the virtual ball.

Specifically, first, a case in which the player hits the ball goes to the virtual opponent on the video.

The virtual opponent has the virtual opponent on the image so that the virtual ball corresponds to the ball motion model by using the ball motion model calculated by the sensing processing unit by analyzing the image acquired by the camera unit according to the player's ball strike. Implement a simulation image moving toward the virtual court.

At this time, the control device determines whether the virtual ball simulated on the image is in-play. If it is not in play, it is processed out and the rally starts again from the beginning. If it is in play, the control device uses an artificial intelligence that is set in advance so that the virtual opponent can hit the virtual ball accordingly. .

On the other hand, a case in which the ball hit by the virtual opponent player on the video goes to the player with respect to the player hitting the ball.

The artificial intelligence implemented by the controller is simulated by the player's skill level, difficulty, play style, and the like by the player's hitting, and takes into consideration the batting direction and pitch (ball speed, etc.) of the virtual ball. Determine the motion condition of the virtual ball (receiving) the virtual player's ball (the starting speed and direction of the virtual ball, or the direction and size of the vector if it is set as a vector) and determine the motion condition of the ball. As a result, a simulation image in which the virtual ball is hit and moved is implemented.

At this time, the controller sets the direction and speed of the ball (actual ball) to be provided to the player by the ball machine according to the determined motion condition of the virtual ball, and extends the trajectory of the virtual ball hit by the virtual opponent player. The ball machine provides the ball in the direction and speed of the set ball to cause the player to hit the ball fired from the ball machine.

In this case, when the player hits the ball provided by the ball machine, 'the player hitting the ball goes to the virtual opponent player on the video' and 'the ball hit by the virtual opponent player on the video is returned to the player. If you go repeatedly, the player and the virtual opponent on the video will realize the tennis rally as they cross reality and virtual reality.

As described above, the virtual tennis simulation system according to the present invention, as well as sensing the ball hit by the layer, as well as sensing and analyzing the movement of the player, and further various information such as the sensing and analysis of the movement of the racket held by the player Through the sensing and analysis of the present invention, it is possible to enhance the interest for players who enjoy the virtual tennis play by providing a dynamic and realistic virtual tennis play that cannot be realized in the conventional screen tennis.

100: ball machine, 200: sensing device
210: camera unit, 220: sensing processing unit
230: camera control unit, 300: controller
310: control unit, 320: data storage unit
330: image processing unit, 340: image output unit

Claims (21)

A screen provided in front of a floor of a court in which a player plays tennis to project a virtual tennis simulation image;
A ball machine provided at the rear of the screen to provide a ball to the player;
A camera unit for continuously acquiring an image for a shooting range including a preset area;
A sensing processing unit configured to calculate a ball motion model for the ball by extracting and analyzing an object corresponding to the ball from the acquired image; And
Implement an image of a virtual counterpart player that is the counterpart that the player plays, and control the virtual counterpart player to play according to a predetermined artificial intelligence, and use the ball motion model calculated by the sensing processing unit to perform the virtual A control device for generating and processing tennis simulation images and controlling the ball machine;
The sensing processing unit,
A three-dimensional space on each image of the ball moving by analyzing the image taken by the camera unit in advance by defining a coordinate system of the three-dimensional space and the movement of the ball in each coordinate axis direction in which the player plays. Detect a coordinate data of the image and apply the detected coordinate data of the ball to a function predefined for each coordinate axis direction to calculate a ball movement model for the ball being hit and moved by the player,
The control device,
The virtual counterpart on the image is a virtual ball on the image so as to correspond to the ball motion model by using the ball motion model calculated by the sensing processing unit by analyzing the image acquired by the camera unit according to the ball strike of the player. Implement a simulation video to move towards the virtual court where the player is located,
Implement a simulation image in which the hit virtual ball is moved as the virtual opponent player hits the virtual ball according to a predetermined artificial intelligence,
In such a way that the ball machine provides the ball so that the player can hit it on the extension of the virtual ball,
And the virtual counterpart player on the image can implement a tennis rally while repeating the virtual counterpart player hitting the virtual ball in response to the player hitting the ball. The method of claim 1, wherein the control device,
In response to the ball hit by the camera unit and the sensing processing unit, the virtual ball simulated on the image is determined to be in-play, and in the case of in-play, the virtual opponent player is hit according to the artificial intelligence. Determine the motion conditions of the virtual ball,
Setting the direction and speed of the ball to be provided to the player by the ball machine according to the determined movement condition of the virtual ball,
A manner in which the ball machine provides the ball in the direction and speed of the set ball on an extension line of the trajectory of the virtual ball hit by the virtual opponent player to cause the player to hit the ball fired from the ball machine And the virtual counterpart player on the video with the player can implement a tennis rally. The method of claim 1,
The sensing processing unit,
It is configured to detect the position of the player by extracting and analyzing the object corresponding to the player from the image acquired by the camera unit,
The control device,
And the artificial intelligence determines the motion condition of the virtual ball when the virtual opponent player strikes based on the position of the player detected by the sensing processing unit. A screen provided in front of a floor of a court in which a player plays tennis to project a virtual tennis simulation image;
A ball machine provided at the rear of the screen to provide a ball to the player;
A camera unit for continuously acquiring an image for a shooting range including a preset area;
Sensing to detect the position of the player by extracting and analyzing the object corresponding to the ball from the acquired image to calculate the ball motion model for the ball, and extracting and analyzing the object corresponding to the player from the acquired image Processing unit; And
The virtual tennis simulation image is generated and processed and the ball machine is controlled using the ball motion model calculated by the sensing processing unit, and the player is controlled based on the position of the player detected by the sensing processing unit. A controller for determining a direction of a provided ball and correspondingly processing the virtual tennis simulation image and controlling the ball machine;
The sensing processing unit,
A three-dimensional space on each image of the ball moving by analyzing the image taken by the camera unit in advance by defining a coordinate system of the three-dimensional space and the movement of the ball in each coordinate axis direction in which the player plays. Detect a coordinate data of the image and apply the detected coordinate data of the ball to a function predefined for each coordinate axis direction to calculate a ball movement model for the ball being hit and moved by the player,
The control device,
Implements an image of a virtual counterpart player and a virtual court played by the virtual counterpart player in correspondence with a court bottom part played by the player, and based on the ball movement model calculated by the sensing processing unit, Virtual tennis simulation system, characterized in that configured to implement a simulation image of a virtual ball to exercise the virtual opponent player. delete The method of claim 4, wherein the sensing processing unit,
When the ball extracted from the acquired image is out of a predetermined range from the calculated ball movement model, when the average height of the ball in the calculated ball movement model is calculated and positioned below the average height of the calculated ball And calculating the average speed of the calculated ball motion model to determine whether it is at least one of the cases having a speed lower than the calculated average speed of the ball motion model and using the ball for the player's play. To be categorized as a floor ball rather than a floor ball,
The control device,
When the number of balls classified as the bottom balls is checked by the sensing processing unit and the number of the balls classified as the bottom balls exceeds a preset number, an alarm for warning a danger to the player or the ball machine And provide an alarm requesting the ballbox to fill the ball. The method of claim 4, wherein the sensing processing unit,
The position of the player is tracked by detecting the player from the image of every frame acquired, and the movement distance of the player and the play time of the player are detected by the tracking, and from the detected movement distance and play time. Calculate a momentum of the player,
The control device,
Virtual tennis simulation system, characterized in that configured to provide the player through the video or audio information of the exercise amount calculated by the sensing processing unit. The method of claim 4, wherein the sensing processing unit,
A preset dangerous area as a preset area adjacent to the screen, and detects an object within the area corresponding to the dangerous area on the image of every frame to be detected, and detects a human body from the detected object,
The control device,
And if the human body is detected within the dangerous area by the sensing processing unit, controlling the ball machine to stop the operation of the virtual tennis simulation system. delete delete delete delete The method of claim 4, wherein the sensing processing unit,
An object moving at or above the image acquisition speed of the camera unit through a high-speed image between the image acquired in real time by the camera unit and the image selected as a difference operation among the previously acquired images of the acquired image is a racket object. And extracting the racket object extracted from each of the plurality of difference images extracted through the high-speed vehicle image, and detecting the position and the moving direction of the racket to detect whether the player's swing is forehand or backhand. Virtual tennis simulation system, characterized in that. delete delete delete delete delete delete delete delete

Images