TW201738848A - Method and system for real time tracking of camera movement based on graphic recognition which generate a 3D animation image according to a movement information - Google Patents

Abstract

Provided is a method for real time tracking of camera movement based on graphic recognition, which comprises: arranging a plurality of graphic blocks each of which corresponds to an identification code on the inner wall surface of each side plate of a box body, taking an image of the inside of the box body by using a camera, and transmitting the image and a visual angle corresponding to the image to an arithmetic device, checking and finding three-dimensional coordinates corresponding to each of the identification codes from a comparison table according to the identification codes corresponding to the graphic blocks contained in the image, calculating a two dimensional coordinate of each graphic block in the image, then calculating a coordinate and an angle of the camera according to the three-dimensional coordinates, the two-dimensional coordinates and the visual angle of the plurality of graphic blocks, and transmitting the movement information containing the coordinates, the angles and the visual angles to a 3D animation drawing device so as to generate a 3D animation image according to the movement information, and transmitting the 3D animation image to a display device of the camera for display. The present invention further provides a system for real time tracking of camera movement based on graphic recognition, which is used for providing a movement information of the camera to a 3D animation drawing device so as to generate a real time 3D animation image according to movement information; the system comprises a box body, which is provided with an opening, five connected side plates, the inner surface of each side plate is distributed with a plurality of graphic blocks and each graphic block corresponds to only one identification code; an arithmetic device, which is electrically coupled with the 3D animation drawing device and comprises a processor and a storage unit; and a camera, which is electrically coupled with the arithmetic device and the 3D animation drawing device, and comprises a lens and a display device.

Description

Camera motion instant tracking method and system based on graphic recognition

The invention relates to a camera motion tracking method, in particular to a camera recognition based motion tracking method for drawing a 3D animation scene.

The existing 3D animation drawing software needs to instantly change the angle of view, displacement, rotation and zoom in the 3D scene according to the coordinates, angles and angles of view provided by a virtual camera, so that the 3D animation scene presents the movie and wants to shoot. The angle and the picture, so the operator needs to set the coordinates, angle and angle of view of the virtual camera through an operation interface, such as a mouse, to determine the 3D animation scene. However, the operator controls the virtual camera through the operation interface. It is difficult to control the virtual camera to move and rotate smoothly as if the real camera is operating. Therefore, in the process of creating a 3D animation scene, the virtual camera needs to be constantly modified. The parameters, and the resulting 3D animation scene is not as natural as the image produced by the real camera.

Therefore, if the trajectory of the real camera mirror can be tracked in real time, and the coordinates, angles and angles of view included in the trajectory are used as parameters of the virtual camera for the 3D animation drawing software, the 3D animation drawing software can be generated. The angles and images presented by the 3D animated scene are as natural as the real camera.

Therefore, the object of the present invention is to provide a camera motion tracking method and system based on pattern recognition, which can provide a motion track of a camera to a 3D animation drawing device, so that a 3D animation image can be instantly generated according to the motion track.

Therefore, the present invention is based on a pattern recognition camera motion tracking method for providing a motion information of a camera to a 3D animation drawing device; the method comprises: providing a box having an opening and five connected side panels, A plurality of tiles of the same size are arranged on the inner wall surface of each side plate, and each of the tiles corresponds to a unique identification code; a camera is used to enter the interior of the box from the opening, so that the interior of the box is continuously captured. An image, and immediately transmitting the image and a corresponding angle of view to an operation device; the operation device stores a comparison table, wherein a three-dimensional coordinate corresponding to the identification code of each tile is recorded, and the three-dimensional coordinate indicates the corresponding The coordinate position of the block in the box, and the computing device analyzes the identification code corresponding to the plurality of tiles included in the image transmitted by the camera, so as to find out from the comparison table, corresponding to each identification code. a three-dimensional coordinate, and calculating a two-dimensional coordinate of each of the tiles in the image, to calculate the photography according to the three-dimensional coordinates, the two-dimensional coordinates of the tiles, and the viewing angle When the image is taken, a mark inside the box is marked with an angle, and the motion information including the coordinate, the angle and the angle of view is transmitted to the 3D animation drawing device; and the 3D animation drawing device generates a motion according to the motion information. The 3D animated image is transmitted to a display provided on the camera.

In addition, the present invention provides a camera-based motion tracking system for implementing the above method, which provides a motion information of the camera to a 3D animation drawing device, so that a 3D animation image is instantly generated according to the motion information; the system includes a a box, an arithmetic device and a camera. The box body has an opening and five connected side plates, and the inner wall surface of each side plate is arranged with a plurality of tiles of the same size, each tile corresponding to a unique identification code; the computing device and the 3D animation drawing device are electrically Coupled to include a processor and a storage unit, the storage unit stores a look-up table, wherein a three-dimensional coordinate corresponding to the identification code of each tile is recorded, and the three-dimensional coordinate indicates that the corresponding tile is in the box a coordinate position of the body; the camera is electrically coupled to the computing device and the 3D animation drawing device, and includes a lens and a display. When the camera enters the interior of the casing from the opening, the lens captures the interior of the casing An image of the image and a corresponding view thereof are transmitted to the computing device, and the display displays the 3D animated image transmitted by the 3D animation drawing device; wherein the processor of the computing device analyzes the image included The identification code corresponding to the plurality of tiles is used to find a three-dimensional coordinate corresponding to each identification code from the comparison table, and calculate a two-dimensionality of each tile in the image. Marking, according to the three-dimensional coordinates, the two-dimensional coordinates of the tiles, and the viewing angle, calculating a mark and an angle inside the box when the camera takes the image, and transmitting the coordinates, the angle, and the angle of view including the coordinates The motion information is given to the 3D animation drawing device.

In an embodiment of the invention, the identification code is generated using Hamming code.

In an embodiment of the invention, the camera captures N (N≧30) images per second and transmits the image to the computing device. The processor of the computing device generates the camera to capture the N images per second. N coordinates and angles of the time, and the N coordinates and angles, and the N angles corresponding to the N images constitute a motion trajectory per second of the camera, so that the 3D animation drawing device generates per second according to the motion trajectory Continuous N 3D animated images.

The utility model has the advantages that the plurality of tiles are arranged on the inner wall surface of the box, and the image inside the box is photographed by the camera and transmitted to the computing device, so that the computing device checks according to the identification code corresponding to the tile included in the image. Calculating a three-dimensional coordinate of each tile in the box, and calculating a two-dimensional coordinate of each tile in the image, according to the three-dimensional coordinate, the two-dimensional coordinate of the tile, and a viewing angle of the image, The camera captures an image at an angle and an angle, and then provides a motion track of the camera mirror to the 3D animation drawing device, so that the 3D animation image is generated according to the motion track for the operator to view, so that the operator can display from the camera display. The 3D animation scene directly and intuitively moves the camera to the camera, so that the scene presented by the 3D animation image is naturally smooth as the real camera lens.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are denoted by the same reference numerals.

1 is an embodiment of a camera motion tracking system based on graphics recognition, which is used to implement a camera motion tracking method based on graphics recognition in the present invention to provide a motion track of a physical camera to a 3D animation drawing device. 1; wherein the 3D animation drawing device 1 mainly executes a 3D animation drawing software, such as 3DMAX, Maya, etc., but is not limited thereto. And the 3D animation drawing software needs to generate the angle and the image to be presented by the 3D animation scene according to parameters such as coordinate, angle and angle of view provided by a virtual camera, and this embodiment is to move the physical camera by the virtual camera. (The parameters including the coordinates, angle and angle of view of the solid camera) are provided to the 3D animation drawing software to generate the angle and the image to be rendered by the 3D animation scene according to the motion trajectory of the physical camera.

Therefore, in order to obtain the motion track of the physical camera, the embodiment mainly includes a box 2, an arithmetic device 3, and a camera 4.

As shown in FIG. 1 , the box body 2 is rectangular and has an opening 21 and five connected side plates 22 . For example, the box body 2 is a square body, and the size of each side plate is, for example, 100×100 cm, but not limited thereto. And a plurality of tiles 23 of the same size are disposed on the inner wall surface of each of the side plates 22. For example, each of the inner wall faces is arranged in a matrix form with an average of 14×14 tiles 23, and each tile is, for example, 6×6 cm, but not limited thereto. Moreover, each of the blocks 23 corresponds to a unique identification code; for example, if the box body 2 is a square body and each inner wall surface is provided with 14×14 tiles 23, the five inner wall surfaces are provided with 14×14×5 tiles 23, and There are 14x14x5 corresponding identification codes respectively; more precisely, each tile 23 has a picture as shown in FIG. 1 (only one picture is illustrated in FIG. 1), which is corresponding to The identification code is a code generated by an encoding method such as Hamming Code encoding method, but not limited thereto, that is, the graphic code may also be a QR code or other symbols or numbers. Therefore, the code can be inversely decoded according to the coding method used to obtain the corresponding identification code.

The computing device 3 is electrically coupled to the 3D animation drawing device 1 and mainly includes a processor 31 and a storage unit 32. The storage unit 32 stores a comparison table as shown in Table 1 below, wherein the record of each tile 23 is recorded. A three-dimensional coordinate corresponding to the identification code, the three-dimensional coordinate indicating the coordinate position of the corresponding tile 23 in the box 2. Therefore, if 500 blocks 23 are arranged in the casing 2, m in Table 1 is 500. Table 1

The camera 4 is electrically coupled to the computing device 3 and the 3D animation drawing device 1 and includes a lens 41 and a display 42.

Thereby, when an operator 5 holds the camera 4 and projects the camera 4 into the interior of the casing 2 from the opening 21 of the casing 2, the lens 41 will continuously capture an image 6 inside the casing 2, for example, 30 images 6 are captured in seconds, and the image 6 includes at least a partial image of an inner wall surface of the casing 2, such as shown in FIG. 2, and the images 6 and their corresponding angles of view (angle of view, The field of view is transmitted to the arithmetic unit 3 in sequence.

After receiving the image 6, the processor 31 analyzes the image on the inner wall surface of each image 6 to include several complete tiles 23. Taking FIG. 2 as an example, the processor 31 first determines that the image 6 contains A~ E has five complete tiles 23, and then recognizes the code in each tile 23 and decodes the code to obtain the five identification codes corresponding to the five graphics codes, and from the above list 1 The five-dimensional coordinates corresponding to the five identification codes are found in the comparison table, for example, the identification codes of the five blocks 23 of A~E are 1, 25, 34, 75, 62, respectively, and Find five three-dimensional coordinates corresponding to the five identification codes 1, 25, 34, 75, 62, for example, (X1, Y1, Z1), (X25, Y25, Z25), (X34, Y34, Z34) , (X75, Y75, Z75), (X62, Y62, Z62).

And the processor 31 calculates a two-dimensional coordinate of each of the tiles A~E in the image 6, such as (Xa, Ya), (Xb, Yb), (Xc, Yc), (Xd, Yd), (Xe, Ye), then, the processor 31 can calculate the coordinates of the camera 4 inside the box 2 according to the three-dimensional coordinates of the above-mentioned blocks A~E, the two-dimensional coordinates, and the angle of view when the image 41 is captured by the lens 41. The angle (including the horizontal rotation angle and the vertical elevation angle), and transmits a motion information including the coordinates, the angle, and the angle of view to the 3D animation drawing device 1.

Then, the processor 31 continues to perform the above-mentioned processing procedure for each image 6 continuously transmitted by the camera 4, so as to continuously calculate the coordinates and angles at which the camera 4 captures each image 6, and immediately includes the camera 4 to shoot each The motion information of coordinates, angles, and angles of view of one image is transmitted to the 3D animation drawing device 1. Therefore, the computing device 3 can calculate a motion of the camera 4 in the casing 2 according to the coordinate information of the tile 23 disposed in the casing 2 and the internal image of the casing 2 captured during the mirroring of the camera 4. The trajectory (including coordinates, angles, and angles of view when the camera 4 captures each image 6), and immediately transmits the motion trajectory to the 3D animation drawing device 1. For example, if the camera 4 captures 30 images per second and transmits them to the computing device 3 in real time, the computing device 3 will generate 30 coordinates and angles when the camera 4 captures the 30 images, and the 30 coordinates and angles and The 30 viewing angles corresponding to the 30 images constitute a motion trajectory per second of the camera 4, so that the 3D animation drawing device 1 generates 30 consecutive 3D animation images per second according to the motion trajectory.

Thereby, the 3D animation drawing device 1 supplies the motion track of the camera 4 transmitted from the computing device 3 to the 3D drawing software through the virtual camera, and the 3D drawing software will immediately generate the 3D animation image according to the motion track, and the generated 3D animation will be generated. The image is instantly transmitted to the display 42 of the camera 4, so that the 3D animated image is instantly displayed for viewing by the operator 5. Therefore, the operator 5 can view the scene in the 3D animation image generated by the 3D animation drawing device 1 according to the movement track of the camera 4 through the display 42 while the camera 4 is in the cabinet 2, and can be displayed according to the display 42 by the display 42. The displayed 3D animation scene directly and intuitively adjusts the angle, displacement, rotation and zoom of the 3D animation image scene by changing the position or angle of the camera 4, so that the scene presented by the 3D animation image is as natural as the real camera mirror. Smooth.

It is worth mentioning that the arithmetic device 3 of the present embodiment can also be integrated in the 3D animation drawing device 1.

In summary, in the above embodiment, the plurality of tiles are disposed on the inner wall surface of the casing 2, and the image inside the casing 2 is captured by the camera 4 and transmitted to the arithmetic device 3, so that the computing device 3 includes the tiles according to the image. Corresponding identification code, find the three-dimensional coordinates of each tile in the box 2, and calculate a two-dimensional coordinate of each tile in the image, according to the three-dimensional coordinates and two-dimensional coordinates of the tiles And a view angle of the image, calculating a target and an angle when the camera 4 captures the image, and then providing a motion track of the camera mirror to the 3D animation drawing device 1 to generate a 3D animation image according to the motion track for the operator 5 viewing, allowing the operator 5 to directly and intuitively view the camera 4 from the 3D animation scene displayed on the display 42 of the camera 4, so that the scene presented by the 3D animation image is naturally smooth as the real camera mirror, and indeed achieves the present invention. The efficacy and purpose.

However, the above is only the embodiment of the present invention, and the scope of the invention is not limited thereto, and all the equivalent equivalent changes and modifications according to the scope of the patent application and the patent specification of the present invention are still The scope of the invention is covered.

1‧‧‧3D animation drawing device
2‧‧‧ cabinet
3‧‧‧ arithmetic device
4‧‧‧ camera
5‧‧‧ Operator
6‧‧‧Image
21‧‧‧ openings
22‧‧‧ side panels
23‧‧‧Picture
31‧‧‧ Processor
32‧‧‧ storage unit
41‧‧‧ lens
42‧‧‧ display
A~E‧‧‧ tiles

Other features and effects of the present invention will be clearly shown in the embodiments of the present invention, wherein: FIG. 1 is a schematic diagram of an apparatus mainly used in an embodiment of a camera motion tracking system based on graphics recognition of the present invention; FIG. 2 is a schematic diagram of image content captured by the camera of the embodiment.

1‧‧‧3D animation drawing device

2‧‧‧ cabinet

3‧‧‧ arithmetic device

4‧‧‧ camera

5‧‧‧ Operator

6‧‧‧Image

21‧‧‧ openings

22‧‧‧ side panels

23‧‧‧Picture

31‧‧‧ Processor

32‧‧‧ storage unit

41‧‧‧ lens

42‧‧‧ display

Claims (8)

A camera motion-based real-time tracking method for providing a motion information of a camera to a 3D animation drawing device; the method comprising: providing a box having an opening and five connected side panels, each side panel The inner wall surface is provided with a plurality of tiles of the same size, each tile corresponding to a unique identification code; a camera enters the interior of the bin from the opening to take a mirror image, so as to continuously capture an image inside the box. And transmitting the image and its corresponding view to an operation device; the operation device stores a comparison table, wherein a three-dimensional coordinate corresponding to the identification code of each tile is recorded, and the corresponding tile is indicated by the three-dimensional coordinate a coordinate position in the box, and the computing device analyzes the identification code corresponding to the plurality of tiles included in the image transmitted by the camera, to find a three-dimensional coordinate corresponding to each identification code from the comparison table, And calculating a two-dimensional coordinate of each of the tiles in the image, to calculate according to the three-dimensional coordinates, the two-dimensional coordinates of the tiles, and the viewing angle When the camera captures the image, the camera is marked with an angle inside the box, and transmits the motion information including the coordinate, the angle and the angle of view to the 3D animation drawing device; and the 3D animation drawing device generates the motion information according to the motion information. A 3D animated image is transmitted to a display provided on the camera. The image recognition based camera motion tracking method according to claim 1, wherein the identification code is generated by Hamming code. The image recognition-based camera motion tracking method according to claim 1, wherein the box body is a square body, each side plate has a size of 100×100 cm, and an inner wall surface of each side plate is arranged with an average of 14×14 tiles, and each block has a layout of 14×14 tiles. The size is 6x6 cm. The image recognition-based camera motion tracking method according to claim 1, wherein the camera captures N (N≧30) images per second and transmits the image to the computing device, and the computing device generates the camera every second to capture the N. N coordinates and angles of the image, and the N coordinates and angles, and the N angles corresponding to the N images constitute a motion trajectory per second of the camera, so that the 3D animation drawing device according to the motion trajectory The second produces a continuous N 3D animated image. A camera motion real-time tracking system for providing a motion information of a camera to a 3D animation drawing device to generate a 3D animation image according to the motion information; the system comprises: a box having an opening and Five connected side panels, each of which is provided with a plurality of tiles of the same size, each tile corresponding to a unique identification code; an arithmetic device electrically coupled to the 3D animation drawing device and including a a processor and a storage unit, wherein the storage unit stores a comparison table, wherein a three-dimensional coordinate corresponding to the identification code of each tile is recorded, and the three-dimensional coordinate indicates a coordinate position of the corresponding tile in the box; and a camera is electrically coupled to the computing device and the 3D animation drawing device, and includes a lens and a display. When the camera enters the interior of the casing from the opening, the lens captures an image of the interior of the casing. And transmitting the image and its corresponding view to the computing device, and the display displays the 3D transmitted by the 3D animation drawing device Drawing an image; wherein the processor of the computing device analyzes the identification code corresponding to the plurality of tiles included in the image, to find a three-dimensional coordinate corresponding to each identifier from the comparison table, and calculate each a two-dimensional coordinate of the image in the image, according to the three-dimensional coordinate, the two-dimensional coordinate of the image, and the viewing angle, calculating a point and an angle inside the box when the camera takes the image, and transmitting The motion information including the coordinates, the angle, and the angle of view is given to the 3D animation drawing device. The image recognition based camera motion tracking system according to claim 5, wherein the identification code is generated by Hamming code. The image recognition-based camera motion tracking system according to claim 5, wherein the box body is a square body, each side plate has a size of 100×100 cm, and an inner wall surface of each side plate is arranged with an average of 14×14 tiles, and each block has The size is 6x6 cm. The image recognition-based camera motion tracking system according to claim 5, wherein the lens captures N (N≧30) images per second and transmits the image to the computing device at a time, and the processor of the computing device generates the image every second. The N coordinates and the angle when the camera captures the N images, and the N coordinates and the angle, and the N viewing angles corresponding to the N images constitute a motion trajectory of the camera per second, so that the 3D animation drawing device is based on The motion trajectory produces a continuous N 3D animated images per second.