WO2018188088A1

WO2018188088A1 - Clay toy system based on augmented reality and digital image processing and method therefor

Info

Publication number: WO2018188088A1
Application number: PCT/CN2017/080654
Authority: WO
Inventors: 王荣根
Original assignee: 广州千藤玩具有限公司
Priority date: 2017-04-14
Filing date: 2017-04-14
Publication date: 2018-10-18
Also published as: CN110741327B; CN110741327A

Abstract

A clay toy system based on augmented reality and digital image processing and a method therefor. The system comprises an image collection module, an AR identification module, a display, an arithmetic unit and an I/O interactive control system, wherein the image collection module is used for collecting an image or image data of a physical clay model and transmitting same to the AR identification module; the AR identification module is used for generating a corresponding three-dimensional clay model for the acquired image data of the physical clay model and transmitting same to the arithmetic unit; the arithmetic unit is used for carrying out special effect processing on the acquired three-dimensional clay model, adding an audio sound effect, matching an action, and transmitting the model to the display for displaying; and the I/O interactive control system is used for controlling the arithmetic unit, such control comprising selecting the added special effect, audio sound effect and action. A memory is used for storing data concerning the special effect, the audio sound effect and the action. The system extends the ways in which a traditional clay toy can be played with and solves the problem of the identification of a non-planar entity by means of augmented reality technology.

Description

Clay toy system based on augmented reality and digital image processing and method thereof

Technical field

The invention relates to the application of augmented reality technology in the field of children's toys, in particular to a clay toy system based on augmented reality and digital image processing and a method thereof.

Background technique

Augmented reality technology is a new technology that integrates real world information and virtual world information "seamlessly". It is an entity information (visual information, sound, taste, etc.) that is difficult to experience in a certain time and space of the real world. Touch, etc.), through computer and other science and technology, simulation and then superimposition, the virtual information is applied to the real world, perceived by human senses, thus achieving a sensory experience beyond reality. The real environment and virtual objects are superimposed in real time on the same picture or space. Traditional clay toys have the benefits of improving children's creativity, hands-on ability, and color observation. However, because the plasticity is too strong, there are many skills to master. In the absence of adult education, most children can't make excellent works in the early stage. After the initial freshness, they will not continue to learn and create, and they will not get the growth they should have.

Summary of the invention

In view of the deficiencies of the prior art, the present invention aims to provide a clay toy system and method based on augmented reality and digital image processing, which can quickly and stably identify a three-dimensional physical clay model and submit corresponding recognition accuracy. , complete the evaluation of the work, keep on following Trace positioning synchronization output, combining the virtual 3D model with the real world, and then playing the game interaction of single or multiplayer, which not only extends the traditional clay toy game, but also solves the problem of augmented reality technology recognition of non-planar entities.

In order to achieve the above object, the present invention adopts the following technical solutions:

A clay toy system based on augmented reality and digital image processing, including an image acquisition module, an AR recognition module, a display, an arithmetic unit, and an I/O interactive control system;

The image acquisition module is configured to collect image data of a physical mud glue model and transmit the image data to an AR identification module;

The AR identification module is configured to generate a corresponding three-dimensional mud glue model for the image data of the obtained physical mud model and transmit the image to the operator;

The operator is configured to perform special effects processing on the acquired three-dimensional clay model, add an audio sound effect, and perform an action, and transmit the image to a display for display;

The I/O interactive control system is configured to control an operator, including selecting added effects, audio effects, and actions;

Memory: for storing special effects, audio sound effects, and motion data, the I/O interactive control system and an arithmetic unit are both connected to the memory.

It should be noted that a plurality of image acquisition modules and an operator are included, each image acquisition module respectively corresponding to an operator, and all image acquisition modules and operators are connected to the AR recognition module, and each of the operators respectively Connected to an I/O interactive control system; different image acquisition modules are used to collect image data of different physical mud models and transmit them to the AR recognition module, and the AR recognition module generates corresponding 3D clay models and transmits them to corresponding ones. In the operator, special effects processing is performed by the corresponding operator, and audio sound effects are added. And with the action; the effects, audio effects and actions used by each operator are controlled by the I/O interactive control system to which they are connected.

It should be further noted that the online identification engine module and the battle scene image module are further disposed between the AR identification module and one of the computing units, and the AR identification module, the online battle engine module, the battle scene image module, and the computing device are further disposed. Connected in order;

It should be noted that each I/O interactive control system is connected to a gyroscope control module for inputting control commands to the I/O interactive control system through accelerations in different directions.

It should be noted that a mode selection module is also included for selecting a multi-person interaction mode and a stand-alone mode.

The above method for using the clay toy system based on enhanced display and digital image processing includes the following steps:

S1 uses a mode selection module to select a multi-person interaction mode or a stand-alone mode;

S2 When the multi-person interaction mode is selected, the I/O interactive control system loads the battle engine, and each user uses different image acquisition modules to collect the image data of the physical mud model produced by the user, and selects itself through the I/O interactive control system. The required special effects, audio sound effects and motion processing, the image data of the final physical mud model is processed by the AR recognition module to generate a three-dimensional mud glue model, and then processed by the special effects, audio sound effects and actions of the operator and output to the display for display; All the user's physical mud model is displayed on the display through the AR recognition module and the operator's 3D clay model, thus interacting with other 3D clay models;

When the stand-alone mode is selected, the image acquisition module collects the image of the physical mud glue model and then generates the corresponding three-dimensional mud glue model by the AR recognition module. Shown on the display.

It should be noted that the control command is input to the I/O interactive control system through the gyroscope control module, and the accelerations in different directions respectively represent a control command.

It should be noted that, in step S2, the specific processing method of the AR recognition module of the multi-person interaction mode or the stand-alone mode is:

2.1) using the image acquisition module to perform multi-dimensional data sampling of the physical mud glue model;

2.2) The AR identification module performs coordinate analysis on the collected data to identify the shape rule or irregularity of the physical mud glue model;

2.3) If the physical mud glue model is regular, the sampled 2D/3D model is called to specify the color of the point and perform compensation processing, and then the 2D/3D model is subjected to color matching processing to generate a 3D clay model;

If the physical mud model is irregular:

If it is a pure white base map, the sampling map is generated by supplementing the chromatic aberration, and then the sampling map and the sampled 2D/3D model feature points are matched to finally generate a 3D clay model of the physical mud glue model;

If it is not a pure white background map, the coordinates and color values of the physical feature points are calculated according to the sampled data, and then the three-dimensional mud glue model is combined by the point cloud coordinates, and the feature points of the three-dimensional mud glue model and the physical mud glue model are matched after the combination is completed;

2.4) The memory outputs the action, sound effect and special effect data selected by the I/O interactive control system to the operator, and the operator processes the action, sound effect and special effect on the three-dimensional mud glue model.

The invention has the beneficial effects that the three-dimensional physical mud glue model can be quickly and stably recognized, the corresponding recognition accuracy is submitted, the work evaluation is completed, and the tracking and positioning are continuously stopped. Simultaneous output, combining the virtual 3D model with the real world, and then playing the game interaction of single or multiplayer, not only extends the gameplay of traditional clay toys, but also solves the problem of recognition of non-planar entities by augmented reality technology.

DRAWINGS

Figure 1 is a schematic diagram of the system of the present invention.

[Correct according to Rule 91 26.05.2017]
FIG. 2 is a schematic diagram of a specific processing flow of an AR identification module according to the present invention.

detailed description

The present invention will be further described with reference to the accompanying drawings. It should be noted that the present embodiment provides a detailed implementation manner and a specific operation procedure on the premise of the present technical solution, but the protection scope of the present invention is not limited to the present invention. Example.

As shown in FIG. 1 , a clay toy system based on augmented reality and digital image processing includes an image acquisition module, an AR recognition module, a display, an arithmetic unit, and an I/O interactive control system;

The operator is configured to perform special effects processing on the acquired three-dimensional clay model, add an audio sound effect, and perform an action of the action, and transmit the image to the display for display;

The I/O interactive control system is used to control the operator, including selecting added effects, audio effects, and actions.

Further, the system includes a plurality of image acquisition modules and an operator, each image acquisition module respectively corresponding to an operator, and all image acquisition modules and operators are connected to the AR identification module, and each operator is respectively connected In an I/O interactive control system; different image acquisition modules are used to collect image data of different physical mud models and transmit them to the AR recognition module, and the AR recognition module generates corresponding three-dimensional mud glue models and then transmits them to corresponding operations. In the device, the corresponding operator performs special effects processing, adds audio effects, and is equipped with actions; the effects, audio effects, and actions used by each of the operators are controlled by the I/O interactive control system to which they are connected.

Further, an online competition engine module and a competition scene image module are further disposed between the AR identification module and one of the computing units, and the AR identification module, the online competition engine module, and the battle scene image module and the computing device are further disposed. Connected in sequence; the online battle engine module and the battle scene image module are used to add a match scene to the display.

Further, each I/O interactive control system is connected to a gyroscope control module for inputting control commands to the I/O interactive control system through accelerations in different directions.

Further, the system further includes a mode selection module for selecting the multi-person interaction mode and the stand-alone mode.

S2 When the multi-person interaction mode is selected, the I/O interactive control system loads the battle engine, and each user uses a different image acquisition module to collect the physical mud model produced by itself. Image data, and through the I / O interactive control system to select their own special effects, audio sound effects and motion processing, the image data of the final physical mud model is processed by the AR recognition module to generate a three-dimensional clay model, and then through the special effects of the operator, Audio sound effects and motion processing are output to the display for display; all user's physical mud models are displayed on the display through the AR recognition module and the operator's 3D clay model, thus realizing other three-dimensional clay models. interactive;

When the stand-alone mode is selected, the image acquisition module collects the image of the physical mud glue model and then generates the corresponding three-dimensional mud glue model by the AR recognition module, which is displayed on the display after being processed by the arithmetic unit.

The generated 3D clay model can be shared on social networking sites, can be developed into games, can be photographed or applied to the knowledge science system.

The generated 3D clay model can also carry out intelligent rating activities, intelligently rating the 3D clay model according to the preset rules through the intelligent rating function, and if the intelligent rating is passed, the next activity can be carried out, for example, in the simulation management system. Simulate sales of business props or cards, manage currency, interact with other users, etc.; also pass the customs clearance system, send corresponding rewards according to the results of the rating, and perform photo sharing through the exchange sharing system; if the rating does not pass, you need Transfer to the image acquisition module and continue to create a new physical clay model to challenge.

It should be noted that when the multi-person interaction mode is selected, the battle scene can also be added to the display through the online battle engine and the battle scene module.

It should be noted that the step of generating the three-dimensional mud glue model by the AR identification module includes: analyzing and extracting pattern tracking features for sampling, matching, analyzing, feature value screening, and real-time tracking.

It should be noted that, as shown in FIG. 2, in step S2, the specific processing method of the AR identification module in the multi-person interaction mode or the stand-alone mode is:

If the physical mud model is irregular:

Various changes and modifications can be made by those skilled in the art based on the above technical solutions and concepts, and all such changes and modifications should be included. Within the scope of the claims of the present invention.

Claims

A clay toy system based on augmented reality and digital image processing, comprising: an image acquisition module, an AR recognition module, a display, an arithmetic unit, and an I/O interactive control system; wherein

The image acquisition module is configured to collect image data of a physical mud glue model and transmit the image data to an AR identification module;

The AR identification module is configured to generate a corresponding three-dimensional mud glue model for the image data of the obtained physical mud model and transmit the image to the operator;

The operator is configured to perform special effects processing on the acquired three-dimensional clay model, add an audio sound effect, and perform an action, and transmit the image to a display for display;

The I/O interactive control system is configured to control an operator, including selecting added effects, audio effects, and actions;

Memory: data for storing effects, audio effects, and actions, both of which are coupled to the memory.
The clay toy system based on augmented reality and digital image processing according to claim 1, comprising a plurality of image acquisition modules and an operator, each image acquisition module corresponding to an operator, and all images The acquisition module and the operator are both connected to the AR identification module, and each of the operators is respectively connected to an I/O interactive control system; different image acquisition modules are used to collect image data of different physical mud models and transmit them to the AR. The identification module, the AR recognition module generates a corresponding three-dimensional mud glue model and then transmits it to the corresponding operator, and the corresponding operator performs special effect processing, adds audio sound effects and matches actions; special effects and audio used by each operator Sound effects and actions are controlled by the I/O interactive control system to which they are connected.
The clay toy system based on enhanced display and digital image processing according to claim 2, wherein the AR identification module and one of the computing devices are further provided with an online battle engine module and a battle scene image module. The AR identification module, the online battle engine module, and the battle scene image module and the computing device are sequentially connected;
The clay toy system based on enhanced display and digital image processing according to claim 1 or 2, wherein each I/O interactive control system is connected to a gyroscope control module for accelerating in different directions Input control commands to the I/O interactive control system.
The clay toy system based on enhanced display and digital image processing according to claim 1, further comprising a mode selection module for selecting a multi-person interaction mode and a stand-alone mode.
The method for using a clay toy system based on enhanced display and digital image processing according to claim 1, comprising the steps of:

S1 uses a mode selection module to select a multi-person interaction mode or a stand-alone mode;

S2 When the multi-person interaction mode is selected, the I/O interactive control system loads the battle engine, and each user uses different image acquisition modules to collect the image data of the physical mud model produced by the user, and selects itself through the I/O interactive control system. The required special effects, audio sound effects and motion processing, the image data of the final physical mud model is processed by the AR recognition module to generate a three-dimensional mud glue model, and then processed by the special effects, audio sound effects and actions of the operator and output to the display for display; All the user's physical mud model is displayed on the display through the AR recognition module and the operator's 3D clay model, thus interacting with other 3D clay models;

When the stand-alone mode is selected, the image acquisition module collects the image of the physical mud glue model and then generates the corresponding three-dimensional mud glue model by the AR recognition module, which is displayed on the display after being processed by the arithmetic unit.
The method according to claim 6, wherein the control command is input to the I/O interactive control system by the gyro control module, and the accelerations in different directions respectively represent a control command.
The method of using the method according to claim 6, wherein the specific processing method of the AR recognition module of the multi-person interaction mode or the stand-alone mode in step S2 is:

2.1) using the image acquisition module to perform multi-dimensional data sampling of the physical mud glue model;

2.2) The AR identification module performs coordinate analysis on the collected data to identify the shape rule or irregularity of the physical mud glue model;

2.3) If the physical mud glue model is regular, the sampled 2D/3D model is called to specify the color of the point and perform compensation processing, and then the 2D/3D model is subjected to color matching processing to generate a 3D clay model;

If the physical mud model is irregular:

If it is a pure white base map, the sampling map is generated by supplementing the chromatic aberration, and then the sampling map and the sampled 2D/3D model feature points are matched to finally generate a 3D clay model of the physical mud glue model;

If it is not a pure white background map, the coordinates and color values of the physical feature points are calculated according to the sampled data, and then the three-dimensional mud glue model is combined by the point cloud coordinates, and the feature points of the three-dimensional mud glue model and the physical mud glue model are matched after the combination is completed;

2.4) The memory outputs the action and sound selected by the I/O interactive control system to the operator Effectiveness and special effects data, the operator processes the motion, sound effects and special effects of the 3D clay model.