WO2018006377A1

WO2018006377A1 - Holographic projection system and method for real-time interactive animation, and artificial intelligence robot

Info

Publication number: WO2018006377A1
Application number: PCT/CN2016/089221
Authority: WO
Inventors: 杨新宇; 王昊奋; 邱楠
Original assignee: 深圳狗尾草智能科技有限公司
Priority date: 2016-07-07
Filing date: 2016-07-07
Publication date: 2018-01-11
Also published as: CN106462257A

Abstract

A holographic projection system (10) for real-time interactive animation, the system comprising: a motion capturing module (11), comprising at least four image acquisition units (111) and used to capture motion data of an object from at least four angles; an image processing module (12) used to receive and process the motion data in real-time to generate motion drive data, and to drive, according to the motion drive data, rendering of a virtual object to generate. in real-time, animation image data; and a holographic projection module (13) used to project and display the animation image data on a holographic apparatus (131). The present invention further provides a holographic projection method for real-time interactive animation, and artificial intelligence robot (100) employing the holographic projection system (10) and method for real-time interactive animation.

Description

Holographic projection system, method and artificial intelligence robot for real-time interactive animation

Technical field

The invention relates to the field of artificial intelligence, in particular to a holographic projection system, method and artificial intelligence robot for real-time interactive animation.

Background technique

VR (Virtual Reality, VR for short) is a technology that comprehensively utilizes computer graphics systems and various interfaces such as reality and control to provide an immersive feeling in a three-dimensional environment that can be generated on a computer. The front-projected holographic display, also known as virtual imaging technology, is a technique for recording and reproducing real three-dimensional images of objects using the principles of interference and diffraction. At present, holographic projection technology has been widely used in the display of VR images.

In addition, the interaction between robots and people is mainly based on voice or simple actions. This kind of interaction is relatively simple, and with the development of technology, these interaction methods are more and more common, and more and more can not meet the user's pursuit. . Therefore, combining VR technology with robot interaction is an urgent problem for modern technology.

Summary of the invention

In order to solve the above problems, the present invention provides a holographic projection system, method and artificial intelligence robot for real-time interactive animation.

In one embodiment, a holographic projection system for real-time interactive animation is provided, comprising: a motion capture module, comprising at least four image capturing units for capturing motion data of at least 4 angles of an acquired object in real time; an image processing module, Receiving and processing the motion data in real time to generate motion drive data, and driving the rendering virtual object to generate animated image data in real time according to the motion driving data; and a holographic projection module for projecting the animated image data to Displayed on the holographic device.

In another embodiment, a holographic projection method for real-time interactive animation is provided, comprising: capturing motion data of at least 4 angles of an acquired object in real time through at least 4 image capturing units; receiving and processing the motion data in real time to generate Motion drives data and drives rendering based on the motion driven data The virtual object generates animation image data in real time; and projects the animation image data onto a holographic device and displays it.

In yet another embodiment, an artificial intelligence robot is provided, coupled to an artificial intelligence server, and including at least the above-described real-time interactive animation holographic projection system, the holographic projection system projecting according to an output instruction of the artificial intelligence server Holographic animation.

The artificial intelligence robot of the present invention, by combining a holographic projection system, the system comprises: a motion capture module, comprising at least four image capturing units for capturing motion data of at least four angles of the collected object in real time; and an image processing module Receiving and processing the motion data in real time to generate motion drive data, and driving the rendered virtual object to generate animated image data in real time according to the motion drive data; and a holographic projection module for projecting the animated image data to the hologram The device is displayed on the device, which enables the combination of virtual reality technology and artificial intelligence, which greatly improves the user experience with the robot.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any inventive labor.

1 is a schematic diagram of an operation state of an artificial intelligence robot according to an embodiment of the present invention;

2 is a functional block diagram of a real-time interactive animation holographic projection system according to an embodiment of the present invention;

3 is a flowchart of a holographic projection method for real-time interactive animation according to an embodiment of the present invention;

FIG. 4 is a flowchart of a holographic projection method for real-time interactive animation according to another embodiment of the present invention.

detailed description

The technical solutions of the present invention are further described in detail below with reference to the accompanying drawings and specific embodiments. It is apparent that the described embodiments are only a part of the embodiments of the invention, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

Referring to FIG. 1, the present invention provides an artificial intelligence robot 100 that is in data connection with an artificial intelligence server 200. The robot 100 includes at least a real-time interactive animation holographic projection system 10. The holographic projection system 10 projects a hologram animation according to an output instruction of the artificial intelligence server 200.

Referring to FIG. 2 , in particular, the real-time interactive animation holographic projection system 10 includes a motion capture module 11 , an image processing module 12 , and a holographic projection module 13 .

The motion capture module 11 includes at least four image capturing units 111 for capturing motion data of at least 4 angles of the captured object in real time. In this embodiment, the motion capture module 11 includes four image capturing units 111, and the image capturing units 111 respectively capture motion data of four angles of front, back, left, and right of the object. In addition, in this embodiment, the collection object is a 3D software model with an animation effect, and the motion data includes at least one of the following: expression data, motion data, special effect data, and control instructions, where the motion data can be obtained from Intercepted in the motion model.

In one embodiment, the motion capture module includes at least one of an expression data acquisition unit and an action data acquisition unit. The expression data collection mode is configured to analyze image data including the face of the collection object in the model, and determine the expression data according to the facial image data. Wherein, the image data of the face includes at least one of the following: a position of the overall contour of the face, a position of the eye, a position of the pupil, a position of the nose, a position of the mouth, and a position of the eyebrow. The motion data collection unit is configured to collect motion data corresponding to the collection object, where the motion data includes a position and a rotation angle of a world coordinate of a current motion of the collection object.

In other embodiments, the motion capture module may also include more types of acquisition tiles, such as a perimeter environment acquisition module.

The image processing module 12 is configured to receive and process the motion data in real time to generate motion driving data, and drive the rendering virtual object to generate animated image data in real time according to the motion driving data. In this embodiment, the image processing module 12 includes at least a receiving unit 121 and a driving rendering unit 122.

The receiving unit 121 is configured to respectively receive the motion data of the at least 4 angles through one or more channels. In this embodiment, the receiving module receives motion data of four angles of the 3D model from the four image capturing units 111. The motion data is a planar image of 4 angles of view of the 3D picture, and the plane image is simultaneously displayed on one screen in a quarter screen manner.

The driving rendering unit 122 is configured to drive and render the virtual objects corresponding to different perspectives of the viewports of the plurality of image capturing units 111 according to the motion driving data, and the virtual objects of different perspectives to be rendered The animated image data of the multiviewport is synthesized.

The holographic projection module 13 is configured to project the animation image data onto the holographic device 131 (FIG. 1) and display it. In this embodiment, the holographic device 131 is a device with a quadrangular pyramid as an imaging carrier.

The artificial intelligence server 200 is configured to: at least one of: performing time synchronization processing on the motion data to synchronize the motion data after synchronization processing with the collected audio data; and performing uniform frame on the motion data Rate of processing.

Referring to FIG. 3, the present invention also provides a holographic projection method for real-time interactive animation, including:

In step S301, motion data of at least four angles of the acquisition object is captured in real time by at least four image capturing units 111. In this embodiment, the collection object is a 3D model with an animation effect, and the motion data includes at least one of the following: expression data, motion data, special effect data, and control instructions. More specifically, in this embodiment, the step S301 includes at least one of the following sub-steps:

And analyzing image data of the face of the acquisition object in the model, and determining the expression data according to the facial image data, wherein the image data of the face comprises at least one of: a position of an overall contour of the face, an eye Position, position of the pupil, position of the nose, position of the mouth, position of the eyebrows;

Collecting motion data corresponding to the collection object, wherein the motion data includes a position and a rotation angle of world coordinates of a current motion of the collection object.

Step S302, receiving and processing the motion data in real time to generate motion driving data, and driving the rendering virtual object according to the motion driving data to generate animation image data in real time. Specifically, the step of driving the rendering virtual object to generate the animation image data in real time includes: driving and rendering the virtual object corresponding to different perspectives of the viewports of the plurality of image capturing units 111 according to the motion driving data. And the virtual objects of different perspectives that are rendered are combined into the animation image data of the multi-viewport.

In step S303, the moving image data is projected onto the holographic device 131 and displayed.

Referring to FIG. 4, in another embodiment, the method further includes:

Step S304, performing time synchronization processing on the motion data, so that the motion data after the synchronization processing is synchronized with the collected audio data;

Step S305, performing processing of a uniform frame rate on the motion data.

The artificial intelligence robot 100 of the present invention, by combining the holographic projection system 10, the system includes: a motion capture module 11 including at least four image capturing units 111 for capturing motion data of at least 4 angles of the collected object in real time; The processing module 12 is configured to receive and process the motion data in real time to generate motion driving data, and drive the rendering virtual object to generate real-time motion according to the motion driving data. The image data is displayed; and the holographic projection module 13 is configured to project the animation image data onto the holographic device 131 and display the virtual reality technology and the artificial intelligence, thereby greatly improving the interaction experience between the user and the robot.

It should be noted that, through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary hardware platform, and of course, all can be implemented by hardware. Based on such understanding, all or part of the technical solution of the present invention contributing to the background art may be embodied in the form of a software product, which may be stored in a storage medium such as a ROM/RAM, a magnetic disk, an optical disk, or the like. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention or in some portions of the embodiments.

The above are only the preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, and the equivalent changes made by the claims of the present invention are still within the scope of the present invention.

Claims

A real-time interactive animation holographic projection system, comprising:

The motion capture module includes at least four image capturing units for capturing motion data of at least 4 angles of the collected object in real time;

An image processing module, configured to receive and process the motion data in real time to generate motion drive data, and drive the rendered virtual object to generate animated image data in real time according to the motion drive data;

A holographic projection module for projecting the animation image data onto a holographic device and displaying the same.
The system according to claim 1, wherein the acquisition object is a 3D model having an animation effect, and the motion data comprises at least one of the following: expression data, motion data, special effect data, and control instructions.
The system of claim 2 wherein said motion capture module comprises at least one of:

An expression data collecting unit, configured to analyze image data including the face of the collection object in the model, and determine the expression data according to the facial image data, wherein the image data of the face includes at least one of the following: a face The position of the overall contour, the position of the eye, the position of the pupil, the position of the nose, the position of the mouth, the position of the eyebrows;

The action data collecting unit is configured to collect action data corresponding to the collected object, wherein the action data includes a position and a rotation angle of a world coordinate of a current action of the collected object.
The system of claim 1 wherein said system is further coupled to an artificial intelligence server, said server being for at least one of:

Performing time synchronization processing on the motion data to synchronize the motion data after synchronization processing with the collected audio data;

The motion data is subjected to uniform frame rate processing.
The system of claim 1 wherein said image processing module comprises at least:

a receiving unit, configured to receive the motion data of the at least 4 angles;

Driving a rendering unit, configured to drive and render the virtual object corresponding to different perspectives of the plurality of image capturing unit viewports according to the motion driving data, and combine the rendered virtual objects of different viewing angles into multiple viewports The animated image data.
The system of claim 1 wherein said holographic device is formed by a quadrangular pyramid A device like a carrier.
A real-time interactive animation holographic projection method, comprising:

Acquiring motion data of at least 4 angles of the collected object in real time by at least 4 image capturing units;

Receiving and processing the motion data in real time to generate motion drive data, and driving the rendered virtual object to generate animated image data in real time according to the motion drive data;

The animated image data is projected onto a holographic device and displayed.
The method according to claim 7, wherein the collection object is a 3D model having an animation effect, and the motion data comprises at least one of the following: expression data, motion data, special effect data, and control instructions.
The method according to claim 8, wherein the step of capturing motion data of at least 4 angles of the acquired object in real time comprises at least one of the following:

And analyzing image data of the face of the acquisition object in the model, and determining the expression data according to the facial image data, wherein the image data of the face comprises at least one of: a position of an overall contour of the face, an eye Position, position of the pupil, position of the nose, position of the mouth, position of the eyebrows;

Collecting motion data corresponding to the collection object, wherein the motion data includes a position and a rotation angle of world coordinates of a current motion of the collection object.
The method of claim 7 further comprising:

Performing time synchronization processing on the motion data to synchronize the motion data after synchronization processing with the collected audio data;

The motion data is subjected to uniform frame rate processing.
The method according to claim 7, wherein the step of driving the rendered virtual object to generate the animated image data in real time comprises:

And driving the virtual object corresponding to the different viewing angles of the plurality of image capturing unit viewports according to the motion driving data, and synthesizing the rendered virtual objects of different viewing angles into the moving image of the multi-viewport data.
An artificial intelligence robot connected to an artificial intelligence server and comprising at least the real-time interactive animation holographic projection system according to any one of claims 1 to 6, the holographic projection system being output according to the artificial intelligence server Command, project hologram animation.