WO2017054115A1

WO2017054115A1 - Projection method and system with augmented reality effect

Info

Publication number: WO2017054115A1
Application number: PCT/CN2015/090974
Authority: WO
Inventors: 那庆林; 黄彦; 麦浩晃
Original assignee: 神画科技（深圳）有限公司
Priority date: 2015-09-28
Filing date: 2015-09-28
Publication date: 2017-04-06

Abstract

Disclosed are a projection method and system with an augmented reality effect. In a projection display region formed by projection lens units, there are two or more planes for projection images to fall on, wherein at least two planes are intersection planes with intersection angles of less than 180 degrees with respect to a viewer. After being processed by an image processing unit, a pre-set projection application picture forms a picture performing graphic rectification based on each of the planes, and is then mapped by the projection lens units and forms, in the projection display region, a projection image matching each of the planes and having an expected augmented reality effect . The system also comprises an interaction operation unit capable of performing an interaction operation based on the projection image.

Description

Projection method and system with augmented reality effect

Technical field

The present invention relates to projection systems, and more particularly to a projection method and system having an augmented reality effect.

Background technique

The normal projection imaging surface is a smooth plane or a circular arc surface. That is to say, the projection display area of the projector is usually a plane or a circular arc surface, and there is no three-dimensional intersection surface or even a three-dimensional object or space within the projection display area. .

With the continuous expansion of the application of projection technology, products with stereoscopic scene projection functions have also been introduced one after another. For example, in some theater scene layouts or lighting show programs, such projectors are used to project virtual scenes in combination with the shape of the projected object. However, such a virtual scene is usually still a substantially planar projection area, and does not consider how to project an image of a simulated real scene or a virtual object on the intersecting surface through a picture correction on a solid object or a stereoscopic space, and because In theaters and light shows, the audience and the projection screen are far apart, and the augmented reality image is usually designed without considering the preset narrow viewing angle, and it usually does not have interactive manipulation functions.

Summary of the invention

In view of the above-mentioned drawbacks of the prior art, the present invention solves the problem that the conventional projection system can only project a virtual image in a substantially planar projection area, and cannot map a completely virtual image on the intersection surface.

In order to solve the above technical problem, the present invention provides a projection method having an augmented reality effect, in which two or more faces on which a projected image can fall are present in a projection display area, and at least two faces are opposite The intersection angle of the viewer is less than 180 degrees; the method includes the following steps: S1, preset a projection application screen corresponding to each surface; S2, process the preset projection application screen, Forming a picture for performing pattern correction based on the respective faces, and then performing projection by the projection lens unit And projecting, in the projected display area, a projected image having an expected augmented reality effect matched to the respective faces.

In the projection method of the present invention, the projection application screen includes at least one overall image to be mapped on at least two intersecting surfaces; the whole image simultaneously displays a part of the screen on each surface, or continuously maps to each surface. And part of the continuous action is displayed on each face in turn.

The projection method of the present invention further includes the step of preset an optimal viewing angle, the viewer needs to view the best effect at the optimal viewing angle; and apply the screen to the preset projection in the step S2. When processing is performed, pattern correction is performed based on the optimal viewing angle, the respective faces, and the mutual relationship therebetween, and then mapped by the projection lens unit.

The projection method of the present invention further includes a spatial recognition step of performing three-dimensional spatial information collection on a projection space in which the projection display area is located to identify whether there are at least two intersecting surfaces and/or at least one solid object in the projection space. And performing three-dimensional spatial information reconstruction; when processing the preset projection application screen in the step S2, performing graphics based on the optimal viewing angle, the three-dimensional spatial information, and the relationship between them The correction is further mapped by the projection lens unit, and a projection image having an augmented reality effect matching the at least two intersecting faces and/or the at least one solid object is formed in the projection display area.

In the projection method of the present invention, the spatial recognition step further includes recognizing an incident angle of the projected ray with two intersecting faces and/or at least one solid object angle, and performing three-dimensional spatial information reconstruction; When the preset projection application screen is processed in S2, the image is corrected based on the information of the incident angles of the projected rays and the angles of the two intersecting surfaces and/or the at least one solid object, and then mapped by the projection lens unit. And forming a projected image having an augmented reality effect matching the at least two intersecting faces and/or the at least one solid object in the projection display area.

In the projection method of the present invention, in the step S1, a projection application screen matching the at least two intersecting faces and/or at least one solid object is selected from the preset projection application screens in an automatic or manual manner.

In the projection method of the present invention, each of the faces includes a base face on which the projected image is mapped, and one or more additional faces; each of the additional faces is one of the following solid objects: positive A combination of one or more of a square, a cuboid, a pyramid, a cone, a sphere, a cylinder, or a platform.

The projection method of the present invention further includes the step of adjusting a current viewing angle of the viewer, wherein the adjusting manner comprises a manual or an automatic manner, and when the preset projection application screen is processed in the step S2 to form a graphic correction screen, Processing is performed based on the current viewing angle information of the viewer.

In the projection method of the present invention, when the current viewing angle of the viewer is automatically adjusted, the method further includes the step of recognizing the current viewing angle of the viewer, and when the preset projection application screen is processed to form the graphic correction screen in the step S2, Processing is also performed based on the identified viewer current view information.

In the projection method of the present invention, in the step of automatically recognizing the current viewing angle of the viewer, using the sensor to position the viewer, including causing the viewer to wear the glasses with the position sensor, thereby locating the approximate spatial position of the viewer's eyes, in the step The preset projection application screen in S2 is processed according to the viewer's eye space position to form a graphic correction screen.

The present invention also provides a projection system having an augmented reality effect, comprising a projection lens unit and an image processing unit; wherein two or more projection images are projected in the projection display area formed by the projection lens unit The upper surface, wherein at least two of the faces are intersecting faces with an angle of intersection of less than 180 degrees with respect to the viewer; the system further includes a picture storage unit for storing preset projection application screens corresponding to the respective faces; The image processing unit processes the projection application screen to form a screen for performing pattern correction based on the respective faces, and then mapping by the projection lens unit, and forming and each of the projection display regions Face-matched projected image with the desired augmented reality effect.

The projection system of the present invention further includes: a spatial recognition unit configured to perform three-dimensional spatial information collection on a projection space in which the projection display area is located, to identify whether there are at least two intersecting surfaces and/or at least one in the projection space. Stereoscopic objects and performing three-dimensional spatial information reconstruction; wherein the image processing unit processes the selected projection application screen based on the three-dimensional spatial information to form based on the at least two intersecting surfaces and/or at least a picture in which a stereoscopic object is graphically corrected, and then mapped by the projection lens unit, and an augmented reality effect is formed in the projection display area to match the at least two intersecting surfaces and/or the at least one solid object Project an image.

In the projection system of the present invention, the space recognition unit includes a monitoring module and an image analysis and recognition module; the monitoring module performs image signal collection on the projection space; the image analysis The identification module analyzes the monitoring image signal of the monitoring module to identify whether there are at least two intersecting surfaces and/or at least one solid object in the projection space, and performs three-dimensional spatial information reconstruction.

In the projection system of the present invention, the monitoring module is a visible light based monitoring module or an infrared light based monitoring module; the visible light based monitoring module comprises a single monitoring lens or a dual monitoring lens; the infrared light based monitoring module comprises a single monitoring lens, Dual monitor lens, TOF sensor, or structured light sensor.

The projection system of the present invention further includes a viewing angle recognizing unit for recognizing a current viewing angle of the viewer, and the image processing unit is further configured based on the viewing angle recognizing unit when processing the projection application screen to form a graphic correction screen. The viewer's current perspective is processed.

The projection system of the present invention further includes glasses with a position sensor for the viewer to wear the belt for positioning the position of the viewer to position the viewer's eye space, the image processing unit to the projection When the application screen is processed to form a graphic correction screen, processing is also performed based on the viewer's eye space position to form a graphic correction screen.

The projection system of the present invention further includes an information processing unit and an interactive manipulation unit that can perform interactive manipulation based on the projection image, the interaction manipulation unit working in cooperation with the information processing unit to implement interaction based on the projection image Control.

In the projection system of the present invention, the interactive manipulation unit includes a remote controller based on wireless communication, and the remote controller includes a combination of one or more of a mobile phone, a tablet computer, a game controller, or an air mouse; the remote controller uses 2.4G, Bluetooth, or WIFI communication mode to transmit interactive control signals.

In the projection system of the present invention, the interactive manipulation unit includes an infrared remote-based external remote controller and an infrared monitoring lens; the external remote controller may form an infrared spot in the projection display area, and the infrared monitoring lens captures the The infrared light spot, the image processing unit displays a corresponding visible icon according to the position information of the infrared light spot, and controls the visible icon through the external remote controller to implement an interactive manipulation function.

In the projection system of the present invention, the interactive manipulation unit includes a direct touch type interactive manipulation unit, and selects one of the interactive manipulation modes from the dual monitor lens mode, the TOF sensor mode, or the structured light sensor mode.

In the projection system of the present invention, the interactive manipulation unit may share some or all of the functions of the monitoring module with the space recognition unit.

According to the technical solution of the present invention, the actual object in the projection display area can be combined with the preset picture in the projector, and the image content is generated by the “Augmented Reality” (AR) technology and mapped onto the object, and combined with the projection interaction. Technology that generates special projection games to increase the realism of the game, creating a unique game that is not currently available on the market.

DRAWINGS

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

1A, 1B, and 1C are schematic diagrams of a projection system and a scene thereof in a preferred embodiment 1 of the present invention;

2A, 2B, and 2C are schematic diagrams of a projection system and a scene thereof in a preferred embodiment 2 of the present invention;

3A and 3B are schematic diagrams showing a projection system and a scene thereof in a preferred embodiment 3 of the present invention;

4A is a schematic view showing an intersecting surface and an intersecting angle when a rectangular parallelepiped is placed;

Figure 4B is a schematic view showing the intersecting faces and the intersecting angles when the cylinder is placed;

4C is a schematic view showing an intersecting surface and an intersecting angle when a hemisphere is placed;

FIG. 5A is a schematic view showing a window when two intersecting walls are projected into a projection area;

5B is a schematic view showing a triangular block when two intersecting walls are projected as a projection area;

5C is a schematic view of the three intersecting wall surfaces of the corner as a projection area;

6A is a schematic view showing a viewing effect of a viewer when a preset viewing angle is preset;

6B is a schematic diagram of a viewing effect when the viewer does not preset the optimal viewing angle;

6C is a schematic view showing the viewing effect after adjusting the viewer angle on the basis of FIG. 6B.

Figure 7 is a schematic block diagram of a projection system in accordance with one embodiment of the present invention.

detailed description

FIG. 7 is a schematic block diagram of a projection system according to a preferred embodiment of the present invention. The projection system includes a projection lens unit, an image processing unit, and a picture storage unit. The picture storage unit is configured to store a preset projection application screen. .

In operation, the projection display area formed by the projection lens unit has two or more faces on which the projection image is mapped, wherein at least two faces are intersecting faces of less than 180 degrees with respect to the viewer; The image processing unit processes the projection application screen to form a graphic based on each surface The corrected picture is then mapped by the projection lens unit, and a projected image having an expected augmented reality effect matched to each face is formed in the projected display area.

The projection system further includes a spatial recognition unit for performing three-dimensional spatial information collection on the projection space defined by the projection imaging unit and the projection display area thereof to identify whether there are at least two intersecting surfaces and/or at least one in the projection space. Three-dimensional objects and three-dimensional spatial information reconstruction. That is, the spatial recognition unit can automatically recognize the three-dimensional object or the intersecting surface in the projection space, and the image processing unit processes the projection application image based on the three-dimensional spatial information output by the spatial recognition unit to form the at least two intersecting surfaces based on the at least two intersecting surfaces. And/or the at least one solid object is subjected to pattern correction, and then mapped by the projection lens unit, and a projection image matching the at least two intersecting surfaces and/or the at least one solid object is formed in the projection display area.

As can be seen from FIG. 7, in the projection system, the space recognition unit includes a monitoring module and an image analysis and recognition module; the monitoring module performs image signal collection on the projection space; and the image analysis and recognition module analyzes and processes the monitoring image signal of the monitoring module. To identify whether there are at least two intersecting faces and/or at least one solid object in the projection space, and perform three-dimensional spatial information reconstruction.

In a specific implementation, the monitoring module may be a visible light based monitoring module or an infrared light based monitoring module; the visible light based monitoring module may be a single monitoring lens or a dual monitoring lens; the infrared light based monitoring module may be a single monitoring lens, dual monitoring Lens, TOF sensor, or structured light sensor.

As can be seen from FIG. 7, the projection system further includes an information processing unit and an interactive manipulation unit that can perform interactive manipulation based on the projection image, and the interactive manipulation unit cooperates with the information processing unit to implement an interactive manipulation based on the projected image. . Wherein, the interactive control unit can share some or all of the functions of its monitoring module with the space recognition unit. For example, the interactive control unit and the space recognition unit share the monitoring lens in the monitoring module. When the monitoring lens is a dual lens, the interactive control unit shares at least one of the monitoring lenses.

As shown in FIG. 1A, FIG. 1B and FIG. 1C, this is a game application example. The projector 100 is located at the upper end, and the projection lens unit 102 can project a preset projection application screen to the projection display. Within area 118. There is a rectangular parallelepiped 108 in the projection display area 118. From the perspective shown in the figure, the human eye can see the upper surface (A side) 110, the right side surface (C surface) 114 of the rectangular parallelepiped 108, And a front surface (B side) 112. The other area than the rectangular parallelepiped 108 in the projection display area 118 is a normal plane (D plane) 116.

The projector 100 is provided with a picture storage unit for storing a preset projection application screen; after the projection application screen is processed by the image processing unit in the projector 100, a screen for performing pattern correction based on the rectangular parallelepiped 108 is formed, and then the projection lens unit 102 performs Projection is performed and a projected image having the desired augmented reality effect matched to the cuboid 108 is formed within the projection display area 118.

As shown in FIG. 1B, a button graphic, a wall graphic, and a staircase graphic are projected on the rectangular body 108. Specifically, the first button 122 is projected on the A surface 110, the wall pattern is projected on the B surface 112, and the staircase pattern 120 is projected between the D surface and the C surface.

Regarding the expected augmented reality effect, as shown in FIG. 1B, after the rectangular parallelepiped 108 is projected, the B-face 112 is changed to a virtual wall surface, and a virtual button is generated on the A-side 110. Another kind of picture that is expected to augment the reality effect is taken as an example of the step pattern 120 in FIG. 1B, in which a step pattern 120 needs to be projected between the D side and the C side, that is, the step pattern 120 is an overall image. It is simultaneously mapped to two intersecting faces, that is, the C face and the D face, and the staircase pattern 120 displays a part of the screen on the C face and the D face, respectively.

In the specific processing, according to the position and angle of the rectangular parallelepiped 108, the corresponding stair steps, the angles to be stretched, the shadows to be generated, and the like need to be matched, so the graphics of the graphics library must be stretched in advance or in real time (or Partial stretching) and graphic correction form a step that augments the reality effect and generates a second button 124 next to the step. If there is no such rectangular object in the projection area, the projected image does not need to be stretched or corrected as described above, and the projected image will be completely different.

In a specific implementation, the preset projection application screen that has been subjected to the graphics correction may be projected to the rectangular parallelepiped 108 at the designated location. The position of the rectangular parallelepiped 108 can also be changed, and then the position of the rectangular parallelepiped 108 is detected, and then the corresponding projection application screen is selected, and after being processed by the image processing unit, a graphic correction based on the rectangular parallelepiped 108 is formed, and then projected. The lens unit is projected.

The invention further includes an information processing unit and an interactive manipulation unit that can perform interactive manipulation based on the projection image, and the interactive manipulation unit cooperates with the information processing unit to realize interactive manipulation based on the projection image. In this embodiment, a direct touch type interactive control unit, specifically a dual monitor lens mode, is provided. Two

monitoring lenses

104 and 106 are disposed on both sides of the projection lens unit 102, and the projector 100 is also provided in the projector 100. An infrared light source is integrated that emits infrared light that covers the entire projected display area 118. The infrared light source can also be mounted outside the projection lens to directly emit infrared light and cover the entire projection display area 118. When a human finger enters the projection scene to touch the projected image, the two infrared monitoring lenses simultaneously transmit the captured image to the interactive algorithm module to calculate the spatial position of the finger, thereby realizing the stereo image touch operation. Among them, the two infrared monitoring lenses need to be scaled and corrected first, and the image disparity map is acquired at the same time. The spatial reconstruction is realized by image tracking, image segmentation and image recognition, and then the three-dimensional spatial position of the finger is calculated by the algorithm; The three-dimensional high and low undulating field shadows.

In this embodiment, when the human hand enters the projection scene, the two

monitoring lenses

104 and 106 can monitor the spatial position information of the finger, and can further determine the currently performed touch operation by monitoring the motion track of the finger and the spatial coordinate information. For example, clicking, sliding, etc. are realized at different heights of the space. As shown in FIG. 1B, in the game, the finger can click on the top of the game villain to activate or suspend the game villain, and the finger sliding direction is the direction of the game villain; when the game villain is about to board the A side 110 along the virtual ladder If the finger clicks on the first button 122, the map graphic of the A side 100 can be replaced with a swimming pool graphic, and the villain enters the virtual swimming pool and swims happily, as shown in FIG. 1C. If the finger clicks the second button 124, the swimming pool on the A side will disappear and be replaced with the original graphic, and the finger can be used to control the villain to leave to other places.

Among them, the game villain also has the expected augmented reality effect. In the process of boarding the A side along the virtual ladder from the D plane, the game villain is an overall image, which is first mapped on the D surface and then mapped to the virtual ladder. The upper and the second are mapped on the A surface, that is, continuously mapped to each surface, and a part of the continuous motion is sequentially displayed on each surface.

The direct touch interactive control unit in this embodiment may also be a TOF (Time of Flight) sensor mode or a structured light sensor mode.

The basic principle of the TOF sensor is to receive the light returned from the object by the sensor, and to obtain the target distance by detecting the flight (round trip) time of the light pulse. The TOF sensor is similar to the general machine vision imaging process. It consists of several units, such as a light source, an optical component, a sensor, a control circuit, and a processing circuit. The TOF sensor is obtained by the target distance acquired by the in-and-out-reflection detection. . By reconstructing the stereoscopic information of the projection space by the TOF sensor, the position of the finger in the three-dimensional space can be calculated, thereby determining the touch of the space.

The structured light sensor works by using continuous light (near-infrared) to encode the measurement space, reading the encoded light through the sensor, and decoding it by the wafer operation to produce a depth image. However, unlike the traditional structured light method, the light source is not a periodic two-dimensional image coding, but a "body coding" with three-dimensional depth. This kind of light source is called Laser Speckle, which is a random diffraction spot formed when the laser is irradiated to a rough object or penetrates the frosted glass. These speckles are highly random and will change pattern with distance. Any two spots in the space will have different patterns, which is equivalent to marking the entire space, so any object enters the space and When moving, the position of the object can be recorded exactly. By reconstructing the stereoscopic information of the projection space by the structured light sensor, the position of the finger in the three-dimensional space can be calculated, thereby determining the touch of the space.

In the above embodiment, a space recognition unit may be further provided for performing three-dimensional spatial information collection on the projection space where the projection display area is located, to identify whether there are at least two intersecting surfaces and/or at least one solid object in the projection space. And reconstructing the three-dimensional spatial information; that is, even if the rectangular parallelepiped shown in FIG. 1A is randomly placed, the subsequent steps can be automatically identified and completed. At this time, the image processing unit processes the selected projection application screen based on the three-dimensional spatial information to form a screen for performing pattern correction based on each surface of the rectangular parallelepiped, and then is mapped by the projection lens unit, and is formed in the projection display area. The cuboid matches a projected image with an augmented reality effect.

The space recognition unit includes a monitoring module and an image analysis and recognition module; the monitoring module performs image signal collection on the projection space; and the image analysis and recognition module analyzes and processes the monitoring image signal of the monitoring module to identify whether at least the projection space has at least Two intersecting faces and/or at least one solid object and performing three-dimensional spatial information reconstruction. Specifically, the interactive control unit in the first embodiment can also be used as part of the spatial recognition unit. The monitoring module may be a visible light based monitoring module or an infrared light based monitoring module; the visible light based monitoring module may be a single monitoring lens or a dual monitoring lens; the infrared light based monitoring module may be a single monitoring lens, a dual monitoring lens, TOF sensor, or structured light sensor.

The second embodiment of the present invention is shown in FIG. 2A, FIG. 2B and FIG. 2C, wherein the projector 200 is located at the upper end, and the projection lens unit 202 can project a preset projection application screen to the projection display area 218. Inside. There is a conical object 208 in the projection display area 118. From the perspective shown in the figure, the human eye can see the A side 110 and the B side 112 of the conical object. Other areas than the tapered object in the projected display area 118 are conventional planes (D-face) 216.

In this embodiment, the tapered object 208 can be mapped into a pyramid, and the rendered texture is first matched from the graphics library, and then a reasonable stretch is calculated according to the position, the angle, the height of the projector, and the like of the tapered object. (or partial stretching) and graphic correction to generate a solid pyramid of augmented reality effects, creating a virtual pedestal 240 below the A and B faces of the pyramid.

In this embodiment, an interactive control based on a wireless communication remote control is used, specifically an external remote controller 230. The remote controller can be a conventional game controller with front, rear, left, right and other moving buttons and various action buttons. The commands of the remote control can be transmitted to the projector via 2.4G, Bluetooth, or WIFI communication. The remote control 230 shown in the figure is a double mode handle, and the player can select the single mode or the double mode, and the player can control the movement of the game villain in the projection screen, and the game villain can establish the protection circle 242 around the pyramid, or climb Climb the pyramid, you can also graffiti on the pyramid.

The third embodiment of the present invention is shown in FIG. 3A and FIG. 3B, wherein the projector 300 is located at the upper end, and the projection lens unit 302 can project a preset projection application screen into the projection display area 318. There is a cube 308 in the projection display area 318. From the perspective shown in the figure, the human eye can see the upper surface (A side) 310, the right side surface (C surface) 314, and the front surface (B side) of the square body 308. ) 312. Other areas than the cube 308 in the projection display area 318 are the regular plane (D plane) 316.

As shown in FIG. 3B, this is an example of an ancient war game. In the projection display area 318, the A side of the cube 308 is mapped into a city head, and the C side 314 is mapped into a city gate 372. The two sides of the gate are augmented reality. The effect of the city wall 370. Players are free to choose Siege Party 376 or Shoucheng Party 374.

When choosing the defending city, the player can use various resources to defend the city, such as bows and arrows, rolling wood, meteorites, oil pans, etc.; when the siege strength is reduced, the defending city can open the gate to rush out; The irritating nature of the game can also enhance the reality around the city walls into protective guards such as moats and city gate suspension bridges. This game still requires pre-graphic correction of objects placed in the projection area so that the mapped image can generate the background and plot needed in the game.

In this embodiment, the infrared light-based external remote controller 360 is used for the touch operation. The remote controller emits infrared light of a specific wavelength to the projection screen, the projector monitors the lens to recognize, and generates an indication icon 362 at a relative position of the projection display area, and performs front, rear, left and right movement and other corresponding interactive operations according to the remote controller instruction. This makes the game operation similar to the touch panel in the projector environment, and the projection augmented reality effect far exceeds the user experience of the general tablet products. For the working principle of the infrared remote control, refer to the contents of the Chinese invention patent application filed as CN104834394A and CN104020897A. The interactive manipulation methods in the foregoing two patent applications can be used in this embodiment.

In the Chinese invention patent application with the publication number CN104834394A, an interactive display system is disclosed, which comprises an infrared light source and a monitoring device; the monitoring device comprises an interactive module and an interactive control unit; the interactive module comprises a monitoring lens, a beam splitting component, a visible light monitoring component and an infrared light monitoring component; the visible light projection image captured by the monitoring lens is imaged onto the visible light monitoring component to form a first imaging image; the infrared light spot captured by the monitoring lens is imaged onto the infrared light monitoring component to form a second imaging image; The control unit is connected with the visible light monitoring component and the infrared light monitoring component, and outputs position information of the infrared spot to the display unit, and the display unit displays the corresponding visible icon according to the position information of the infrared spot, and the visible icon can realize click, drag, zoom in and out Wait for the operation to achieve the function of human-computer interaction. In the third embodiment, the monitoring lens 304 can be used as a spatial recognition unit, and can detect the position of the rectangular parallelepiped 312 by using visible light and output spatial information for real-time graphic correction and the like.

In the Chinese invention patent application with the publication number CN104020897A, an interactive display system is disclosed, comprising a main control unit for performing information processing, a display unit for generating a display screen according to the received display information, one or more Remote control unit and monitoring module. The main control unit is in communication with the display unit and the monitoring module. Each remote control unit includes an infrared emitting module, and the infrared transmitting module is configured to emit infrared light of two different wavelengths to the display screen to generate an infrared spot. The monitoring module includes a monitoring lens for capturing an infrared spot in the display screen, a first beam splitting element, and a first infrared monitoring component and a second infrared monitoring component for respectively inputting the first infrared light and the second infrared light of different wavelengths . By implementing the technical solution of the present invention, multiple points of interaction operations can be performed in one display screen, and each point can realize operations such as clicking, dragging, zooming in and out of visible icons, thereby improving the user experience of human-computer interaction.

In the embodiment shown in FIG. 4A, a square object 410 is placed on the projection table. In the illustrated visual range, four sets of two-two intersecting planes can be seen, that is, the front front 412 intersects the desktop 408, and the right side 413 is The table top 408 intersects, the upper surface 411 intersects the front side 412, and the upper surface 411 intersects the front side 412. The best visual intersecting plane refers to the intersecting plane with an included angle of less than 180°. In Fig. 4A, the angle 414 and the angle 415 are less than 180°, which are the best visual intersecting planes. Between such intersecting planes, pattern correction processing or 3D image mapping can be performed; and angle 416 and angle 417 are greater than 180° (up to 270°), which is not the best visual intersection plane, and cannot be subjected to pattern correction processing or 3D image. Mapping.

4B shows a cylindrical object 420 placed on a projection table, producing two sets of two-two intersecting curved surfaces, namely a cylindrical surface 422 (the front half cylindrical surface visible to the human eye) intersecting the table top 408, and an upper surface 421 and a cylindrical surface 422. Intersecting, where angle 425 is less than 180°, is the best visual intersecting surface; and angle 426 is greater than 180°, which is not the best visual intersecting surface.

4C shows that the hemispherical object 430 is placed on the projection table, and only one set of two intersecting faces, that is, the spherical surface 431 intersects the table top 408, and the angle 432 is less than 180° in the figure, which belongs to the best visual intersecting curved surface, and can be subjected to graphic correction processing or 3D image mapping.

Fig. 5A is a wall projection embodiment, and the intersecting wall surface is a common space environment in daily life. The projector projects an image onto the intersecting walls, forming a projected image 510 at the left and right wall 506 and wall 508. In this embodiment, spatial three-dimensional position information of the two

wall surfaces

506 and 508 with respect to the projector can be calculated, and the image processing unit performs image correction on the image information on the wall surfaces 506 and 508, and projects half of the two front images. The fan window forms a half-open window 512, as shown in Figure 5A; or an augmented reality image such as a stereoscopic image, such as a virtual triangle image 516 of 5B, can be projected between the two wall corners.

Figure 5C is a corner projection embodiment in which the projector projects an image onto a corner of the room to image the image on top surface 504 and left and right wall surfaces 506, 508. The projection surface without the graphic correction processing is represented by a broken line in the image of the three walls, and the image appears to be an irregular polygon. In this embodiment, spatial three-dimensional position information of three faces can be calculated, and the image processing unit performs image information of three faces. Graphic correction, augmented reality projection mapping is performed at the corners (ie, intersections of the three sides A, B, and C) to form a virtual bird's nest image 518.

The projection system of the invention has a preset optimal viewing angle, and the viewer needs to view the best effect at the optimal viewing angle; when processing the preset projection application screen, the optimal viewing angle and each surface are based on And the interrelationship between them is graphically corrected and then mapped by the projection lens unit.

The current viewing angle of the viewer can also be adjusted as needed, either manually or automatically. In the automatic adjustment of the current viewing angle of the viewer, the method further includes the step of identifying the viewer's current viewing angle, and specifically using the sensor to position the viewer, including causing the viewer to wear the glasses with the position sensor to position the approximate spatial position of the viewer's eyes. The projection application screen for the preset is processed according to the viewer's eye space position to form a graphic correction screen.

As shown in FIG. 6A, the desktop 602 has a rectangular parallelepiped 604. The projected image of the projector 600 is a rectangle 606. When the viewer views the preset optimal viewing angle (the wide side of the table), the image processing unit passes through the image processing unit. After the graphics correction processing is performed on the preset projection image, the projection image seen by the viewer is still rectangular.

If the viewer chooses to view on the short side of the table, then the image seen by the viewer is no longer a rectangle, but rather a graphic as shown in Figure 6B. This is also the reason why the best vision needs to be emphasized in the present invention.

When the viewer's perspective is manually or automatically adjusted, if the image is corrected according to the viewer's transition to the short side of the table, as shown in FIG. 6C, the viewer still sees a rectangle.

Claims

A projection method with an augmented reality effect, wherein there are two or more faces on the projection display area on which the projected image falls, and at least two faces are intersected with respect to the viewer An intersecting surface having an angle of less than 180 degrees; the method includes the following steps:

S1, preset a projection application screen corresponding to each of the faces;

S2, processing the preset projection application screen to form a screen for performing graphics correction based on the respective surfaces, and then mapping by the projection lens unit, and forming a matching with the respective surfaces in the projection display area A projected image with the expected augmented reality effect.
The projection method with augmented reality effect according to claim 1, wherein the projection application screen includes at least one overall image to be mapped on at least two intersecting surfaces; the overall image is simultaneously on each surface A part of the screen is displayed separately, or continuously mapped to each surface, and a part of the continuous motion is sequentially displayed on each surface.
A projection method with an augmented reality effect according to claim 2, further comprising the step of preset an optimal viewing angle at which the viewer needs to view the best effect; When the preset projection application screen is processed in step S2, the image is corrected based on the optimal viewing angle, the respective faces, and the relationship between them, and then mapped by the projection lens unit.
The projection method with augmented reality effect according to claim 3, further comprising a spatial recognition step of performing three-dimensional spatial information collection on a projection space in which the projection display area is located, to identify whether at least the projection space has at least Two intersecting faces and/or at least one solid object and performing three-dimensional spatial information reconstruction;

When the preset projection application screen is processed in the step S2, the image is corrected based on the optimal viewing angle, the three-dimensional spatial information, and the relationship between them, and then performed by the projection lens unit. Mapping and forming at least two intersecting faces in the projected display area A projected image with an augmented reality effect matched to at least one solid object.
The projection method with augmented reality effect according to claim 4, wherein the spatial recognition step further comprises: recognizing an incident angle of the projected ray with two intersecting faces and/or at least one solid object angle, And performing three-dimensional spatial information reconstruction;

When the preset projection application screen is processed in the step S2, the image is corrected based on the information of the incident angles of the two intersecting surfaces and/or the at least one solid object angle, and then projected by the projection light. The lens unit performs mapping and forms a projected image having an augmented reality effect matching the at least two intersecting faces and/or the at least one solid object within the projected display area.
The projection method with augmented reality effect according to claim 5, wherein in the step S1, the at least two intersecting faces are selected from the preset projection application screens in an automatic or manual manner and/or Or a projection application screen that matches at least one solid object.
A projection method with an augmented reality effect according to any one of claims 1 to 6, wherein each of the faces includes a base face on which the projected image is mapped, and one or more additional faces Each of the additional faces is one of the following solid objects: a combination of one or more of a cube, a cuboid, a pyramid, a cone, a sphere, a cylinder, or a table.
The projection method with an augmented reality effect according to any one of claims 3-6, further comprising the step of adjusting a current viewing angle of the viewer, the adjustment manner comprising manual or automatic mode, in the step S2 When the preset projection application screen is processed to form a graphic correction screen, the processing is further performed based on the current viewing angle information of the viewer.
The projection method with augmented reality effect according to claim 8, wherein when the current viewing angle of the viewer is automatically adjusted, the method further includes the step of recognizing the current viewing angle of the viewer, and the preset in the step S2 When the projection application screen is processed to form a graphics correction screen, processing is also performed based on the identified viewer current perspective information.
The projection method with augmented reality effect according to claim 8, wherein in the step of automatically recognizing the current viewing angle of the viewer, using the sensor to position the viewer, including causing the viewer to wear the glasses with the position sensor, thereby positioning The approximate spatial position of the viewer's eyes is processed in the step S2 for the preset projection application screen according to the viewer's eye space position to form a graphic correction screen.
A projection system with an augmented reality effect, comprising a projection lens unit and an image processing unit;

The projection display area formed by the projection lens unit has two or more faces on which the projected image is mapped, wherein at least two faces are intersecting faces with an angle of intersection of less than 180 degrees with respect to the viewer;

The system further includes a picture storage unit for storing preset projection application screens corresponding to the respective faces;

The image processing unit processes the projection application screen to form a screen for performing pattern correction based on the respective faces, and then mapping by the projection lens unit, and forming and each of the projection display regions Face-matched projected image with the desired augmented reality effect.
The projection system with augmented reality effect according to claim 11, wherein the system further comprises:

a space recognition unit, configured to perform three-dimensional spatial information collection on a projection space where the projection display area is located, to identify whether there are at least two intersecting surfaces and/or at least one solid object in the projection space, and perform three-dimensional spatial information reconstruction ;

The image processing unit processes the selected projection application screen based on the three-dimensional spatial information to form a screen for performing graphics correction based on the at least two intersecting surfaces and/or at least one solid object, and then The projection lens unit performs mapping and forms a projected image having an augmented reality effect matching the at least two intersecting faces and/or at least one solid object in the projected display region.
The projection system with augmented reality effect according to claim 12, wherein the space recognition unit comprises a monitoring module and an image analysis and recognition module; the monitoring module performs image signal collection on the projection space; The image analysis and recognition module analyzes the monitoring image signal of the monitoring module to identify whether there are at least two intersecting surfaces and/or at least one solid object in the projection space, and performs three-dimensional spatial information reconstruction.
The projection system with augmented reality effect according to claim 13, wherein the monitoring module is a visible light based monitoring module or an infrared light based monitoring module; the visible light based monitoring module comprises a single monitoring lens or a dual monitoring lens. The infrared light-based monitoring module includes a single monitoring lens, a dual monitoring lens, a TOF sensor, or a structured light sensor.
A projection system with an augmented reality effect according to claim 11, further comprising a view recognition unit for recognizing a current view of the viewer, said image processing unit processing said projected application picture to form a picture correction At the time of the screen, the processing is also performed based on the current viewing angle of the viewer obtained by the viewing angle recognition unit.
A projection system with an augmented reality effect according to claim 15, further comprising glasses with a position sensor for the viewer to wear the belt for positioning the position of the viewer to position the viewer's eyes The spatial position, when the image processing unit processes the projection application screen to form a graphics correction screen, further processes based on the viewer's eye space position to form a graphics correction screen.
A projection system with an augmented reality effect according to claim 11, further comprising an information processing unit, and an interactive manipulation unit that can perform interactive manipulation based on the projection image, the interactive manipulation unit and the information processing The unit cooperates to achieve an interactive manipulation based on the projected image.
The projection system with augmented reality effect according to claim 17, wherein the interactive manipulation unit comprises a remote controller based on wireless communication, and the remote controller comprises a mobile phone, a tablet computer, a game controller, or an air mouse. One or more combinations; the remote control transmits an interactive control signal using 2.4G, Bluetooth, or WIFI communication.
The projection system with augmented reality effect according to claim 17, wherein the interactive manipulation unit comprises an infrared light-based external remote controller and an infrared monitoring lens; the external remote controller can be in the projection display area Forming an infrared spot, the infrared monitoring lens capturing the infrared spot, the image processing unit displaying a corresponding visible icon according to position information of the infrared spot, and controlling the visible icon by the external remote controller to implement interactive control Features.
The projection system with augmented reality effect according to claim 17, wherein the interactive manipulation unit comprises a direct touch interaction control unit and is selected from a dual monitor lens mode, a TOF sensor mode, or a structured light sensor mode. One of the interactive manipulation modes.
The projection system with an augmented reality effect according to claim 17, wherein the interactive manipulation unit and the space recognition unit share part or all of the functions of the monitoring module.