WO2018112898A1

WO2018112898A1 - Projection method and device, and robot

Info

Publication number: WO2018112898A1
Application number: PCT/CN2016/111754
Authority: WO
Inventors: 骆磊
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2016-12-23
Filing date: 2016-12-23
Publication date: 2018-06-28
Also published as: CN106797455A

Abstract

Disclosed are a projection method and device, and a robot. The method comprises: determining the position of a local projection area from a panoramic projection area (40); and adjusting a projection direction of a projection apparatus according to the position of the local projection area, and enabling the projection apparatus to project a local projection picture, corresponding to the local projection area, to the local projection area (42). By means of the method, the projection apparatus can project the current local projection picture into the local projection area where projection needs to be performed currently when panoramic projection cannot be satisfied, and when the local projection area changes each time, the local projection picture of the current local projection area can be adjusted in real time so as to complete a relevant operation.

Description

Projection method, device and robot

Technical field

The present application relates to the field of projection imaging technology, and in particular, to a projection method, apparatus, and robot.

Background technique

The existing robot has a projection function, can flexibly change the projection position, and project a preset panoramic projection screen to a corresponding area, so that the user can complete the related work according to the panoramic projection screen. For example, in some special engineering construction processes, the projection device of the robot projects the entire projection picture required for construction onto the construction wall, so that the construction personnel can carry out the construction on the construction wall according to the whole projection picture.

In the process of projection of the existing robot, for example, when the entire projection screen is projected onto the construction wall for construction, the actual projection screen is too large, and the entire projection image cannot be completely projected onto the construction wall at a time. In order to solve the above technical problems, the prior art robot projects with a high configuration and an expensive projection device, or a plurality of projection devices simultaneously project to perform one-time projection of the entire projection image.

However, in some applications, the user does not need to complete the related work according to the entire projection picture, and only needs the partial projection picture in the whole projection picture to complete the work within the current range, for example, during the construction process, the construction personnel Only need to obtain the partial projection picture in the current construction range, the construction within the current range can be completed, and when the position of the construction personnel changes, the partial projection picture in the next position range can be acquired, and the construction for the next position can be completed. . However, the inventors of the present application have found that the existing robot or projection apparatus has not been able to solve the technical problem of selecting a partial projection picture for partial projection in a panoramic projection picture.

Summary of the invention

The technical problem to be solved by the present application is to provide a projection method, a device and a robot. The main purpose of the present invention is to solve the technical problem that the projection device of the prior art has not been able to solve the problem of selecting a partial projection image for partial projection in a panoramic projection image.

To solve the above technical problem, the embodiment of the present application provides the following technical solutions:

In a first aspect, an embodiment of the present application provides a projection method, including: determining a position of a partial projection area from a panoramic projection area; adjusting a projection direction of the projection apparatus according to a position of the partial projection area, and The projection device projects a partial projection picture corresponding to the partial projection area to the partial projection area.

Optionally, determining the location of the partial projection area from the panoramic projection area comprises: determining a position of the user in the panoramic projection area; determining, in the panoramic projection area, according to the determined location of the user in the panoramic projection area The location of the local projection area.

Optionally, the determining a location of the user in the panoramic projection area comprises: capturing a panoramic picture of a location where the panoramic projection area is located; determining a location of the user in the panoramic projection area from the panoramic picture.

Optionally, the partial projection picture is obtained by cropping from a pre-stored panoramic projection image, wherein when the pre-stored panoramic projection image is projected to the panoramic projection area, the pre-stored panoramic projection screen covers the location The panoramic projection area.

Optionally, adjusting the projection direction of the projection device according to the position of the partial projection area comprises: moving the projection device according to the position of the partial projection area until the projection optical axis of the moved projection device is perpendicular to the The plane in which the local projection area is located.

Optionally, adjusting the projection direction of the projection device according to the position of the partial projection area comprises: adjusting a projection angle of the projection device according to a position of the partial projection area until a projection optical axis of the adjusted projection device Focusing on the partial projection area.

In a second aspect, an embodiment of the present application provides a projection apparatus, the apparatus comprising: a determining module, configured to determine a position of a partial projection area from a panoramic projection area; and an adjustment module, configured to determine a position according to the partial projection area And adjusting a projection direction of the projection device, and causing the projection device to project a partial projection image corresponding to the partial projection region to the partial projection region.

Optionally, the determining module includes: a first determining unit, configured to determine a location of the user in the panoramic projection area; and a second determining unit, configured to determine, according to the determined location of the user in the panoramic projection area, The position of the partial projection area is determined in the panoramic projection area.

Optionally, the first determining unit includes: a first determining subunit, configured to capture a panoramic picture of a location where the panoramic projection area is located; and a second determining subunit, configured to determine, from the panoramic picture, the user The position of the panoramic projection area.

Optionally, the adjustment module includes: a first adjustment unit, configured to move the projection device according to a position of the partial projection area, until a projection optical axis of the moved projection device is perpendicular to a plane where the partial projection area is located .

Optionally, the adjusting module includes: a second adjusting unit, configured to adjust a projection angle of the projection device according to the position of the partial projection area, until the projection optical axis of the adjusted projection device is focused on the partial projection area .

In a third aspect, an embodiment of the present application provides a robot, including: at least one processor; and a projection device; an adjustment device, configured to adjust a projection direction of the projection device; and communicatively coupled to the at least one processor a memory; wherein the processor is coupled to the projection device, the adjustment device, and the memory, respectively; the memory stores an instruction program executable by the at least one processor, the instruction program being The at least one processor executes to enable the at least one processor to perform the method described above based on the projection device and the adjustment device.

In various embodiments of the present application, the position of the partial projection area is determined from the panoramic projection area, and the projection direction of the projection apparatus is adjusted according to the position of the partial projection area, so that the projection apparatus will have a partial projection picture corresponding to the partial projection area, Project to the local projection area.

In the embodiment of the present application, the partial projection picture in the panoramic projection picture can be projected into the local projection area of the current desired projection in the case that the panoramic projection cannot be satisfied, and can be real-time every time the partial projection area changes. Adjust the partial projection picture of the current local projection area.

DRAWINGS

The one or more embodiments are exemplified by the accompanying drawings in the accompanying drawings, and FIG. The figures in the drawings do not constitute a scale limitation unless otherwise stated.

1 is a schematic structural diagram of a projection apparatus according to an embodiment of the present application;

2 is a schematic structural diagram of a projection apparatus according to another embodiment of the present application;

3 is a schematic structural diagram of hardware of a robot provided by an embodiment of the present application;

3a to 3c are schematic views of a construction scenario provided by an embodiment of the present application;

4 is a schematic flow chart of a projection method provided by an embodiment of the present application;

Figure 5 is a schematic flow chart of step 40 in Figure 4;

FIG. 6 is a schematic flow chart of step 401 in FIG. 5.

detailed description

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.

Please refer to FIG. 1. FIG. 1 is a schematic structural diagram of a projection apparatus according to an embodiment of the present application. As shown in FIG. 1 , the projection device 10 includes a determination module 101 and an adjustment module 102 .

The determining module 101 is configured to determine a location of the partial projection area from the panoramic projection area;

The adjustment module 102 is configured to adjust a projection direction of the projection device according to the position of the partial projection area, and cause the projection device to project the partial projection picture corresponding to the partial projection area to the partial projection area.

The partial projection area is a partial projection area in the panoramic projection area, and the projection device 10 uses the panoramic projection area as the desired projection area according to the task requirement during the projection process. When the projection device 10 is unable to meet the projection of the entire panoramic projection area at a time, the determination module 101 may select a partial region from the panoramic projection region as a partial projection region, and project a corresponding partial projection image to the partial projection region.

For example, the panoramic projection area is square, and the corner points of the panoramic projection area are A(0,0), B(0,100), C(150,0), D(150,100), and the partial projection area can also be square. The coordinates of each corner point are: E (30, 40), F (100, 40), G (30, 50), H (100, 50). Of course, in other alternative embodiments, the partial projection area may not be square, or may be other shapes, such as a circle, a triangle, or the partial projection area may be other irregular shapes.

It should be noted that, when the position of each partial projection area changes, the range of the re-determined partial projection area may be re-determined according to the current operation requirement, instead of limiting the range of the partial projection area in advance. For example, the panoramic projection area is a rectangular area of 1000 cm in length and 500 cm in width, and the current partial projection area may be a rectangular area having a length of 150 cm and a width of 100 cm in the panoramic projection area, in the current partial projection area. After the related operation is completed, the position of the partial projection area changes, and the next partial projection area may be a circular area having a radius of 120 cm in the panoramic projection area.

In the embodiment of the present application, the current partial projection image can be projected into the local projection area of the current desired projection without satisfying the panoramic projection, and the current local region can be adjusted in real time when each partial projection region changes. A partial projection image of the projection area enables dynamic local projection.

FIG. 2 is a schematic structural diagram of a projection apparatus according to another embodiment of the present application. As shown in FIG. 2, the projection device 20 includes a determination module 201 and an adjustment module 202. The determination module 201 and the adjustment module 202 are based on the same idea as the above embodiment, and will not be further described herein. The projection device 20 is different from the above embodiment in that the determination module 201 includes a first determination unit 2011 and a second determination unit 2012.

The first determining unit 2011 is configured to determine a position of the user in the panoramic projection area; and the second determining unit 2012 is configured to determine the position of the partial projection area in the panoramic projection area according to the determined position of the user in the panoramic projection area. Since the partial projection area is determined according to the position of the user in the panoramic projection area, local projection is performed following the position of the user.

It is worth noting that there are various ways to determine the position of the user in the panoramic projection area, such as image recognition, infrared detection, speech recognition, etc. The following is an example of image recognition, infrared detection and speech recognition:

(1) Image recognition, image recognition refers to first acquiring a panoramic picture of the position of the full-image projection area, and then determining the position of the user in the panoramic picture, thereby determining the position of the user in the full-image projection area, the first determining unit 2011 includes A determining subunit 20111 and a second determining subunit 20112.

The first determining subunit 20111 is configured to capture a panoramic picture of the location where the panoramic projection area is located; the second determining subunit 20112 is configured to determine the location of the user in the panoramic projection area from the panoramic picture. Wherein, the position of the user in the panoramic picture relative to the panoramic picture, that is, the position of the user in the panoramic projection area.

Further, when the user moves or stops in front of the panoramic projection area, the first determining subunit 20111 can capture the position of the panoramic projection area according to the acquisition frequency, and output the captured panoramic picture. Of course, in other alternative embodiments, the user's moving speed can also be recognized, and the size of the collecting frequency is adjusted according to the user's moving speed to ensure that the collecting frequency matches the user's moving speed.

Of course, the panoramic projection area corresponds to a panoramic projection screen, and when the panoramic projection screen is projected, the panoramic projection screen can cover the panoramic projection area. The panoramic projection screen is pre-stored in the projection device. After the local projection region is determined, the partial projection image can be obtained by cropping from the panoramic projection image according to the position and size of the local projection region, for example, the second determining subunit 20112 According to the local projection area EFGH are: E (30, 40), F (100, 40), G (30, 50), H (100, 50), according to the size, position and panorama of the panoramic projection area of the partial projection area. Projection area and panoramic view The scale of the picture, the local projection area is mapped into the panoramic projection picture, and the clipping range and position are determined, for example, when the size of the panoramic projection picture is 285 mm x 210 mm, and the respective size parameters of the panoramic projection area are coordinates of four points respectively. Where A(0,0), B(0,100), C(150,0), D(150,100), the size of the partial projection picture is cropped from the panoramic projection picture as obtained by the following four point coordinate parameters:

The abscissa of K1 is (30/150)*285=57mm, and the ordinate is (40/100)*210=84mm, therefore, K1=(57,84);

The abscissa of K2 is (100/150)*285=190mm, and the ordinate is (40/100)*210=84mm, therefore, K2=(190,84);

The abscissa of K3 is (30/150)*285=57mm, and the ordinate is (50/100)*210=105mm, therefore, K3=(57,105);

The abscissa of K4 is (100/150)*285=190mm, and the ordinate is (50/100)*210=105mm, therefore, K3=(190,105);

Therefore, the size of the partial projection picture (190 mm - 57 mm) x (105 - 84) = 133 mm x 21 mm. And the first side of the rectangle has a first side length of 57 mm, and the second side of the rectangle has a second side length of the panoramic picture (285-190=95 mm), and the third side of the rectangle has a long distance The third side of the picture has a length of 84 mm, and the fourth side of the rectangle has a fourth side length of the panoramic picture (210-105 = 105 mm). Since the pre-stored panoramic projection image covers the panoramic projection area, the cropped partial projection image also forms a shape to cover the partial projection area.

Certainly, in some embodiments, a plurality of users may be in front of the panoramic viewing area. When there are multiple users, one user may be matched according to the character characteristics, and the local projection area is determined according to the one user, specifically: first The determining unit 2011 collects the image of the entire wall through the camera, extracts the character features of each character from the image, and matches the extracted character features with the pre-existing character features. If the matching is successful, the matching successful character is the user. And the position of the successfully matched person in the image is the position of the user in the panoramic projection area. Here, the character characteristics are not limited, for example, the character makes a thumbs up gesture, or the character makes a fist gesture, etc., and the matching of the character features can well solve the problem of multiple characters standing on the wall. , causing technical problems in identifying confusion.

(2) Speech recognition: For example, the user stands at a certain position on the wall to issue an "I am here" voice indication, when the voice is recognized, and the position of the user in the panoramic projection area is determined by the directivity of the voice.

(3) Infrared detection: for example, detecting the wall surface by an infrared detector, when the heat is at a certain position on the wall When the heat is different from other locations, it is determined that there is a user at the location, thereby determining the location of the user in the panoramic projection area.

It should be noted that the position of the user in the scene projection area only defines the position of the partial projection area, and the range of the partial projection area is not limited, and the range can be determined according to actual business requirements. For example, the user itself is 180 cm high and 50 cm wide, and then the local part is local. The range of the projection area may cover the location where the user is located or the vicinity of the location of the user according to actual needs, and the range of the partial projection area is not limited to 180 cm*50 cm. Of course, in some embodiments, the position of the partial projection area in the panoramic projection area may also be a fixed range of a certain proportion.

In order to bring about a change in the position of the local projection area in response to the user's movement, in some embodiments, the projection apparatus may adjust its position according to the position of the partial projection area to output a non-distorted partial projection picture. For example, the adjustment module 202 includes a first adjustment unit (not shown). The first adjusting unit is configured to move according to the position of the partial projection area until the projection optical axis of the moved projection device is perpendicular to a plane where the partial projection area is located. Of course, in other alternative embodiments, the adjustment module 202 includes a second adjustment unit (not shown). The second adjusting unit is configured to adjust the projection angle of the projection device according to the position of the partial projection area until the projection optical axis of the adjusted projection device is focused on the partial projection area.

In the embodiment of the present application, a panoramic picture of a position where the user is located in the panoramic projection area can be captured, and the position of the user in the panoramic projection area is determined by the panoramic picture, and a partial projection picture related to the position of the user is obtained by the panoramic picture cropping. In the case that the panoramic projection cannot be satisfied, the current partial projection image is projected into the local projection area of the current desired projection, and each time the local projection area changes, the current partial projection area can be adjusted in real time according to the position of the user. The screen is partially projected, and the projection direction of the projection device is adjusted according to the position of the user to realize dynamic partial projection.

FIG. 3 is a schematic diagram of a hardware structure of a robot provided by an embodiment of the present application. As shown in FIG. 3, the robot 30 is configured to perform a projection method as described above, the robot 30 comprising: one or more processors 310, a projection device 320, an adjustment device 330, and a memory 340. Wherein, one processor 310 is taken as an example in FIG.

The processor 310, the projection device 320, the adjustment device 330, and the memory 340 may be connected by a bus or other means, and the connection through the bus is taken as an example in FIG. Projection device 320 is used to project a local projection area. The adjustment device 330 is used to adjust the projection direction of the projection device. The processor 310 is configured to determine a position of the partial projection area from the panoramic projection area, and according to the position of the partial projection area, The projection direction of the projection device 320 is adjusted by the adjustment device 330, and the projection device 320 is caused to project a partial projection image corresponding to the partial projection region to the partial projection region.

In particular implementations, adjustment device 330 herein may refer to a device for adjusting the position and/or posture of the robot to adjust projection device 320 on the robot. For example, the control robot moves along the projected wall to control the position of the projection. It may also be a device dedicated to adjusting the projection device 320 rather than for adjusting the position and/or posture of the entire robot. The adjusting device 330 may include a driving shaft on which the rotating shaft and the driving rotating shaft rotate, and the projection device 320 is mounted on the rotating shaft and rotatable in accordance with the rotation of the rotating shaft. The processor 310 transmits a driving command to the driving controller of the adjusting device 330 to rotate the rotating shaft of the driving adjusting device 330 to adjust the projection device 320 to a suitable position, so that the projection device 320 will display a partial projection screen corresponding to the partial projection region. , projected to the local projection area.

Optionally, the local projection area may be determined according to the position of the user in the panoramic projection area, and the position of the user is dynamically followed to be projected, and the processor 310 is specifically configured to determine the position of the user in the panoramic projection area; The position of the panoramic projection area determines the position of the partial projection area in the panoramic projection area. Further, when determining the position of the user in the panoramic projection area, it may be determined according to the manner of image recognition. As shown in FIG. 3, the robot further includes an image collection device 350. The processor 310 determines the position of the user in the panoramic projection area, including: A panoramic picture of the position where the panoramic projection area is located is captured by the image capture device 350; the position of the user in the panoramic projection area is determined from the panoramic picture. Of course, in other alternative embodiments, other ways of determining the location of the user in the panoramic projection area, such as speech recognition and infrared recognition, may also be employed.

Of course, the panoramic projection area may correspond to a panoramic projection screen, and the panoramic projection screen refers to a panoramic projection screen covering the panoramic projection area when the panoramic projection screen is projected to the panoramic projection area. The panoramic projection screen is pre-stored in the projection device. After the local projection region is determined, the partial projection image can be obtained by cropping from the panoramic projection image according to the position and size of the partial projection region.

Alternatively, in order to bring about a change in the position of the partial projection area in response to the movement of the user, in some embodiments, the projection apparatus may adjust its position according to the position of the partial projection area to output an undistorted partial projection picture. The adjustment device can also adjust the position of the robot, and the processor 310 adjusts the projection direction of the projection device according to the position of the partial projection area, including: the processor 310 moves the projection device through the adjustment device according to the position of the partial projection area until the projection after the movement The projection optical axis of the device is perpendicular to the plane in which the local projection area is located. Or alternatively, the processor 310 is based on the local projection area The position of the projection device adjusts the projection direction, and the processor 310 adjusts the projection angle of the projection device according to the position of the partial projection region until the projection optical axis of the adjusted projection device is focused on the partial projection region.

It should be noted that the operations performed by the processor 310 may be stored in the memory 320 in the form of program instructions/modules, which are used as a non-volatile computer readable storage medium for storing non-volatile software programs, The non-volatile computer executable program and the module, such as the program instructions/modules corresponding to the projection method in the embodiment of the present application (for example, the respective modules shown in FIG. 1, the respective modules and the respective units shown in FIG. 2). The processor 310 performs various functional applications and data processing of the projection apparatus by executing non-volatile software programs, instructions, and modules stored in the memory 320, that is, implementing the projection method of the following method embodiments and the apparatus embodiments described above. The function of each module.

Memory 320 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 320 can optionally include memory remotely located relative to processor 310, which can be coupled to processor 301 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The program instructions/modules are stored in the memory 320, and when executed by the one or more processors 301, perform a projection method in any of the method embodiments described below, for example, performing the following description of FIG. 4 to The respective steps shown in Fig. 6 can also realize the functions of the respective modules shown in Fig. 1 described above, or the functions of the respective modules and the respective units shown in Fig. 2.

A person skilled in the art should further appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, in order to clearly illustrate hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application. The computer software can be stored in a computer readable storage medium, which, when executed, can include the flow of an embodiment of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only storage memory, or a random storage memory.

For the convenience of the reader, the following examples are explained:

In some operations, the robot needs to project the projection screen required for the job to the work area. In order to cooperate with the operator to complete the work, the robot can select two ways to project during the projection process: the first type: the robot uses the entire work area as the panoramic projection area, and projects the panoramic projection picture corresponding to the panoramic projection area to The panoramic projection area. Secondly, the robot selects a local work area currently associated with the worker from the entire work area as a partial projection area according to the work range of the worker, and projects a partial projection screen corresponding to the partial projection area to the partial projection area. If the current projection device cannot project the panoramic projection screen corresponding to the panoramic projection area to the panoramic projection area, the second operation mode can be selected to complete the task. This embodiment mainly describes the second operation mode.

Referring to FIG. 3a, in FIG. 3a, the construction wall 3a0, that is, the panoramic projection area, includes a bookshelf area 3a01, a background area 3a02, a television installation area 3a03, and a potting area 3a04, and each area is located at the position of the panoramic projection area 3a0. The designer pre-plans so each area has a specified size. Before the decoration, the construction personnel need to carry out marking and marking for each area, so that the follow-up construction personnel can complete the next operation. If the first method is used for projection, since the width of the house is limited, the projection screen cannot be filled once to the construction wall 3a0, and the construction personnel generally do not simultaneously construct the entire construction wall 3a0. Further, before the construction, the construction personnel generally have a familiarity with the engineering design drawings required for the construction. During the construction, it is only necessary to obtain a design plan within a certain range around the current construction to start the precise construction. Therefore, in order to realize the projection of the panoramic projection picture, the robot of the first embodiment needs to be equipped with a high projection device, which increases the design cost of the robot, and the second method can meet the construction requirements.

Referring to Figure 3b, the constructor 3b0 stands on the ground to mark and construct the side elevation of the bookshelf 3b1. At this time, the robot can use the side elevation of the bookshelf 3b1 as a partial projection area to project the local projections of the current construction personnel. The screen 3b2 (that is, the projection screen related to the side elevation of the bookshelf 3b1) is projected to the partial projection area, where the partial projection area and the partial projection screen 3b2 overlap.

When the construction position of the construction worker 3b0 changes, for example, referring to FIG. 3c, the constructor 3b0 stands on the line 3c1 of the right edge of the background area 3a02 of the ladder 3c0 and the construction line, and the robot can set the background area 3a02 The right edge scribe line region 3c1 is used as a partial projection region, and the local projection screen 3c2 related to the current constructor (ie, the projection image related to the right edge of the background region 3a02) is projected to the local projection region, where the partial projection region and the partial projection are respectively The screen 3c2 overlaps.

In the case that the panoramic projection cannot be satisfied, the robot can project the current partial projection image into the local projection area of the current desired projection, and can be rooted every time the partial projection region changes. The partial projection picture of the current partial projection area is adjusted in real time according to the position of the user, and the projection direction of the projection device is adjusted according to the position of the user to realize dynamic partial projection.

It should be noted that the explanation of the embodiment of the projection method based on the robot here is only an embodiment, and the projection method and the device included in the robot are not limited, and any local projection region is determined from the panoramic projection region. Position and adjust the projection direction of the projection device according to the position of the partial projection area, so that the projection device can project the partial projection image of the position when only the partial projection picture of the current position is needed, which should fall within the protection of the present application Within the scope.

4 is a schematic flow chart of a projection method provided by an embodiment of the present application. As shown in FIG. 4, the projection method includes:

Step 40: Determine a position of the partial projection area from the panoramic projection area;

Step 42: Adjust the projection direction of the projection device according to the position of the partial projection area, and cause the projection device to project the partial projection picture corresponding to the partial projection area to the partial projection area.

The partial projection area is a partial projection area in the panoramic projection area. During the projection process, the robot will use the panoramic projection area as the desired projection area according to the task requirements. When the robot cannot satisfy the projection of the entire panoramic projection area at a time, the robot can select a partial region from the panoramic projection region as a partial projection region, and project a corresponding partial projection image to the partial projection region.

In the embodiment of the present application, it is capable of projecting the current partial projection image into the local projection area of the current desired projection without satisfying the panoramic projection, and in each partial projection area. When the domain changes, the local projection picture of the current local projection area can be adjusted in real time to realize dynamic local projection.

In some embodiments, the robot can accurately project according to the needs of the user located in front of the panoramic projection area so that the user can efficiently accomplish the work purpose. Optionally, as shown in FIG. 5, the difference from the foregoing embodiments is that step 40 includes:

Step 401: Determine a location of the user in the panoramic projection area;

Step 402: Determine the position of the partial projection area in the panoramic projection area according to the determined position of the user in the panoramic projection area.

Since the partial projection area is determined according to the position of the user in the panoramic projection area, local projection is performed following the position of the user.

(1) Image recognition, image recognition refers to capturing a panoramic picture of the location of the full-image projection area, and then determining the position of the user in the panoramic picture, thereby determining the position of the user in the full-image projection area. Specifically, as shown in FIG. 6, step 401 includes:

Step 4011: Capture a panoramic picture of the location where the panoramic projection area is located;

Step 4012: Determine the position of the user in the panoramic projection area from the panoramic picture.

Of course, the panoramic projection area corresponds to a panoramic projection screen, and when the panoramic projection screen is projected, the panoramic projection screen can cover the panoramic projection area. The panoramic projection screen is pre-stored in the projection device. After the local projection region is determined, the partial projection image can be obtained by cropping from the panoramic projection image according to the position and size of the local projection region. For example, the robot can be based on the local projection region. EFGH are: E (30, 40), F (100, 40), G (30, 50), H (100, 50), according to the size, position and panoramic projection area of the panoramic projection area according to the local projection area. The proportion of the panoramic projection screen is mapped to the panoramic projection screen to determine the clipping range and position, for example, when the size of the panoramic projection screen is 285 mm x 210 mm, and the respective size parameters of the panoramic projection area are respectively Is the coordinates of four points, where A(0,0), B(0,100), C(150,0), D(150,100), the size of the partial projection picture is cropped from the panoramic projection picture as the following four points The coordinate parameters are obtained:

Of course, in some embodiments, there may be multiple users in front of the panoramic viewing area. When there are multiple users, one user may be matched according to the character characteristics, and the local projection area is determined according to the one user, specifically: the robot may The image of the entire wall is captured by the camera, the character features of each character are extracted from the image, and the extracted character features are matched with the pre-existing character features. If the matching is successful, the matching character is the user, and the matching is performed. The position of the successful character in the image is the position of the user in the panoramic projection area. Here, the character characteristics are not limited, for example, the character makes a thumbs up gesture, or the character makes a fist gesture, etc., and the matching of the character features can well solve the problem of multiple characters standing on the wall. , causing technical problems in identifying confusion.

(3) Infrared detection: For example, the wall surface is detected by an infrared detector. When the heat of a certain position on the wall surface is different from the heat of other positions, it is determined that the user exists at the position, thereby determining the position of the user in the panoramic projection area.

It should be noted that the position of the user in the projection area of the scene only defines the position of the partial projection area, The range of the local projection area is defined, and the range can be determined according to actual business requirements. For example, the user itself is 180 cm high and 50 cm wide, and the range of the partial projection area can cover the location of the user or the location of the user according to actual needs. The vicinity of the area is not limited, and the range of the partial projection area is not limited to 180 cm * 50 cm. Of course, in some embodiments, the position of the partial projection area in the panoramic projection area may also be a fixed range of a certain proportion.

In order to bring about a change in the position of the local projection area in response to the user's movement, in some embodiments, the projection apparatus may adjust its position according to the position of the partial projection area to output a non-distorted partial projection picture. For example, the robot moves according to the position of the partial projection area until the projection optical axis of the moved projection device is perpendicular to the plane in which the partial projection area is located. Since the projection optical axis of the projection device is perpendicular to the plane where the partial projection region is located, the partial projection image projected by the projection device can ensure no distortion. For example, when the partial projection area is located to the left of the projection device, and the projection device detects that the angle between the current projection optical axis and the plane where the partial projection area is located is 30 degrees, the projection device can move along the horizontal line until the projection device The projection optical axis is perpendicular to the plane of the local projection area, and the projection device projects the partial projection image to the partial projection area, thereby enabling the user to complete the job more accurately.

Optionally, the robot can also implement the adjusted control device according to the position of the partial projection area without rotating the projection device, but only by rotating the projection angle of the projection device to change the direction of the projection optical axis of the projection device. The projection optical axis is focused on the local projection area, thereby effectively projecting a partial projection area of the partial projection picture.

Further, when the partial projection image is projected by rotating the projection device, since the projection optical axis and the partial projection region of the projection device are inclined, in some implementation scenarios, the partial projection image projected by the projection device may be distorted (partial projection image) Deviating from the desired projection picture), therefore, in order to output the undistorted partial projection picture, the projection device corrects the partial projection picture in advance by the image correction algorithm when projecting the partial projection picture, so that the projection device can be tilted at a certain horizontal angle and vertical The tilt angle responds to the local projection area at different positions, and outputs an undistorted partial projection picture, thereby enabling the user to complete the job more accurately.

In the embodiment of the present application, a panoramic picture of a position where the user is located in the panoramic projection area can be captured, and the position of the user in the panoramic projection area is determined by the panoramic picture, and a partial projection picture related to the position of the user is obtained by the panoramic picture cropping. In the case that the panoramic projection cannot be satisfied, the current partial projection image is projected into the local projection area of the current desired projection, and in each session When the projection area changes, the partial projection screen of the current partial projection area can be adjusted in real time according to the position of the user, and the projection direction of the projection device is adjusted according to the position of the user to realize dynamic partial projection.

The above description is only the embodiment of the present application, and thus does not limit the scope of the patent application, and the equivalent structure or equivalent process transformation made by using the specification and the drawings of the present application, or directly or indirectly applied to other related technologies. The fields are all included in the scope of patent protection of this application.

Claims

A projection method, comprising:

Determining the position of the partial projection area from the panoramic projection area;

And adjusting a projection direction of the projection device according to the position of the partial projection area, and causing the projection device to project a partial projection picture corresponding to the partial projection area to the partial projection area.
The method according to claim 1, wherein the determining the location of the partial projection area from the panoramic projection area comprises:

Determining the location of the user in the panoramic projection area;

The position of the partial projection area is determined in the panoramic projection area based on the determined position of the user in the panoramic projection area.
The method according to claim 2, wherein the determining the location of the user in the panoramic projection area comprises:

Capture a panoramic picture of the location of the panoramic projection area;

A position of the user in the panoramic projection area is determined from the panoramic picture.
The method of claim 1 wherein

The partial projection picture is cropped from a pre-stored panoramic projection picture, wherein the pre-stored panoramic projection picture covers the panoramic projection area when the pre-stored panoramic projection picture is projected to the panoramic projection area.
The method according to any one of claims 1 to 4, wherein the adjusting the projection direction of the projection device according to the position of the partial projection area comprises:

According to the position of the partial projection area, the projection device is moved until the projection optical axis of the moved projection device is perpendicular to the plane in which the partial projection area is located.
The method according to any one of claims 1 to 4, wherein the adjusting the projection direction of the projection device according to the position of the partial projection area comprises:

According to the position of the partial projection area, the projection angle of the projection device is adjusted until the projection optical axis of the adjusted projection device is focused on the partial projection area.
A projection device, comprising:

a determining module for determining a position of the partial projection area from the panoramic projection area;

And an adjustment module, configured to adjust a projection direction of the projection device according to the position of the partial projection area, and cause the projection device to project a partial projection image corresponding to the partial projection area to the partial projection area.
The apparatus according to claim 7, wherein the determining module comprises:

a first determining unit, configured to determine a location of the user in the panoramic projection area;

And a second determining unit, configured to determine a location of the partial projection area in the panoramic projection area according to the determined location of the user in the panoramic projection area.
The apparatus according to claim 8, wherein the first determining unit comprises:

a first determining subunit, configured to capture a panoramic picture of a location where the panoramic projection area is located;

a second determining subunit, configured to determine, from the panoramic picture, a location of the user in the panoramic projection area.
The device of claim 7 wherein:

The partial projection screen is cropped from a pre-stored panoramic projection screen, wherein the pre-stored panoramic projection screen covers the panoramic projection area when the pre-stored panoramic projection screen is projected to the panoramic projection area .
The apparatus according to any one of claims 7 to 10, wherein the adjustment module comprises:

The first adjusting unit is configured to move the projection device according to the position of the partial projection area until the projection optical axis of the moved projection device is perpendicular to a plane where the partial projection area is located.
The apparatus according to any one of claims 7 to 10, wherein the adjustment module comprises:

The second adjusting unit is configured to adjust a projection angle of the projection device according to the position of the partial projection area until the projection optical axis of the adjusted projection device is focused on the partial projection area.
A robot characterized by comprising:

At least one processor; and,

Projection device

An adjusting device for adjusting a projection direction of the projection device;

a memory communicatively coupled to the at least one processor; wherein

The processor is respectively connected to the projection device, the adjusting device and the memory;

The memory stores an instruction program executable by the at least one processor, the instruction program being executed by the at least one processor to enable the at least one processor to be based on the projection device and the adjustment device The method of any of claims 1-6.