WO2017036285A1 - Method and system for communicating and indicating information of same scene at different places - Google Patents

Abstract

A method and system for communicating and indicating information of a same scene at different places. The method for communicating and indicating information of the same scene at different places comprises: an image of a scene seen by eyes of an indication receiver is acquired; the acquired image of the scene is transmitted to an indication provider, the indication provider adds an indication icon at an indicated position in the image of the scene and transmits the image back to the indication receiver, wherein the added indication icon at least comprises an indication of action or direction; the indication receiver acquires the image of the scene containing the indication icon and acts according to the indication of the indication icon, and an image captured after the indication receiver acts according to the indication icon of the indication provider is fed back to the indication provider. Also disclosed is a system for communicating and indicating information of a same scene at different places, comprising a camera component, an indication provider intelligent terminal and an indication receiver intelligent display component. The present invention can effectively eliminate communication barriers when cognitive differences towards an object being described exist between two parties and realize remote acquisition of the synchronous field of view among a large group of people, thus enabling an increased efficiency, accuracy and real-time synchronization of communication through the indication icon.

Description

Method and system for indicating information exchange in different scenes in different places

 Technical field

 The present invention relates to the field of remote communication indication, and in particular relates to a method and system for indicating information exchange in a different place.

 Background technique

At present, we often encounter two random people in different places, we need to point out the specific targets in the indicated scene. It is best to have the same field of view, and both sides can clearly point out the targets in the same field of view. The characteristics of the target in different orientations of the space. In reality, communication between people is hindered by cognitive differences. When two people face the same picture, they cannot guarantee the common cognitive ability of objects in the picture due to differences in cognitive levels. Make two people with the same high degree of education, but due to different professions, different memory, different ages, etc., there is no guarantee that the objects in the picture have common cognitive ability.

 Suppose two people see the same picture at the same time. There are only two simple objects, cuboids and pentahedrons, when A and B are When the two are in the same place and face to face, regardless of whether any party has the concept of a cuboid and a pentahedron, he only needs to point the finger at the object, and the other party can understand; but when A, B When the two are in different places, although the pictures are the same, only when (a) both A and B have the concept of cuboid and pentahedron in their consciousness, A uses the language of B to understand the name of the cuboid, B In order to understand which object is the cuboid, A, B No matter which of the two, no one has the concept of cuboid and pentahedron in the consciousness, communication with each other will become very difficult and inefficient, which means that the existing two people A and B The longer communication line with the scene is: by A-> object concept -> AB common language identification -> object concept -> B If there is a problem in any of the information exchange lines, the entire information exchange line will be interrupted.

 The objects contained in the real picture and the concepts required in it are much more than this example, random two people A, B The difference in cognitive levels is also a world apart. The most extreme is the very complex operating system in spacecraft and aircraft. The operating system consists of dozens to hundreds of subsystems. Astronauts and pilots often need support experts from various subsystems, astronauts and In order to achieve the same or close cognitive level with the support experts of each subsystem, pilots need a lot of long-term training, and the cost and time for establishing information exchange is very high.

The difference in product perception between the manufacturer and the user is inherently large. The most common in the industry is in more complex electromechanical products, such as a machine, which may be produced by machinery, electrical appliances, electronics, hydraulics, pneumatics, materials, The working medium and software are composed of many elements, each part of which must be recognized by the professional engineer, but the buyer user of the machine may be just a general operation, so the difference in cognitive level between the two sides is very large, The exchange of information between the manufacturer and the user is impossible to establish immediately. It takes a long time of on-site training and repeated operations to be gradually established, which also increases the cost and time for the correct use of the user.

Between the user and the manufacturer, especially between the new user and the manufacturer, the two parties have a very disparity in the description of the target, and the two parties are in a guiding and guided relationship. When the demander fails, because the product structure and parts are not familiar, Through the language and the supplier's instructors can not fully communicate, speak to themselves, do not know what to say, through the video does not know which object in the image or which space in the target, and finally have to send service personnel to On-site settlement, which caused additional costs for both parties.

The most common is in the sales activities of all products, especially in the popular online shopping, when e-commerce users look at their e-commerce products, they hope that their eyes and hands and feet can be locked at the e-commerce product site. It also needs the eyes of the scene to move around the locking product to obtain different viewing angles. It is also better to go deep into the internal observation of the object to achieve a complete and detailed observation of the target, so that the internal structure of the product can be understood without leaving the house, especially complex. The products, if the manufacturer can display all aspects of the product, including internal, according to the user's required perspective, will greatly facilitate transactions, reduce transaction costs, and eliminate mis-purchase caused by cognitive errors.

 Current video communication such as QQ And WeChat uses the field of view of the camera of the intelligent terminal to describe the specific target in the field of view with text. The premise is that both parties have the same and similar understanding and understanding of the described object, and there is no object to the target on both sides. Cognitive difference A-> Object Concept -> AB Common Language Logo -> Object Concept -> B This traditional cognitive approach.

Therefore, it is hoped that there will be no impact on any random encounters, and the products can be locked at different distances and at different distances according to their own requirements. The eyes of the scene can also obtain the locked target products around the locked target product relative motion. An image of the spatial direction.

 Summary of the invention

 The object of the invention is to provide An information exchange indication method in the same scene in the same place, which is used for timely and precise action indication of targets in the same scene between different places. In remote education, telemedicine, remote tour guides, remote command, off-site training, remote sales service and services, remote location exchanges require precise indications and have a wide range of uses.

A method for instructing information exchange in a different scene, comprising: an indication party located at a different place, an indicated party: acquiring a scene image seen by an eye of the indicated party; transmitting the acquired scene image to the indication party, wherein the indication party is in the scene Instructing the position indication icon in the image and returning it to the indicated party, the indicated indication icon includes at least the indication meaning of the action or the orientation; the instructed party acquires the scene image with the indication icon and indicates the action according to the indication icon. The instructed party returns the image feedback indicator according to the instruction indicator icon.

Preferably, the indicator party only includes information indicating that the icon and its coordinate position on the scene image are transmitted back to the indicated party, and the indicator party obtains the superimposed image of the indication icon and the scene image;

 Or the instructing party returns the indication image and the overall image of the scene image to the indicated party, and the instructed party obtains the overall image.

 Or, a method for indicating information exchange in the same scene, including:

 Obtaining a scene image within the field of view of the indicated party; the image herein includes pictures and images;

 Adding an indication icon for indicating an action to the acquired scene image;

 The indicating party selects a required indication icon according to the scene image with the indication icon;

 The instructed party acquires the scene image with the indication icon and performs the corresponding action according to the indication icon selected by the instructing party.

Preferably, the indicator adds an indication icon to the acquired scene image, and feeds the indication icon and its coordinate position on the scene image to the indicated party, and the indicated party will obtain the indication icon superimposed on the scene image according to its coordinate position. Obtaining a scene image with an indication icon;

 Or, the instructing party adds an indication icon to the acquired scene image, and feeds back the scene image with the indication icon to the indicated party.

 The above indication icons are two-dimensional or three-dimensional, and the above icons are static or dynamic.

 Another object of the present invention is to provide an information exchange indication system in the same scene.

 In order to achieve the above object, the technical solution adopted by the present invention is:

 An information exchange indication system in the same scene in the same place, comprising:

 Camera component: used to obtain the image of the scene seen by the indicated party's eyes;

 Instructing party intelligent terminal: configured to display to the indicating party and mark the scene image with the indication icon;

 Instructed party intelligent display component: for displaying to the indicating party the image of the calibration scene with the indicated icon,

 The imaging component, the pointing smart terminal, and the instructed smart display component are connected to each other.

 Or, an information exchange indication system in the same scene, including:

 An image acquisition component, configured to collect a scene image in the field of view of the indicated party;

a pointing party intelligent terminal, configured to receive the scene image and add an indication icon indicating an action on the scene image, where the indicator smart terminal is further configured to display, to the indicator, a scene image collected by the image capturing component and / Or a scene image with an indicator icon;

 The instructed party intelligent display component is configured to receive and display a scene image with an indication icon.

 Or, an information exchange indication system in the same scene, including:

 An image acquisition component, configured to collect a scene image in the field of view of the indicated party;

a pointing party intelligent terminal, configured to receive the scene image and add an indication icon indicating an action on the scene image, where the indicator smart terminal is further configured to display, to the indicator, a scene image collected by the image capturing component and / Or a scene image with an indicator icon;

The instructed party intelligent display component is configured to receive a scene image collected by the image acquisition component, the indication icon, and coordinates of the indication icon on the scene image, and superimpose the indication icon according to the coordinate position thereof on the scene image. A scene image with an indication icon showing a scene image with an indication icon.

Preferably, the indicating intelligent terminal is configured to return information indicating only the indication icon and its coordinate position on the scene image to the indicated intelligent display component;

 Or returning the entire indication image of the indication indicator and the scene image to the indicated intelligent display component.

 Further preferably, the indicated party intelligent display component comprises:

Instructed party intelligent terminal: for converting information including only the indication icon and its coordinate position on the scene image into the first imaging source or converting the entire image of the annotation indication icon and the scene image into the second imaging source;

 Scene image display component: for displaying the scene image and the superposition of the first imaging source to the indicator or displaying the second imaging source to the indicator.

Further preferably, the scene image display component is a helmet display or a glasses display or a combination of a half mirror and a projector.

Further preferably, the helmet display comprises an aperture stop, a relay system, an imaging mirror and a synthetic mirror, the first imaging source or the second imaging source sequentially passes through the aperture stop, the relay system, and the imaging The mirror is finally displayed on the synthetic mirror.

 Further preferably, the helmet display is an off-axis system or a coaxial system.

Further preferably, when the indicated party intelligent terminal converts the entire indication image of the annotation indication icon and the scene image into the second imaging source, the scene image display component displays the display of the second imaging source to the indication party. Screen.

Further preferably, the image capturing component is disposed at a front portion of the scene image display component for capturing an image displayed by the scene image display component.

Further preferably, the front side of the indicated intelligent display unit is provided with a mirror, and the image capturing unit reflects the image displayed by the scene image display unit through the mirror reflection.

 Preferably, the indicated intelligent display component and the imaging component are integrated into an integrated device.

 Further preferably, the integral device is provided by a spectacle frame, an eye cap or a helmet mount.

 Further preferably, the integral device is provided with a concave mirror.

 The indicated party intelligent display component includes:

 A combination of a head-mounted display or a glasses display or a half mirror and a projector.

 Due to the application of the above technical solutions, the present invention has the following advantages and effects compared with the prior art:

The invention realizes that two people simultaneously see the same scene and the action indication icon by the system of two people in different places, and enables the scene person to realize the cognition or action according to the instruction of the instruction icon, thereby eliminating the recognition of the target object by both parties. Knowing the differences in communication barriers, it can achieve long-distance acquisition of synchronized field of view among a large number of people, and improve the efficiency, accuracy and time synchronization of the communication through the indicator icons.

 DRAWINGS

 Figure 1 is a schematic view of the first embodiment;

 Figure 2 is a schematic view of a head mounted display;

 Figure 3 is a schematic view of the second embodiment;

 Figure 4 is a schematic view of the fourth embodiment;

 Figure 5 is a schematic view of the sixth embodiment.

 Among them: 10, the intelligent terminal to be instructed; 11, helmet display; 11a, aperture diaphragm; 11b , relay system; 11c, imaging mirror; 11d, synthetic mirror; 2, indicating intelligent terminal; 3, camera; 4, mirror; 5, display; 6, semi-transparent mirror; ,projector.

 detailed description

 The invention is further described below in conjunction with the drawings and the embodiments.

 Embodiment 1

 The method for indicating information exchange in the same scene in the same place of the present invention includes:

 Obtaining scene images (images and/or images) within the field of view of the indicated party;

 Adding an indication icon for indicating an action to the acquired scene image;

 The indicating party selects a required indication icon according to the scene image with the indication icon;

The instructed party acquires the scene image with the indication icon and performs the corresponding action according to the indication icon selected by the instructing party. The indicator icon is two-dimensional or three-dimensional, indicating that the icon is static or dynamic.

Specifically, in this embodiment, the instructed party acquires a scene image in the field of view (as seen by the instructed eye), and transmits the acquired scene image to the instructing party; the instructing party acquires the scene image and fills the scene image thereon. An indication icon indicating an action (such as an arrow indicating a moving direction, a gesture of stopping, etc.), selecting a required indication icon according to the scene image with the indication icon and notifying the indicated party; the instructed party acquires the scene with the indication icon The image is executed in accordance with the indication icon selected by the instructing party (eg, moving, stopping, etc. according to the corresponding orientation). The above-mentioned indicator and the indicated party include the smart terminal, the camera, and the like used by the indicator and the instructed person.

 As shown in Figure 1: In order to reach the indirect location A, the indicated party B, two people see the same scene at the same time, in the indicator A The pointing party smart terminal 2 is set, and the indicated party smart display component is set at the indicated party B. In this embodiment, the indicated party smart display component includes the indicated party smart terminal 10 and the head mounted display 11 (or glasses display), wherein: the instructed party smart terminal 10, the instructing party smart terminal 2 may be a smart phone or a computer, and the instructed party B wears a head mounted display 11 , set the camera 3 near the eyes.

 As shown in Figure 2, the off-axis optical system of the helmet display 11 includes an aperture stop 11a and a relay system 11b. The imaging mirror 11c and the synthesizing mirror 11d, the synthesizing mirror 11d is a semi-transparent mirror, and the imaging source C sequentially passes through the aperture stop 11a, the relay system 11b, and the imaging mirror 11c. Finally, it is displayed on the synthesizing mirror 11d that the human eye can see the image of the imaging source C through the synthesizing mirror 11d, or the external scene can be seen through the synthesizing mirror 11d. Helmet display 11 An off-axis optical system or a coaxial optical system can be used.

 Its working principle is as follows:

 Camera 3 will directly indicate the outside scene B that the eye needs to see D Shooting and capturing the scene image or picture, the relative position of the eye and the camera 3 is poor, the position difference can be corrected by software; the indicated party B can see the outside scene through the synthetic mirror 11d. The scene image network captured by the camera 3 is sent to the pointing party intelligent terminal 2, and the indicator party A passes the pointing party smart terminal 2 On the mouse, buttons (touch screen and voice-activated voice input commands), find the location of the target on the scene image, and add the indicator icon that indicates the meaning of the action or orientation.

The indication icon may include, but is not limited to, an arrow indicating upper, lower, left, and right, a cross indicating a target target (or a target icon such as a finger), a concentric circle indicating a change from large to small, and a backward indicating Concentric circles of small to large changes, indicating the forward and counterclockwise rotation arrows of the left and right rotations, indicating the stop gesture, the panning hand indicating the removal, the flipping hand indicating the open, etc., or Dynamic 2D icons, static 3D icons or dynamic 3D icons.

 After the indication icon is added, the coordinate position and type of the indication icon in the scene image are recorded on the pointing party smart terminal 2 And being sent to the instructed party smart terminal 10 to form a first imaging source C1 containing only the indication icon and its coordinate position on the scene image, projected onto the synthesizing mirror 11d, and the instructed party B passes through the synthesizing mirror 11d When the external scene D and the superimposed image of the first imaging source C1 are seen, the instructed party B completes the action around the target required by the instructing party A by indicating the indication of the icon, and the instructed party B follows the indicating party A. Indicates the image feedback indicator A after the icon is executed.

 Thus: what is finally seen in the eye of the indicated party B is that the coordinate position of the indicator icon is the same as the indicator A. The same scene image in the eye is exactly the same, and the indicated party B can be the presence of the indicator A at the indicated party B under the precise indication of the indicator A, so that the indicator party A obtains the indicated party B. The same field of view for each spatial location of the desired target obtained on site.

 Embodiment 2:

 As shown in FIG. 3, this embodiment is basically the same as the first embodiment except that:

 Helmet display 11 There is a mirror 4 at the front, and the camera 3 reflects the helmet display through the mirror 4 The image displayed by the synthetic mirror 11d.

 Embodiment 3:

 This embodiment is basically the same as the first embodiment or the second embodiment, and the difference is:

 After the instruction icon is added, the indication icon and the scene image are synthesized by the indicator smart terminal 2 and transmitted to the indicated intelligent terminal. 10, the whole image forming the indication icon and the scene image is converted into the second imaging source C2, projected onto the synthetic mirror 11d, and the indicated side B directly sees the second imaging source C2 through the synthetic mirror 11d. .

 Embodiment 4:

 As shown in FIG. 4, this embodiment is basically the same as the third embodiment, and the difference is:

 In this embodiment, the helmet display (or the glasses display) is not used, and the liquid crystal display is used instead. The overall image forming the indication icon and the scene image is converted into the second imaging source and displayed directly through the liquid crystal display 5, and the indicated side B sees through the liquid crystal display 5 The scene image displayed, no real outside scene.

 Embodiment 5:

 This embodiment is basically the same as the fourth embodiment except that:

 LCD screen 5 Integral display component and camera unit are integrated into one device: LCD monitor 15 Directly to the display of the indicated intelligent terminal 10, the camera 3 is directly the camera of the indicated intelligent terminal 10, such as a smart phone or a computer.

The integrated device can be placed through a spectacle frame, an eye cap or a helmet mount, or it can be a spectacle frame, an eye cap or a helmet mount that can hold a smart phone. In order to adjust the focal length, a concave mirror is arranged in front of the integrated device.

 Example 6:

 As shown in FIG. 5, this embodiment is basically the same as Embodiment 1 or Embodiment 2, and the difference is:

 In this embodiment, the helmet display (or the glasses display) is not used, and the half mirror 6 and the projector which are equivalent to the function of the helmet display are used instead. The combination.

If the instructing party is a user and the instructed party is an e-commerce provider, the instructing party can use the system on the Internet to allow the e-commerce to display the part of the product and the internal structure that it wants to see, and to achieve true online shopping, not just photos. And the video content that you don't want to see and the unreasonable language communication barriers, which have great commercial value for the dynamic display, sales and service of complex products on the Internet, can save a lot of personnel travel expenses, and can solve the after-sales service problem in time.

By adopting the invention, when astronauts and pilots switch between the support experts of different subsystems in time, the astronauts and pilots can timely solve the anomalies with various experts on the ground, which can save the training cost of astronauts and pilots, and improve the aerospace. Members and pilots have the ability to stand in the air.

The above embodiments are merely illustrative of the technical concept and the features of the present invention, and the purpose of the present invention is to enable those skilled in the art to understand the present invention and to implement the present invention, and the scope of the present invention is not limited thereto. Equivalent variations or modifications made in accordance with the spirit of the invention are intended to be included within the scope of the invention.

Claims (15)

1. The method for indicating information exchange in the same scene includes: displaying the image of the scene that is displayed by the indicated party's eyes; and transmitting the acquired scene image to the indicator, indicating The party indicates a position filling indication icon in the scene image and returns the indication to the indicated party, and the indicated indication icon includes at least the indication meaning of the action or the orientation; the instructed party acquires the scene image with the indication icon and presses the indication The icon indicates the action, and the instructed party returns the image feedback indicator according to the instruction indicator icon.
2. The method according to claim 1, wherein the indicating party returns only the indication icon and the information of the coordinate position on the scene image to the indicated party, and the instructed party obtains the indication icon. And superimposed images of scene images;

   Or the instructing party returns the indication image and the overall image of the scene image to the indicated party, and the instructed party obtains the overall image.
3. A method for indicating information exchange in a scene in a different place, characterized in that it comprises:

   Obtaining a scene image within the field of view of the indicated party;

   Adding an indication icon for indicating an action to the acquired scene image;

   The indicating party selects a required indication icon according to the scene image with the indication icon;

   The instructed party acquires the scene image with the indication icon and performs the corresponding action according to the indication icon selected by the instructing party.
4. The method according to claim 3, wherein the indicator adds an indication icon to the acquired scene image, and feeds the indication icon and its coordinate position on the scene image to the indicated party. , the instructed party will obtain an indication icon superimposed on the scene image according to its coordinate position to obtain a scene image with an indication icon;

   Or, the instructing party adds an indication icon to the acquired scene image, and feeds back the scene image with the indication icon to the indicated party.
5. An information exchange indication system in the same scene in the same place, comprising:

   a camera component for acquiring an image of a scene seen by an instructed eye;

   a pointing party intelligent terminal, configured to display to the indicating party and mark the scene image with the indication icon;

   The indicated party intelligent display component is configured to display the image of the calibration scene with the indication icon to the indicated party,

   The imaging component, the pointing smart terminal, and the instructed smart display component are connected to each other.
6. An information exchange indication system in the same scene in the same place, comprising:

   An image acquisition component, configured to collect a scene image in the field of view of the indicated party;

   a pointing party intelligent terminal, configured to receive the scene image and add an indication icon indicating an action on the scene image, where the indicator smart terminal is further configured to display, to the indicator, a scene image collected by the image capturing component and / or a scene image with an indicator icon;

   The instructed party intelligent display component is configured to receive and display a scene image with an indication icon.
7. An information exchange indication system in the same scene in the same place, comprising:

   An image acquisition component, configured to collect a scene image in the field of view of the indicated party;

   a pointing party intelligent terminal, configured to receive the scene image and add an indication icon indicating an action on the scene image, where the indicator smart terminal is further configured to display, to the indicator, a scene image collected by the image capturing component and / or a scene image with an indicator icon;

   The instructed party intelligent display component is configured to receive a scene image collected by the image acquisition component, the indication icon, and coordinates of the indication icon on the scene image, and superimpose the indication icon according to the coordinate position thereof on the scene image. A scene image with an indication icon showing a scene image with an indication icon.
8. The system according to claim 5, wherein said indicating intelligent terminal is configured to transmit back information indicating only the indication icon and its coordinate position on the scene image to said indicated intelligent display component;

   Or returning the entire indication image of the indication indicator and the scene image to the indicated intelligent display component.
9. The system of claim 8 wherein said indicated party intelligent display component comprises:

   Instructed party intelligent terminal: for converting information including only the indication icon and its coordinate position on the scene image into the first imaging source or converting the entire image of the annotation indication icon and the scene image into the second imaging source;

   Scene image display component: for displaying the scene image and the superposition of the first imaging source to the indicator or displaying the second imaging source to the indicator.
10. The system of claim 9 wherein said scene image display component is a head mounted display or a glasses display or a combination of a half mirror and a projector.
11. The system of claim 10 wherein said head mounted display comprises an aperture stop, a relay system, an imaging mirror, and a composite mirror, said first imaging source or second imaging source sequentially passing said The aperture stop, the relay system, the imaging mirror, and finally the display on the synthetic mirror.
12. The system according to claim 9, wherein when the indicated intelligent terminal converts the image of the indication indicator and the entire image of the scene image into a second imaging source, the scene image display component is The indicator displays the display of the second imaging source.
13. The system according to any one of claims 5 to 7, wherein the image pickup unit is disposed at a front portion of the scene image display unit for capturing an image displayed by the scene image display unit.
14. The system according to any one of claims 5 to 7, wherein the instructed party intelligent display component and the imaging component are integrated into an integrated device.
15. The system according to claim 6 or 7, wherein said indicated intelligent display component comprises:

    A combination of a head-mounted display or a glasses display or a half mirror and a projector.