WO2016070827A1

WO2016070827A1 - Method and apparatus for issuing and transmitting recognition information, and information recognition system

Info

Publication number: WO2016070827A1
Application number: PCT/CN2015/093896
Authority: WO
Inventors: 刘海军; 罗圣美
Original assignee: 中兴通讯股份有限公司
Priority date: 2014-11-07
Filing date: 2015-11-05
Publication date: 2016-05-12
Also published as: CN105630142A

Abstract

Disclosed are a method and apparatus for issuing and receiving recognition information, and an information recognition system. The method comprises: issuing, by a first device, recognition information, wherein the recognition information at least comprises: a coordinate of an object to be recognized in a first coordinate system and information about the first coordinate system.

Description

Method and device for distributing and transmitting identification information and information recognition system

Technical field

Embodiments of the present invention relate to, but are not limited to, computer vision technology, and more particularly to a method and apparatus for distributing and transmitting identification information and an information recognition system.

Background technique

The development of digital multimedia and networks has enriched the entertainment experience in people's daily lives. The current technology allows people to watch HDTV at home. The source of TV programs may come from digital discs, cable TV, the Internet, etc., and can experience stereo, 5.1-channel, 7.1-channel and even more realistic sound effects, and people can Using a tablet (PAD), mobile terminal to achieve these experiences, people can also transfer digital content between different devices over the network, and control the playback of a device through remote control, gestures, for example, using gesture control to switch One channel, the next channel, and so on.

The traditional method of controlling multiple devices is to control the devices by using the respective remote controllers of the devices, and these remote controllers are often not universal, and most of the remote controllers do not have network functions, such as traditional televisions and stereos. There are also some remote controllers that support the network, such as loading software that supports interworking protocols on devices with computing and network capabilities (such as mobile terminals, PADs, etc.) to control another device.

With the development of technology, there are more and more content sharing and transfer requirements between multiple devices. The above control method is not convenient enough. For example, people need to select a remote controller corresponding to the device in a pile of remote controllers, and As the device changes, it is necessary to constantly change the remote control, or to control a single device with simple gestures. Users often need to learn how to use different remote controls when controlling different devices. Users may prefer to use a simpler, more natural way of operating to control more devices in a smaller range.

Gesture control is a relatively novel control method, that is, the camera on one device monitors the gesture and analyzes it and converts it into control of the device; or the user uses the wearable device through the hand and the arm. Wearing a ring, watch, vest, etc. on the body and on the body To identify user actions to achieve control of the device. Related technologies and some products enable users to manipulate gestures using gestures. For example, by adding a camera to the television, collecting and recognizing the user's gesture, and then following the correspondence between the predefined gesture and the manipulation command, thereby achieving the effect of manipulating the television through the gesture, the currently implemented manipulation includes changing the channel. , change the volume, etc.

The gesture recognition devices currently on the market and in the laboratory are each equipped with a camera, and it is often necessary to input a "boot indication" to the gesture recognition device, such as a specific gesture, or other control commands. When there are many gesture recognition devices in the home and office scenes, such as TV, set-top box, player, air conditioner, and audio in the home, gesture recognition is supported. When the user controls these devices at the same time, the "starting instruction" is often switched. This experience is not After the friendly, or "starting gesture", since the gesture recognition devices only process the recognized gesture commands, the gesture recognition devices are likely to cause confusion in the recognition instruction input.

At present, mutual discovery and control between devices can be realized. For example, the related technology Universal Plug and Play (UPnP) technology specifies how to send and receive network messages between devices to implement discovery and control. The technology uses network addresses and As the identifier of the device, digital coding is a kind of machine identification. The final control requires the user to select and operate according to the machine identification of the device.

Techniques already under investigation: a gesture recognition device capable of recognizing a device including its position, communication identification parameters, and issuing recognized gesture information; when using gesture manipulation, the controlled device issues a gesture that can be processed: gesture number, gesture text Describe information and digital graphic information.

Despite the above-mentioned devices and technologies, it is difficult to simultaneously recognize all gestures or devices in a space due to limitations of camera position (angle, height, etc.) and performance.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

Embodiments of the present invention provide a method and apparatus for distributing and transmitting identification information and an information recognition system capable of sharing identification information.

The embodiment of the invention provides a method for issuing identification information, including:

The first device issues identification information; the identification information includes at least: coordinates of the identified object in the first coordinate system, and first coordinate system information.

Optionally, the method further includes:

The first device captures an image, and the identified object is identified according to the captured image to obtain the identification information.

Optionally, wherein the coordinates of the identified object in the first coordinate system are coordinates of one or more points on the identified object in the first coordinate system.

Optionally, the identification information further includes one or more of the following information:

The distance between the identified object and the first device, the network address or communication port of the identified object, the identified object identifier, and the first device identifier.

Optionally, wherein the second device is the identification object, and the coordinates of the second device in the first coordinate system are coordinates of the identification object in the first coordinate system.

Optionally, wherein the first device issues the identification information in a manner of broadcasting, or multicasting, or unicasting, or responding to a query request.

The embodiment of the invention further provides a method for transmitting identification information, including:

The second device receives the identification information; the identification information includes at least: coordinates of the identified object in the first coordinate system, and first coordinate system information.

Optionally, the method further includes:

The second device calculates coordinates of the identified object in the second coordinate system according to the identification information.

Optionally, the calculating, by the second device, the coordinates of the identified object in the second coordinate system according to the identification information includes:

The second device calculates a conversion matrix according to coordinates of the second device in the first coordinate system and coordinates of the second device in the second coordinate system, and calculates the recognized according to the conversion matrix and the identification information. The coordinates of the object in the second coordinate system.

Optionally, wherein the identification information further includes one or more of the following information:

The coordinates of the one or more points on the identified object in the first coordinate system, the Identify the network address or communication port of the object, the identified object identifier, and the first device identifier.

Optionally, the method further includes:

The second device communicates with the identified object according to the calculated or received coordinates, network address or communication port of the identified object in the second coordinate system.

An embodiment of the present invention further provides an apparatus for issuing identification information, including at least:

The first network unit is configured to issue identification information; the identification information includes at least: coordinates of the identified object in the first coordinate system, and first coordinate system information.

Optionally, the device further includes:

The first visual recognition unit is configured to capture an image, and identify the identified object according to the captured image to obtain the identification information.

Optionally, the first network unit is further configured to:

The identification information is published in a broadcast, or multicast, or unicast, or in response to a query request.

An embodiment of the present invention further provides an apparatus for transmitting identification information, including at least:

The second network unit is configured to receive the identification information; the identification information includes at least: coordinates of the identified object in the first coordinate system, and the first coordinate system information.

Optionally, the device further includes:

The second visual recognition unit is configured to calculate coordinates of the identified object in the second coordinate system based on the identification information.

Optionally, the second visual recognition unit is configured to:

Calculating a transformation matrix according to coordinates of the second device in the first coordinate system and coordinates of the second device in the second coordinate system, and calculating the identified object in the second coordinate system according to the transformation matrix and the identification information coordinate of.

Optionally, the device further includes:

a storage module, configured to save one or more of the following: coordinates of one or more points on the identified object in a first coordinate system, network address or communication port of the identified object, Identify the object identifier, the first device identifier.

An embodiment of the present invention further provides an information recognition system, including at least:

a first device, configured to issue identification information; the identification information at least: coordinates of the identified object in the first coordinate system, first coordinate system information;

The second device is configured to receive the identification information.

Optionally, wherein the first device is further configured to:

The image is captured, and the identified object is identified based on the captured image to obtain the identification information.

Optionally, the first device is further configured to: publish the identification information in a manner of broadcasting, or multicasting, or unicasting, or responding to a query request.

Optionally, wherein the second device is further configured to:

Calculating coordinates of the identified object in the second coordinate system based on the identification information.

Optionally, wherein the second device is configured to:

Calculating a conversion matrix according to coordinates of the second device in the first coordinate system and coordinates of the second device in the second coordinate system, and calculating the identified object according to the conversion matrix and the identification information The coordinates in the two coordinate system.

Optionally, wherein the second device is further configured to:

Saving one or more of the following information: coordinates of one or more points on the identified object in the first coordinate system, network address or communication port of the identified object, identified object identifier, A device identifier.

Compared with the related art, the embodiment of the present invention includes: the first device issues identification information; the identification information includes at least: coordinates of the identified object in the first coordinate system, and first coordinate system information. With the solution of the embodiment of the present invention, the first device issues the identification information of the identified object, and shares the identification information of the identified object, so that the other device can identify the identified object by receiving the issued identification information, regardless of whether Identify if the object is within its capture range.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

BRIEF abstract

1 is a flowchart of a method for distributing identification information according to an embodiment of the present invention;

2 is a flowchart of a method for transmitting identification information according to an embodiment of the present invention;

3 is a schematic structural diagram of an apparatus for distributing identification information according to an embodiment of the present invention;

4 is a schematic structural diagram of an apparatus for transmitting identification information according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an information recognition system according to an embodiment of the present invention; FIG.

6 is a schematic diagram of a device scenario according to an embodiment of the present invention;

Figure 7 is a schematic diagram of a device unit according to an embodiment of the present invention;

FIG. 8 is a flowchart of a method according to an embodiment of the present invention; FIG.

FIG. 9 is a schematic diagram of message interaction according to an embodiment of the present invention.

Embodiments of the invention

Since the gesture recognition device in the related art does not publish and share the location information of the other device, the recognized location information of the other device, and the gestures that are recognized, the gesture instructions are processed by the gesture recognition device. Identify confusion about command input.

The embodiments of the present invention are described in detail below with reference to the accompanying drawings, and are not intended to limit the scope of the present invention. It should be noted that the embodiments in the present application and the various manners in the embodiments may be combined with each other without conflict.

Referring to FIG. 1, an embodiment of the present invention provides a method for distributing identification information, including:

Step 100: The first device issues the obtained identification information.

In this step, the first device may issue the identification information in a manner of broadcasting, or multicasting, or unicasting, or responding to the query request.

In this step, the identification information includes at least coordinates of the identified object in the first coordinate system and first coordinate system information. The first coordinate system information refers to coordinate description information of the first coordinate system, such as a Cartesian coordinate system or a polar coordinate system, a coordinate origin, a distance ratio (scale bar), a device located at or indicating the coordinate origin, and the like.

The identified object may be a second device, and the coordinates of the identified object in the first coordinate system may be coordinates of the second device in the first coordinate system.

The coordinate of the identified object in the first coordinate system may be the coordinate of the one or more points (such as the center point) on the identified object in the first coordinate system; the second device is in the first coordinate system. The coordinates may be coordinates of one or more points (eg, center points) on the second device in the first coordinate system.

The coordinates of the identified object or the second device in the first coordinate system can be obtained by using a depth camera.

The identification information may further include one or more of the following:

The coordinate origin of the first coordinate system may be the first device, or may be other points in the space.

The identified object identifier may be a visual recognition feature of the recognized object, and the visual recognition feature may be one or more of the following: a barcode label, a two-dimensional code label, a text label, a trademark logo, and the like.

The identified object may be one or more of a television, a player, a storage server, a computer, a stereo, a speaker, a projector, a set top box, a car, a machine tool, a ship, and the like.

In this step, the first device may send the identification information to the second device by means of broadcast or multicast. Among them, Universal Plug and Play (UPnP) technology, or Multicast Domain Name System (mDNS), or Domain Name System Based Service Discovery (DNS-SD) technology can be used to transmit identification information.

The network address or the communication port of the first device that obtains the identified object may be sent by the identified object to the first device.

In the method of the embodiment of the present invention, the first device issues the identification information of the identified object, and shares the identification information of the identified object, so that the other device can identify the identified object by receiving the posted identification information, regardless of whether Identify if the object is within its capture range.

The method also includes:

Step 101: The first device captures an image, and identifies the identified object according to the captured image to obtain the identification information.

The step includes: the first device identifying the identified object to obtain the identification information, or receiving the query request.

In this step, the first device identifying the identified object to obtain the identification information includes:

The first device collects the visual recognition feature of the identified object, and identifies the identified object according to the collected visual recognition feature to obtain the identification information.

The first device collects the visual recognition features of the identified object, including:

The first device may receive a broadcast or multicast message from the identified object; wherein the broadcast or multicast message includes a visually recognizable feature of the identified device; or

The first device sends a query broadcast or multicast message to the identified object and receives a response message from the identified object; wherein the response message includes a visually recognized feature of the identified device.

Wherein, identifying the object to be identified according to the collected visual identification feature comprises obtaining identification information, including:

The images or video segments within the visual range are acquired by the depth camera, and the collected visual recognition features are matched in the acquired image or video segment to obtain the coordinates of the identified object in the first coordinate system.

When the first device receives the query request from the second device, the method further includes:

Step 102: The first device sends the identification information of all the identified objects to the second device.

In this step, the first device may send the identification information to the second device by means of broadcast, multicast, or unicast.

Referring to FIG. 2, an embodiment of the present invention provides a method for receiving or transmitting identification information, including:

Step 200: The second device receives the identification information.

In this step, the identification information includes: coordinates of the identified object in the first coordinate system, and first coordinate system information. Optionally, the identification information may further include coordinates of the second device in the first coordinate system. In addition, optionally, the foregoing first coordinate system information may also be a default configuration to the second device. The coordinate origin of the first coordinate system may be the first device, or may be other points in the space.

In this step, the identification information may further include one or more of the following:

The coordinates of the identified object in one or more points (eg, one or more tangent planes, or one or more points on the contour) in the first coordinate system, the network of the identified object Address or communication port, identified object identifier, first device identifier.

The method also includes:

Step 201: The second device sends a query request.

The method also includes:

Step 202: The second device calculates coordinates of the identified object in the second coordinate system according to the identification information.

In this step, the second device calculates a conversion matrix according to the coordinates of the second device in the first coordinate system and the coordinates of the second device in the second coordinate system, according to the calculated conversion matrix and the identification information (such as being identified therein) The coordinates of the object in the first coordinate system) calculate the coordinates of the identified object in the second coordinate system.

Wherein, the transformation matrix is the product of the coordinate vector of the second device in the second coordinate system and the inverse vector of the coordinate vector of the second device in the first coordinate system.

The coordinates of the identified object in the second coordinate system are the product of the coordinates of the identified object in the first coordinate system and the transformation matrix.

Step 203: The second device saves the identified object identifier in the identification information and the calculated coordinates of the identified object in the second coordinate system.

In this step, the second device may also save one or more of the following:

The coordinates of the one or more points on the identified object (eg, one or more tangent planes, or one or more points on the contour) in the first coordinate system, the network address of the identified object, or the communication port.

The method also includes:

Step 204: The second device communicates with the identified object according to the calculated or received coordinates of the identified object in the second coordinate system, the network address of the identified object, or the communication port.

Referring to FIG. 3, an embodiment of the present invention further provides an apparatus for issuing identification information, including at least:

The first network unit 301 is configured to issue identification information; the identification information includes at least: coordinates of the identified object in the first coordinate system, and first coordinate system information.

The device of the embodiment of the present invention further includes:

The first visual recognition unit 302 is configured to capture an image, identify the identified object based on the captured image, and obtain identification information.

In the apparatus of the embodiment of the present invention, the first network unit 301 is further configured to:

Referring to FIG. 4, an embodiment of the present invention further provides an apparatus for receiving or transmitting identification information, including at least:

The second network unit 401 is configured to receive the identification information from the first device; the identification information includes at least: coordinates of the identified object in the first coordinate system, and first coordinate system information.

In the apparatus of the embodiment of the present invention, the second network unit 401 is further configured to: send a query request.

The device of the embodiment of the invention further includes:

The second visual recognition unit 402 is configured to calculate coordinates of the identified object in the second coordinate system based on the identification information.

In the apparatus of the embodiment of the present invention, the second visual recognition unit 402 is configured to:

Calculating a transformation matrix according to the coordinates of the second device in the first coordinate system and the coordinates of the second device in the second coordinate system, according to the calculated transformation matrix and the identification information (eg, the identified object therein is in the first coordinate system) Coordinates) Calculate the coordinates of the identified object in the second coordinate system.

The device of the embodiment of the present invention further includes:

The storage module 403 is configured to save one or more of the following information:

The coordinates of the one or more points on the tangent plane or the contour of the identified object in the first coordinate system, the network address or communication port of the identified object, the identified object identifier, the first device identification.

Referring to FIG. 5, an embodiment of the present invention further provides an information recognition system, including at least:

The first device 501 is configured to issue identification information; the identification information includes at least: coordinates and coordinate description information of the identified object in the first coordinate system;

The second device 502 is configured to receive the identification information.

In the information recognition system of the embodiment of the present invention, the first device 501 is further configured to:

The image is captured, and the identified object is identified based on the captured image to obtain identification information.

In the information recognition system of the embodiment of the present invention, the first device 501 is further configured to: issue the identification information in a manner of broadcasting, or multicasting, or unicasting, or responding to the query request.

In the information recognition system of the embodiment of the present invention, the second device 502 is further configured to:

The coordinates of the identified object in the second coordinate system are calculated based on the identification information.

Calculating a transformation matrix according to the coordinates of the second device 502 in the first coordinate system and the coordinates of the second device 502 in the second coordinate system in the identification information, according to the calculated transformation matrix and the identification information (such as the identified object therein) The coordinates in the first coordinate system are used to calculate the coordinates of the identified object in the second coordinate system.

In the information recognition system of the embodiment of the present invention, the second device 502 is further configured to: save one or more of the following information:

For the sake of clarity, not all of the conventional features of the device are shown and described in the embodiments of the invention. Of course, it should be understood that in the development of any actual device, it is necessary to make a decision on a particular implementation in order to achieve the developer's specific goals, such as compliance with application and business-related constraints, which are achieved with different implementations. Change and change with different developers. Moreover, it should be understood that such development work is complicated and time consuming, but nevertheless, the technical work performed by one of ordinary skill in the art in light of the disclosure of the embodiments of the present invention is conventional.

In accordance with the subject matter described herein, various components, systems, apparatuses, processing steps and/or data structures can be manufactured, manipulated, and/or executed using various types of operating systems, computing platforms, computer programs, and/or general purpose machines. Moreover, those of ordinary skill in the art will appreciate that less versatile devices may also be utilized. Wherein, the methods included are performed by a computer, device or machine, and the method can be stored as machine readable instructions, which can be stored on a defined medium, such as a computer storage device, including but not limited to ROM (read only memory) Reservoir, FLASH memory, transfer device, etc.) Magnetic storage media (eg, magnetic tape, magnetic disk drives, etc.), optical storage media (eg, CD-ROM, DVD-ROM, paper cards, paper tape, etc.) and other well known types of program memory.

FIG. 6 is a schematic diagram of a device scenario of an embodiment of the present invention, illustrating each device and its relationship.

In Figure 6, there are three devices, two gesture and device identification devices (A, B) and one TV TV. The gesture and device identification device A is the second device according to the embodiment of the present invention. The gesture and device identification device B is also the first device according to the embodiment of the present invention. The TV set TV is also the embodiment of the present invention. Said the identified object. Gesture and device recognition device, also known as (computer) visual recognition device, implements computer vision and image processing technology. In this embodiment, gesture and device recognition are equipped with a camera capable of recognizing gestures, recognition and discovery devices; TV TV Gesture control is supported, but no camera is used to recognize gestures. This is for convenience only. In fact, the gesture and device identification device A can be assembled with the TV TV to become a composite function device.

In a spatial position, the television TV is located within the visual range of the gesture and device identification device B, ie the gesture and equipment recognition device B is able to capture an image of the television TV. Another gesture and the visual range of the device identification device A cannot capture the TV set. The gesture and device recognition devices A and B are located within the visual range of the other party and can capture each other's images.

There are network interfaces on all three devices, such as IEEE 802.11b/g/n or IEEE 802.3, so that you can connect to an IP network. Each device contains a control module that can control sending and receiving messages with other devices, as well as processing control commands or handing over control commands.

The ability of such devices (ie devices with network interfaces) to discover, connect, send and receive messages on the network can be implemented using Universal Plug and Play (UPnP) technology, or by using the Multicast Domain Name System (mDNS) and Based on the Domain Name System-based Service Discovery (DNS-SD) technology, this type of technology is used in IP networks to respond to queries and provide function calls in a unicast or multicast query format according to a predefined message format. For example, UPnP technology specifies how media display devices (such as TV), servers (such as DVD players, home storage servers) respond to queries, and which calling functions are provided.

The gesture recognition control devices A, B include a camera with image and video capture capabilities, and an infrared ranging module. As an embodiment, the gesture recognition control device A or B internal package The image recognition unit, the data storage unit, the control unit, and the network service unit are included.

In this embodiment, all the gesture recognition control devices can identify the devices in the visual range thereof. The implementation method is that the camera and the ranging module rotate and collect images in the three-dimensional space, and analyze and search for the predefined label graphics in the collected images. The analysis is performed. Here, the predefined tags may be pre-stored in the image and graphics in the visual recognition and discovery control device, or may be received images and graphics transmitted from the discovered and recognized devices. After the image recognition unit of the gesture recognition control device analyzes the image, it recognizes that there are multiple identifiers in the image, identifies the identifiers separately and stores the identified information, and then performs distance measurement on the device where the label is located, and stores the identifier information together with the identification information. .

In another embodiment, the visual recognition and discovery control device receives feature images from devices such as a TV TV, a DVD player, and a home storage server, such as device photo data of a plurality of angles, and then captures device photo data from the device. The panoramic image is retrieved to measure and record the location of the device. This embodiment may use techniques such as feature detection and matching in computer vision technology.

In this embodiment, when the TV TV is powered on, the message is sent in a multicast manner, and the message includes:

The network address and unique identifier of the device (ie TV TV);

Supported gesture numbers, such as 1 for 5 fingers open, 2 for two fingers, 10 for fists, 20 for shaking arms, etc.

The meaning of the control corresponding to the gesture, such as 1 means boot, 10 means boot, and so on;

As another embodiment, the three-dimensional coordinates of the two gesture and device identification devices (A, B) and a television TV are expressed as:

In the A coordinate system: gesture and device identification device A = A (0, 0, 0), gesture and device identification device B = B (b1, b2, b3). Of course, you can also select another space as the coordinate origin (0,0,0).

In the B coordinate system: gesture and device identification device B = B (0, 0, 0), gesture and device identification device A = A (a1, a2, a3), TV TV = T (t1, t2, t3) . Of course, you can also select another space as the coordinate origin (0,0,0).

The coordinate point here may be the center point of the device plane, or may be specified as some other point on the device, such as a tangent plane, on the contour.

Figure 7 is a schematic diagram of a device unit in accordance with an embodiment of the present invention.

In this embodiment, the visual recognition device A (ie, the second device), the visual recognition device B (ie, the first device), and the receiving device T (ie, the identified object) are discovered and recognized devices, and the visual recognition device B can Discover and identify device T.

The device T is composed of a network unit, a visual identifier, and a visual recognition response unit.

The function of the network unit, as described in the embodiment of FIG. 6, includes a network interface, mutual discovery and connection capabilities on the network, and the ability to send and receive messages with other devices.

The visual logo is used to visually represent the device, at least the logo attached to the exterior or the photo, picture, trademark logo, video clip representation of the device itself. These photos or graphic images or video clips can be sent as device T to the data in the device B message.

The visual recognition response unit is an internal coordination control unit, including receiving a message of the network unit for analysis, instructing the visual response unit to respond according to the message request, adding visual identification data, visual feature description data, and requesting according to the message in the response message. The network element is instructed to respond to the request message; here the visual feature description data may be a visual identification related signal characteristic including a regular description. As an embodiment, the network message may adopt the UPnP protocol, and fill in the data in the specified field and the extended field according to the protocol.

The visual recognition devices A and B, that is, the visual gesture and device identification device, are composed of a visual data acquisition unit, a storage unit, a network unit, and a visual recognition unit.

The function of the network unit includes a network interface, which supports mutual discovery and connection on the network, and can send and receive messages with other devices.

The visual data acquisition unit of the visual recognition device A, B and the visual identification unit of the device T are corresponding units, that is, the visual identification unit of T provides visual features such as images, emits visual/optical signals, and devices A and B. The vision unit is responsible for identifying the visual features provided by the visual identification unit and matching the recognition according to the predefined features to determine the recognition result.

The visual recognition unit is an internal computing unit that performs visually related calculations, including image recognition, spatial coordinate determination, and object space position coordinate calculations, and the like.

The storage unit stores the identified device information, including device identification such as device T, network address, spatial location, visual feature data, etc., wherein the spatial location may be device T relative to device B. The geometric parameters of the origin of the space coordinate, such as three-dimensional space coordinates, or vector coordinates. These stored data can be used by other units within the device or for other devices. As one of the embodiments, the storage unit may provide the location data of the device T for use by the device A.

In general, the device B issues identification information including at least coordinate system information and coordinate parameters of the identified object such as the device T. The coordinate parameter of the identified object may be the coordinate in the coordinate system described by the identified object in the coordinate system information.

The device B identifies the identified object T by capturing an image, and determines a coordinate system; where the coordinate system can be a point in the image. The identified information about the device T may also include an object contour (such as three points, four points, five points, or other multi-point coordinates), or distance information (such as the identified device to the center point coordinates or to the coordinate origin) The distance) is the coordinates of the device B itself. Other information includes the identification object, that is, the identifier of the device T, or the network address, or the communication port. Here, the identified object is the device T, and the identification information includes its coordinates.

Device B issues identification information, which can be broadcast, multicast, unicast, and query-response.

As the device A, the information of the device T cannot be captured because of the location, but the identification information issued by the device T can be received from the device B. The received information includes the coordinate system information and the coordinate parameter of the identified object T, the identifier of the identified object, Or network address, or communication port.

As in the previous embodiment, device B is issued using the B coordinate system: gesture and device identification device B = B (0, 0, 0), television TV = T (t1, t2, t3). Device A measures the gesture and device identification device B=B (b1, b2, b3) in the A coordinate system and receives the coordinates of T in the B coordinate system. Then you can know the coordinates of T in the A coordinate system through the coordinate transformation matrix:

Let XA=MYB, where X is any point in the A coordinate system, Y is any point in the B coordinate system, and M is a three-dimensional transformation matrix.

(b1, b2, b3) = M(a1, a2, a3), so M = (b1, b2, b3) × (a1, a2, a3)-1

So the position of the TV in the A coordinate system is:

TA=MT

Thus, although the gesture and device recognition apparatus A cannot capture the TV, it can determine the spatial relationship between the gesture and the TA by combining the known TA based on the captured gesture position information.

Using the received identification information, the device A can continue (computer vision) recognition, communicate with the identified object T according to the recognized gesture result, the coordinates of the identified object T, the network address or the communication port, ie, send to the identified object T Gesture recognition result information, or control commands, and the like.

Figure 8 is a flow chart of a method in accordance with an embodiment of the present invention.

The flowcharts described in this embodiment refer to the devices A, B in the embodiment of Fig. 6, and A and B in the embodiment of Fig. 7. This embodiment describes a flow chart of the operation of the apparatus. Here, the visual recognition apparatus 1, 2 can refer to A, B in Figs.

Step 800, the visual recognition device 1 collects data, identifies the device and its location;

The device 1 captures the image in the visual range through the camera, and identifies the device in the image, and finally obtains information such as the device position and network parameters. The collection, identification, calculation and detailed information of the information can refer to all the previous embodiments.

Step 801: The visual recognition device issues a data update notification.

The device 1 issues an update notification about the identified device by means of broadcast, or multicast, or unicast, or in response to a query request.

Step 802, the visual recognition device 2 issues a data query notification;

Or similar to step 801, device 2 actively issues a query request. This step has no relationship with step 801, and only one of them can be used.

Step 803: The visual recognition device 1 responds to the visual recognition device 2, and the response message includes the identified device and its location information.

The visual recognition device 1 transmits the related information of the identified device to the visual recognition device 2, including: a coordinate system, a coordinate parameter of the identified object such as the device T, and a coordinate of the device 2 in the coordinate system, and an object contour of the recognized object ( Such as three points, four points, five points or other multi-point coordinates), or distance information (such as the distance between the identified device to the center point coordinates or the origin of the coordinates), other information includes the identification object, that is, the identification of the device T, Or network address, or communication port.

Step 804, the visual recognition device 2 interacts with the identified device;

Although the identified device T is not within the vision of the visual recognition device 2, there is a visual recognition device The information provided by the device 1 is such that when the user makes a gesture within the visual range of the visual recognition device 2, the visual recognition device 2 can determine the device that the user gesture wants to manipulate, and can send the recognized gesture information or command to The device T is identified. At this time, the user's gesture does not affect the user's operation even if it is not within the visual range of the visual recognition device 1. It is even possible to install the visual recognition device 1 exclusively for recognizing a target device, and the visual recognition device 2 is used for recognizing a gesture and controlling a target device.

In this flowchart, the visual recognition device 2 receives a message related to the target recognized device issued by the visual recognition device 1. The processing flow for this function is:

Step 900, the visual recognition device 1 identifies the target device;

The implementation of this message is illustrated in the embodiment illustrated in Figures 1, 2, 6, and 8. In this step, the identification of the target device by the visual recognition device includes capturing and recognizing the spatial position information of the identified device such as coordinate data and equipment by capturing the image or video segment of the identified device by using a depth camera or a plurality of cameras. The distance data between the two also includes information such as the network address of the identified device, the communication port, the identified device (unique) identifier, and the service capability through the network message.

After this step is completed, the visual recognition device 1 can also perform the following behavior (i.e., the ACTION step shown in Fig. 9): saving the identified information, associating the identification information with each device, and the like.

Step 901: The visual recognition device 1 issues device data of the identified device.

The implementation of this message is illustrated in the embodiment illustrated in Figures 1, 2, 6, and 8. The distribution here also uses network messages, including in broadcast, or multicast, or unicast, where unicast can be in a subscription-publishing manner, for example, the visual recognition device 1 accepts subscription device data of the visual recognition device 2 in advance. The visual recognition device 2 of the update request sends a message reporting the device data of the identified device.

Step 902: The visual recognition device 2 sends a query device data request to the visual identification device 1;

This step can be used in conjunction with the second step, and the order does not require a sequential relationship. The visual recognition device 2 may also use a broadcast, or multicast, or unicast method to send a query request. The figure illustrates the scenario of a peer-to-peer unicast query. After the visual recognition device 1 receives the request, it can also be triggered again. The identification function re-identifies and calculates the identified device, and can also send the already recognized device information that has been saved.

Step 903, the visual recognition device 1 transmits the device data to the visual recognition device 2 as a response to the third step message;

This step is also performed by a network message, which may be any of the information indicated in step 900 or a combination of information therein, or all of the information.

Step 904: The visual recognition device 2 sends a device control command to the identified device.

When the user makes a gesture within the visual range of the visual recognition device 2, the visual recognition device 2 can determine the device that the user gesture wants to manipulate, and then transmit the recognized gesture information or command to the identified device T. The visual recognition device 2 can determine the device that the user gesture wants to control, except that the user gesture is collected and analyzed and recognized, and because the visual recognition device 2 receives the spatial position information of the identified device, thereby being able to determine the user gesture and The positional relationship between the identified devices, and the intention of the user to control.

Step 905: The identified device sends a device execution result response to the visual recognition device 2.

This step is an optional step, and when the target device completes the instruction of the device 2, a response message can be sent. The response message may include a successful control, a failed control, an unrecognized manipulation command, an unsupported manipulation mode, a manipulation result data, and the like.

The above steps 900-903 and 904-905 can be run separately as two separate programs. For example, after the device is recognized, if the user does not perform gesture control or the visual recognition device does not collect the gesture image or video clip, it is equivalent to only Steps 900-903 occur and steps 904-905 do not occur.

In the above embodiment, the television, the player, and the storage server are used as the controlled device, and the embodiment of the present invention is not limited to the device to be controlled, such as the device mentioned in the embodiment, the computer, the audio, the speaker, and the projector. , set-top boxes, etc. can be used as controlled devices, and even other industrial equipment such as automobiles, machine tools, ships, etc. can be manipulated by visual recognition and discovery control devices.

In the above embodiment, the camera of the visual recognition and discovery control device may be of various specifications, for example, may be a fixed focal length or a zoom distance, and the rotating space may be a plurality of angles up, down, left, and right, or only support left and right angles, and only need to be configured. A camera has the various capabilities described in the embodiments. The distance measuring unit can use laser infrared ranging, or can use other bands of light measurement, you can use 3 The camera calculates the ranging, and more cameras can be used to calculate the ranging using weighting adjustments.

The foregoing process is only an embodiment of the present invention, and is not limited to the embodiment, and the method for executing the process is not limited. The embodiment of the present invention may also be implemented in a similar manner, such as indicating the name of the unit and the type of the message. Wait, it's just that the naming form, the message content, etc. are different.

The above embodiments are network-related and can be applied to communication networks based on IEEE 802.3, IEEE 802.11b/g/n, POWELINE, CABLE, PSTN, 3GPP (3rd Generation Partnership Project), 3GPP2 networks, and the like. Supported IP network, each device's operating system can be applied to UNIX-like operating systems, WINDOWS-like operating systems, ANDROID-like operating systems, IOS operating systems, and consumer interfaces can be applied to JAVA language interfaces.

One of ordinary skill in the art will appreciate that all or a portion of the above steps may be accomplished by a program that instructs the associated hardware, such as a read-only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the foregoing embodiment may be implemented in the form of hardware or in the form of a software function module. Embodiments of the invention are not limited to any specific form of combination of hardware and software.

It should be noted that the above-mentioned embodiments are only for the convenience of those skilled in the art, and are not intended to limit the scope of the present invention.

Industrial applicability

With the solution of the embodiment of the present invention, the first device issues the identification information of the identified object, and shares the identification information of the identified object, so that the other device can identify the identified object by receiving the issued identification information, regardless of whether Identify if the object is within its capture range.

Claims

A method of distributing identification information, including:

The first device issues identification information; the identification information includes: coordinates of the identified object in the first coordinate system, and first coordinate system information.
The method of claim 1 further comprising:

The first device captures an image, and the identified object is identified according to the captured image to obtain the identification information.
The method of claim 1 or 2, wherein the coordinates of the identified object in the first coordinate system are coordinates of one or more points on the identified object in the first coordinate system.
The method of claim 1, wherein the identification information further comprises one or more of the following information:

The distance between the identified object and the first device, the network address of the identified object, the communication port of the identified object, the identified object identifier, and the first device identifier.
The method according to claim 1, wherein the second device is the identification object, and the coordinates of the second device in the first coordinate system are coordinates of the recognition object in the first coordinate system.
The method according to any one of claims 1 to 5, wherein the first device issues the identification information in a broadcast, or multicast, or unicast, or in response to a query request.
A method of communicating identification information, including:

The second device receives the identification information; the identification information includes: coordinates of the identified object in the first coordinate system, and first coordinate system information.
The method of claim 7 further comprising:

The second device calculates coordinates of the identified object in the second coordinate system according to the identification information.
The method according to claim 8, wherein the calculating, by the second device, the coordinates of the identified object in the second coordinate system according to the identification information comprises:

The second device is based on coordinates of the second device in the first coordinate system and the second device The transformation matrix is calculated in the coordinates in the second coordinate system, and the coordinates of the identified object in the second coordinate system are calculated according to the transformation matrix and the identification information.
The method according to claim 7 or 8, wherein the identification information further comprises one or more of the following information:

The coordinates of the one or more points on the identified object in the first coordinate system, the network address of the identified object, the communication port of the identified object, the identified object identifier, and the first device identifier.
The method of claim 10, further comprising:

The second device communicates with the identified object according to the calculated or received coordinates, network address or communication port of the identified object in the second coordinate system.
A device for distributing identification information, comprising:

The first network unit is configured to issue identification information; the identification information includes: coordinates of the identified object in the first coordinate system, and first coordinate system information.
The device of claim 12, the device further comprising:

The first visual recognition unit is configured to capture an image, and identify the identified object according to the captured image to obtain the identification information.
The apparatus of claim 12, wherein the first network element is further configured to:

The identification information is published in a broadcast, or multicast, or unicast, or in response to a query request.
A device for transmitting identification information, comprising:

The second network unit is configured to receive the identification information; the identification information includes: coordinates of the identified object in the first coordinate system, and the first coordinate system information.
The device of claim 15 further comprising:

The second visual recognition unit is configured to calculate coordinates of the identified object in the second coordinate system based on the identification information.
The apparatus according to claim 16, wherein said second visual recognition unit is configured to:

Calculating a conversion matrix according to coordinates of the second device in the first coordinate system and coordinates of the second device in the second coordinate system, and calculating the identified object according to the conversion matrix and the identification information The coordinates in the two coordinate system.
The device according to claim 15 or 16, further comprising:

a storage module, configured to save one or more of the following: coordinates of one or more points on the identified object in a first coordinate system, network address or communication port of the identified object, Identify the object identifier, the first device identifier.
An information recognition system comprising:

a first device, configured to issue identification information; the identification information includes: coordinates of the identified object in the first coordinate system, first coordinate system information;

The second device is configured to receive the identification information.
The information recognition system according to claim 19, wherein said first device is further configured to:

The image is captured, and the identified object is identified based on the captured image to obtain the identification information.
The information recognition system according to claim 19, wherein said first device is further arranged to issue said identification information in a broadcast, or multicast, or unicast, or in response to a query request.
The information recognition system according to claim 19, wherein said second device is further configured to:

Calculating coordinates of the identified object in the second coordinate system based on the identification information.
The information recognition system according to claim 22, wherein said second device is configured to:

Calculating a conversion matrix according to coordinates of the second device in the first coordinate system and coordinates of the second device in the second coordinate system, and calculating the identified object according to the conversion matrix and the identification information The coordinates in the two coordinate system.
The information recognition system according to claim 22, wherein said second device is further configured to:

Saving one or more of the following information: coordinates of one or more points on the identified object in the first coordinate system, network address or communication port of the identified object, identified object identifier, A device identifier.
A computer readable storage medium storing program instructions that, when executed, can implement the method of any of claims 1-6.
A computer readable storage medium storing program instructions that, when executed, implement the method of any of claims 7-11.