WO2014101007A1

WO2014101007A1 - Method and apparatus for transmitting information within a preset environment

Info

Publication number: WO2014101007A1
Application number: PCT/CN2012/087549
Authority: WO
Inventors: Wei Zhou; Jun Xu; Xiaojun Ma
Original assignee: Thomson Licensing SAS
Current assignee: Thomson Licensing SAS
Priority date: 2012-12-26
Filing date: 2012-12-26
Publication date: 2014-07-03
Anticipated expiration: 2015-06-26

Abstract

The present invention provides a method and apparatus for transmitting information within a preset environment, wherein the information of an object is transmitted according to the social relationship with a member. The method comprises: recognizing an object according to a feature of the object; determining the social relationship between the object and members within a preset environment; and transmitting information of the object as a function of the determined social relationship with the members.

Description

METHOD AND APPARATUS FOR TRANSMITTING INFORMATION WITHIN A

PRESET ENVIRONMENT

TECHNICAL FIELD

The present invention relates to network technology. In particular, the present invention relates to method and apparatus for transmitting information within a preset environment.

BACKGROUND

This section is to provide the reader with background information to facilitate a better understanding of the various aspects of the present invention. It should not be understood as admissions of prior art.

A conventional home surveillance system may comprise a camera to capture the facial information of a visitor in the form of static image or video, based on which some applications in terms of home surveillance can be designed.

One example of such application is a kind of "remote surveillance" service provided by some mobile operators in combination with the home surveillance system of a family home. With such service, the facial information of a visitor which was captured by the camera of the home surveillance system will be transmitted to the mobile phones of all family members subscribing to this service via wireless network.

Another example is to output the facial information of a visitor, either in the form of static image or video, to one or more devices connected to the home network for displaying on these devices. Any types of devices with a display function can be used in such application, such as television receiver, laptop and desktop.

One disadvantage of the above-described conventional home surveillance system is that the facial information of a visitor will be provided to all subscribing devices of the home surveillance system in an indiscriminate manner, which in some cases may disturb some home members.

Specifically, in the above-described "remote surveillance" service, the coming of a visitor will be notified to all family members (by facial information in this case), no matter whether the visitor has any relationship with a specific family member. However, it is very likely that a family member will not expect to receive a notification of a visitor who has nothing to do with him/her. In addition, this service is inefficient in a sense from the information transmission point of view since unnecessary communications are carried out over the network.

In the second example described above, if the facial information of a visitor is displayed on all devices in the home network, some home members might be interrupted or annoyed, especially when he or she is watching an exciting TV program or working with his/her laptops or desktops.

There is a need for a more efficient and user friendly solution for the information transmission within a present environment in the above or similar applications.

SUMMARY

In view of the above problem in the prior art, the present invention provides a method and apparatus for transmitting information within a preset environment, wherein the information of an object is transmitted according to the social relationship with a member.

According one aspect of the invention, a method is provided. The method comprises: recognizing an object according to a feature of the object; determining the social relationship between the object and members within a preset environment; and transmitting information of the object as a function of the determined social relationship with the members

According another aspect of the invention, an apparatus in home surveillance system is provided. The apparatus comprises: a recognition unit for recognizing a visitor according to a feature of the visitor; a determination unit for determining the social relationship between the visitor and members of the home; and a processing unit for transmitting information of the visitor as a function of the determined social relationship with the members.

It is to be understood that more aspects and advantages of the invention will be found in the following detailed description of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide further understanding of the embodiments of the invention together with the description which serves to explain the principle of the embodiments. The invention is not limited to the embodiments.

In the drawings:

Figure 1 is a diagram showing a flow chart of a method for transmitting information in home surveillance system according to an embodiment of the present invention;

Figure 2 is a diagram showing the structure of a home surveillance system according to an embodiment of the present invention;

Figure 3 is a diagram showing the structure of the database used for human face recognition and social relationship determination according to an embodiment of the present invention;

Figure 4 is a diagram showing a flow chart of a process for transmitting information in home surveillance system; and

Figure 5 is a diagram showing a flow chart of another process for transmitting information in home surveillance system.

DETAILED DESCRIPTION Figure 1 is a diagram showing a flow chart of a method for transmitting information in home surveillance system according to an embodiment of the present invention.

At the step 101 , a visitor is recognized according to a feature of the visitor. The feature could be a facial feature of the visitor which captured by a camera. Conventional human face detection and recognition technologies could be used in this respect.

At the next step 102, the social relationship between the visitor and a family member is determined according to the recognition result. The determination process can be based on a local database of social relationship or a social network.

At the next step 103, information of the visitor is sent to a family member having the social relationship with the visitor. The information could be sent in any proper forms to various devices. For example, it also could be either a text message sent to the mobile phone of the family member or an email sent to the laptop or desktop of the relevant family member. In both cases, facial image of the inventor could be sent along with text message.

Figure 2 is a diagram showing the structure of a home surveillance system according to an embodiment of the present invention for implementing the method described above.

As shown in Figure 2, the home surveillance system 200 comprises a recognition unit 201 for recognizing an object in its detection range. For example, the object could be a person who visits the home by knocking at the door of the home (hereinafter referred to as a visitor of the home). The recognition can be based on one or more features of the object. In the above example that the mentioned object is a visitor, the feature could be the facial feature of the visitor. Next, the embodiment of the invention will be described with reference to the facial feature detection and recognition of a visitor of a home for illustrative purpose.

Any appropriate technologies of human face detection and recognition could be used for the recognition unit 201 of the embodiment of the invention. For example, US patent application US20070098229, "method and apparatus for human face detection and recognition used in a preset environment", WU et al, published on May 2, 2007 disclosed such a solution. This patent application proposes to use cameras to acquire human face image, and then features in the human face image was detected and analyzed based on a member face database to recognize the person. Please note that human recognition will be used for the embodiment of the present invention but is not the topic to be discussed by the invention. Therefore, no further details will be discussed in this respect.

The home surveillance system 200 further comprises a determination unit 202 for determining the social relationship between the visitor and a family member based on the result of the recognition unit 201 .

For purpose of determining the social relationship, a database 204 of social relationships of the family members can be maintained. It could be appreciated by a person skilled in the art that the database for the determination of the social relationship by the determination unit 202 and the database for the recognition process by the recognition unit 201 can be combined into one database.

Figure 3 is a diagram showing the structure of the database used for human face recognition and social relationship determination according to an embodiment of the present invention. As shown in Figure 3, there are 4 tables in the database respectively named "Person" 301 , "Family" 302, "Facial" 303, and "PersonFriend" 304. Table "Person" 301 is used to save a person's personal information. Field "personid" is the primary key of table "Person" 301 . Field "familyid" is a foreign key of the table "Person" 301 and refers to the primary key "familyid" of table "Family" 302. Field "facialid" is another foreign key of the table "Person" 301 which refers to the primary ky "facialid" of table "Facial" 303. In the table "Facial" 303, there are two fields except for the primary key, one is field "facialharr" representing facial HARR wavelet feature, another one is field "facialvq" representing facial vector quantization feature. Associate table "PersonFriend" 304 is used to build up the relations between two persons who belong to the table "Person" 301 . In the table "PersonFriend" 304, both field "personid" and field "friendid" point to the primary key "personid" in table "Person" 301 .

It could be appreciated that the human feature can comprises but is not limited to facial feature. Other features which are applicable for human recognition, such as voice feature, can also be used according to the context of the application. The structure of the database shown in of Figure 3 comprises two fields corresponding to facial feature, which are Facial HARR wavelet feature and Facial VQ (vector quantization) feature, which can be used for human recognition carried out by the recognition unit 101 .

Data on the social relationships of the above-described database can be collected off-line or from social network.

A social network is a social structure made up of individuals (or organizations) called "nodes," which are tied (connected) by one or more specific types of interdependency, such as friendship, kinship, common interest, financial exchange, dislike, or relationships of beliefs, knowledge or prestige. SNS (Social Network System), such as Facebook or renren.com, can provide users / third party companies some interfaces in the manner of web services. The messages got via web services can be XML, or JSON or plain text. The messages can retrieve a user's social network relationships providing he / she provides user name and password for accessing SNS. More specially, the relationships among users can be got via the activities participated by them. By getting the relationships among different people, and providing the facial parameters as an important property for each person. Such data can be used by the embodiment of the invention to have facial recognition, characteristics matching and information transmission to relevant family members.

The data structure described in this invention is universal and highly abstracted data model definition. It has already taken the differences among various SNS into careful consideration. Therefore, it is unnecessary to adapt or modify it to keep compatible to a specific SNS.

At the determination unit 202, the social relationship between the visitor and a family member can be determined according to the recognition result of the recognition unit 1 based on the database of social relationship.

Back to Figure 1 , the home surveillance system 200 further comprises a processing unit 203 for providing the information of the visitor to relevant family members according to the social relationship determined by the determination unit 202. Next, a detailed description will be given to this process.

Firstly, this process will be explained with reference to an example that the above-described "remote surveillance" service is applied in the home surveillance system. Figure 4 is a diagram showing a flow chart of a method for transmitting information in home surveillance system in this case.

As shown in Figure 4, before the processing unit 203 starts to process the information of the visitor, a human face recognition is carried out by the recognition unit 201 at step 401 and then it determines by the determination unit 202 whether the visitor has social relationship with any family members at step 402.

If the visitor has a social relationship with at least one of the family members, in which case he/she is defined as a "guest" welcomed into the family, at step 403 the processing unit 203 will transmit the information of the visitor (obtained from the above database) to those family members who have social relationships with the person. As one example, the information of the visitor could be a text message sent to the mobiles phones of relevant family members, indicating the name of the guest as well as the visiting of the guest. The captured facial image can also be transmitted to mobile phones of relevant family members. It could be appreciated that the forms of the information and devices of relevant family members for receiving the information can be implemented in various forms. For example, it also could be an email sent to the laptop or desktop of relevant family members. Any other suitable forms can also be applied.

The information sent to a family member can request for a response from the relevant family members, based on which some operations could be made to the home surveillance system. For example, if the home surveillance system supports the remote control of the gate, the response could be an instruction to open the gate for the guest.

If it is determined by the determination unit 202 that the visitor does not have any social relationship with all the family members, in which case he/she is defined as a "stranger" to the family, at step 404 the processing unit 203 can transmit the information of the stranger to all family members. Similarly, the forms of the information and devices of family members for receiving the information can be implemented in various forms. For example, a text message can be sent to mobile phones of the family members to notify them that a stranger is visiting and the captured facial image of the stranger can also be transmitted to mobile phones of all family members.

In response to the notification of a stranger, at step 405 any one of the family members can send a message to the processing unit 203 to notify that the "stranger" has some social relationships with him/her along with corresponding information. In this case, at step 405 the processing unit 203 can update the database of social relationships of the family members, for example, by adding one record for this visitor into the database and updating corresponding fields of the record with the information sent by the family member and facial image captured by the camera. It could be appreciated that the above update could be done not only within the home surveillance system 200 but also in the social network of the family members.

Next, the process of the processing unit 203 will be explained with reference to another example that the image of a visitor can be displayed on the devices connected to the home network according to the social relationship determined by the determination unit 202. Figure 5 is a diagram showing a flow chart of a method for transmitting information in home surveillance system in this case.

Each device of the home can be registered in home network to have a family member as an owner or user, based on which the facial image/video of the visitor can be displayed on devices of relevant family members.

As shown in Figure 5, before the processing unit 203 starts to process the information of the visitor, a human face recognition is carried out by the recognition unit 201 at step 501 and then it determines by the determination unit 202 whether the visitor has social relationship with any family members at step 502.

If it is determined by the determination unit 202 that the visitor does not have any social relationship with a family member, in which case he/she is defined as a "stranger" to this family member, at step 503 the processing unit 203 will not transmit the image of the visitor to be displayed on devices of this family member.

If it is determined by the determination unit 202 that the visitor has a social relationship with a family member, in which case he/she is defined as a "guest" to this family member. In this case, the above facial image/video can be displayed on the device of relevant family members in a Picture in Picture (PiP) manner according to the degree of the social relationship. This process is shown by the step 505 in Figure 5.

PiP is a feature of some television receivers and similar devices, with which one program (channel) is displayed on the full screen of television receiver and at the same time one or more other programs could be displayed in inset windows. Sound is usually from the main program only. The first wide spread consumer implementation of PiP was produced in 1983 in high end television sets. A separate video or RF input was available on the back of the set and displayed in black and white in one or the four corners of the screen. Televisions at the time were still analog format and earlier versions of the PiP implemented in analog were too costly. New digital technology allowed the second video signal to be digitized and save in a digital memory chip then replayed in a mini version. While the new technology was not good enough for color or full screen viewing, it did provide a low cost PiP feature.

PiP requires two independent tuners or signal sources to supply the large and the small pictures. Two-tuner PiP television receivers have a second tuner built in, but a single-tuner PiP television receiver requires an external signal source, which may be an external tuner, VCR, DVD player, or a cable box with composite video outputs. For example, all of the Blu-ray Disc titles in 2006 and 2007 that had a PiP track used two separate HD encodings with one of the HD encodings including a hard coded PiP track. Starting in 2008 Blu-ray Disc titles started being released that use one HD and one SD video track which can be combined together with a Bonus View or BD-Live player.

PiP is often used to watch one program while waiting for another to start or advertisements to finish. PiP in the Blu-ray Disc and HD DVD specifications allows viewers to see, for example, the director's comment on a film they are watching.

The PiP can be a rectangle, a circle, an eclipse, or any other shapes.

It is known that social network can provides information on the degree of social relationship between two people. Here degree means the count of the number of ties to other actors in the network, which reflects the closeness that an individual is near all other individuals in a network. In an alternative example, the size of PiP window displayed on a device of a family member can be adapted according to such degree of social relationship between a visitor and the family member.

According to the theory of six degrees of separation, everyone is approximately six or fewer steps away, by way of introduction, from any other person in the world, so that a chain of "a friend of a friend" statements can be made, on average, to connect any two people in six steps. Therefore, as shown in Figure 5, before displaying the image of the inventor in PiP at step 505, at step 504 the degree of social relationship between a visitor and a family can be determined and ranked 1 , 2, 3, 4, 5, and 6, wherein the number 1 represents the closest relation and the number 6 represents the furthest relation. In displaying the image in PiP, at step 506 the size of PiP window can be adapted according to the determined degree of social relationship. Suppose the reference width and height of PiP are respectively w and h, then the actual width and height of PiP can be calculated respectively by w*(7 - D) and h*(7 - D), wherein D refers to the degree of social relationship between a visitor and the family member.

It should be noted that the size of PiP window is only one example of the parameter that could be adapted according to the degree of social relationship between a visitor and the family member. Other parameters, such as the position, color, rendering style, specific icons, audio clips, or any other video or graphics processing solution of the PiP window can also be adapted individually or in combination at step 506 shown in Figure 5.

The degree information can also be manually configured by a family member, and then the record in the database can be updated.

It could be appreciated that if the visitor does not have any social relationship with the family members, then no image or video will be displayed on the devices of the family members. The visiting of the visitor can be notified to the family members in another way, for example, just by ringing the door bell.

It will be understood that the present invention has been described purely by way of example, and modifications of detail can be made without departing from the scope of the invention.

Claims

1. A method, comprising the steps of:

recognizing (101 ) an object according to a feature of the object;

determining (102) the social relationship between the object and members within a preset environment; and

transmitting (103) information of the object as a function of the determined social relationship with the members.

2. The method according to claim 1 , further comprising:

determining the social relationship according to information from a social network.

3. The method according to claim 1 , further comprising:

transmitting information of the object only to the members having social relationship with the object.

4. The method according to claim 1 , further comprising:

transmitting information of the object to the members having social relationship with the object in a way different from that to those not having relationship with the object.

5. The method according to claim 1 , further comprising:

receiving a response from a member to update the information on the social relationship.

6. An apparatus in a home surveillance system, comprising:

a recognition unit (201 ) for recognizing a visitor according to a feature of the visitor;

a determination unit (202) for determining the social relationship between the visitor and members of the home; and a processing unit (203) for transmitting information of the visitor as a function of the determined social relationship with the members.

7. The apparatus according to claim 6, wherein the feature of the visitor is the 5 facial image captured by a camera of the home surveillance system.

8. The apparatus according to claim 7, further comprising displaying the image of the visitor on devices of members having social relationship with the object in a Picture in Picture manner.

o

9. The apparatus according to claim 8, further comprising adapting the size, position, color, rendering style, specific icons or audio clips of the Picture in Picture window as a function of the degree of the social relationship between the visitor and the members.

5