CN114928724B

CN114928724B - Image output control method, system, electronic equipment and storage medium

Info

Publication number: CN114928724B
Application number: CN202210603386.XA
Authority: CN
Inventors: 周迪; 邓松杰; 王殿盟; 袁佳怡; 卓越; 徐爱华; 王威杰
Original assignee: Zhejiang Uniview Technologies Co Ltd
Current assignee: Zhejiang Uniview Technologies Co Ltd
Priority date: 2022-05-31
Filing date: 2022-05-31
Publication date: 2023-11-24
Anticipated expiration: 2042-05-31
Also published as: CN114928724A

Abstract

The embodiment of the invention discloses an image output control method, an image output control system, electronic equipment and a storage medium. The method comprises the following steps: receiving multicast image data of a multicast group where an image data source is located; adding a video port of the display device into an output port list of the multicast routing table item; and according to the outlet port list of the multicast routing table item, decoding the image data and then sending the decoded image data to a video port of the display equipment for display. According to the image output control scheme provided by the embodiment of the disclosure, the image video is acquired in a multicast mode and is output to the video port of the display device after being decoded, so that the existing more preferable display device in the local area network environment can be fully utilized, a more convenient watching mode is provided for a user, and the user experience of related images or video applications is greatly improved.

Description

Image output control method, system, electronic equipment and storage medium

Technical Field

The present disclosure relates to, but not limited to, the field of smart home, and in particular, to an image output control method, system, electronic device, and storage medium.

Background

Home monitoring systems are becoming increasingly popular, and users view video monitoring pictures or historical monitoring videos through home monitoring devices with display screens or display screens of personal intelligent terminals. It can be seen that these existing video display screens are portable devices and are relatively small, which is inconvenient for the elderly and children. How to improve the use convenience of the home monitoring system is a continuous improvement in the field.

Disclosure of Invention

The embodiment of the disclosure provides an image output control method, an image output control system, an electronic device and a storage medium, wherein an image video is acquired in a multicast mode, decoded and output to a video port of a display device, so that the existing more preferable display device in a local area network environment can be fully utilized, a more convenient watching mode is provided for a user, and the user experience of related images or video applications is greatly improved.

The embodiment of the disclosure provides an image output control method, which comprises the following steps:

receiving multicast image data of a multicast group where an image data source is located;

adding a video port of the display device into an output port list of the multicast routing table item;

and according to the outlet port list of the multicast routing table item, decoding the image data and then sending the decoded image data to a video port of the display equipment for display.

The embodiment of the disclosure also provides an image output control system, which comprises:

a display device, an image data source device, and an output control device;

the output control device is configured to receive multicast image data of a multicast group in which the image data source device is located;

the output control device is further configured to add the video port of the display device to an egress port list of the multicast routing table entry; and according to the outlet port list of the multicast routing table item, decoding the image data and then sending the decoded image data to a video port of the display equipment for display.

The embodiment of the disclosure also provides an electronic device, including:

one or more processors;

storage means for storing one or more programs,

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the image output control method as described in any of the embodiments of the present disclosure.

The embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the image output control method according to any of the embodiments of the present disclosure.

Other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of an image output control method according to an embodiment of the present invention;

FIG. 2 is a flowchart of another image output control method according to an embodiment of the present invention

FIG. 3 is a flowchart of another image output control method according to an embodiment of the present invention;

fig. 4 is a schematic application scenario diagram of an image output control method according to an embodiment of the present invention;

FIG. 5 is a flowchart of another image output control method according to an embodiment of the present invention;

fig. 6 is a schematic view of an application scenario of another image output control method according to an embodiment of the present invention;

FIG. 7 is a flowchart of another image output control method according to an embodiment of the present invention;

FIG. 8 is a flowchart of another image output control method according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an image output control system according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of another image output control system according to an embodiment of the present invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that all directional indicators (such as up, down, left, right, front, and rear … …) in the embodiments of the present invention are merely used to explain the relative positional relationship, movement, etc. between the components in a particular posture (as shown in the drawings), and if the particular posture is changed, the directional indicator is changed accordingly.

Furthermore, descriptions such as those referred to as "first," "second," and the like, are provided for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implying an order of magnitude of the indicated technical features in the present disclosure. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.

In the present invention, unless specifically stated and limited otherwise, the terms "connected," "affixed," and the like are to be construed broadly, and for example, "affixed" may be a fixed connection, a removable connection, or an integral body; can be mechanically or electrically connected; either directly or indirectly, through intermediaries, or both, may be in communication with each other or in interaction with each other, unless expressly defined otherwise. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.

In addition, the technical solutions of the embodiments of the present invention may be combined with each other, but it is necessary to be based on the fact that those skilled in the art can implement the technical solutions, and when the technical solutions are contradictory or cannot be implemented, the combination of the technical solutions should be considered as not existing, and not falling within the scope of protection claimed by the present invention.

In some implementations, the home monitoring system views the video monitoring screen or the history monitoring video through the display screen attached to the home monitoring device or the display screen of the personal intelligent terminal, but the display screen of these portable devices is relatively small, and for the old or the children, there may be a use obstacle due to presbyopia or other factors.

The technical staff find that the television is arranged in a common household, and compared with a display screen of a monitoring terminal or a smart phone, the display screen is larger, and the display effect is better. Therefore, the embodiment of the disclosure provides an image output control scheme, which can display image data from a visual doorbell or a monitoring camera in a home monitoring system through a television according to the identification result of whether the old or the child is watching television, so that the convenience of using the home monitoring system by the old or the child is greatly improved.

It should be noted that, the image output control scheme provided by the embodiment of the disclosure is not only suitable for a home monitoring system, but also can be popularized and applied to a plurality of application systems related to image data display, and according to personnel acquisition information, the image data is output to a display device associated with personnel, or the image data is transferred from an original display device corresponding to the image data by default or is newly added to the display device associated with the personnel.

An embodiment of the present disclosure provides an image output control method, as shown in fig. 1, including:

step 110, receiving multicast image data of a multicast group where an image data source is located;

step 120, adding the video port of the display device into the output port list of the multicast routing table item;

and 130, decoding the image data according to the outlet port list of the multicast routing table entry, and sending the decoded image data to a video port of the display equipment for display.

In some exemplary embodiments, after decoding the image data in step 130, sending the decoded image data to a video port of the display device for display, including:

and after the image data are decoded, the image data are overlapped in the original video stream decoded image of the video port of the display equipment, and are sent to the video port of the display equipment for display.

In some exemplary embodiments, the display device is a television. It can be understood that the original video stream of the television is a television video stream, the image data is decoded and then superimposed into the decoded image of the television video stream, and the decoded image is sent to the video port of the television for display, so that the picture-in-picture display effect can be presented on the television.

In some exemplary embodiments, the image data may be picture data or video data.

In some exemplary embodiments, step 120 comprises:

acquiring personnel acquisition information, and determining corresponding display equipment according to the personnel acquisition information;

and under the condition that the personnel acquisition information accords with the set image output condition, adding the video port connected with the display equipment into an outlet port list of the multicast routing table item.

In some exemplary embodiments, the method is applied to a set top box, and the display device is a television; the video port is a port of the set top box connected with the television; and the set top box decodes the television program and then sends the decoded television program to the television from the video port.

In some exemplary embodiments, the set-top box and the image data source that generated the image data reside on the same local area network.

In some exemplary embodiments, the aging time of the video port of the television added to the output port list of the multicast routing table entry of the set-top box is infinite, that is, the video port of the television will not age.

It is appreciated that in some available multicast schemes, the egress port list of the usual multicast routing entry (G) entries contains only network ports. In some exemplary embodiments of the present disclosure, when a television is used as the display device, a video port (non-network port) of the television is added to an egress port list of a multicast routing table entry (x, G) table entry. After the set top box joins the multicast group, the set top box receives the video stream (image data) from the image data source, the destination address of which is H.264 or H.265 code of the multicast group G, and the set top box sends the video stream out from the outlet port list of the (G) table item; if the output port in the list is a network port, the video port is directly forwarded, and if the output port is a video port, the video port is firstly decoded, then the decoded image is overlapped into the decoded television program image and then sent out, so that the picture-in-picture effect is displayed on the television picture.

In some exemplary embodiments, determining a corresponding display device from the personnel acquisition information includes:

And searching a preset corresponding relation between the acquisition equipment and the display equipment according to the source acquisition equipment for acquiring the information by the personnel, and determining the corresponding display equipment.

In some exemplary embodiments, the person acquisition information is a face image, the source acquisition device is a video camera, and the display device is a television. For example, the preset correspondence between the acquisition device and the display device is as follows:

sequence number	Acquisition device	Display apparatus
			1	Video camera 1	TV set in living room
2	Video camera 2	Main horizontal television
			3	Video camera 3	Secondary lying television

It can be understood that when the person acquired information (face image) comes from the camera 1, the corresponding display device is determined to be the living room television according to the corresponding relationship; when the person acquired information (face image) comes from the camera 3, the corresponding display device is determined to be the secondary lying television according to the corresponding relation.

In some exemplary embodiments, the method further comprises: and 100, configuring the corresponding relation between the acquisition equipment and the display equipment.

I.e. the step 100 is performed to perform a pre-configuration of the correspondence between the acquisition device and the display device, so as to perform a search in step 120, and determine the display device.

In some exemplary embodiments, before the multicast image data of the multicast group where the received image data source is located, step 110 further includes:

And joining the multicast group where the image data source is.

In some exemplary embodiments, the joining the multicast group where the image data source is located includes:

acquiring personnel acquisition information;

and sending a multicast group joining message to join the multicast group where the image data source is located under the condition that the personnel acquisition information accords with the set image output condition.

In some exemplary embodiments, sending a multicast group join message includes:

periodically sending PIM joining messages (Join messages) of multicast routing list items (G) to other network ports in the local area network;

the switch in the lan generates a multicast routing table entry (x, G) and adds the port that receives the PIM join message to its own multicast routing table entry (x, G).

In some exemplary embodiments, the multicast group in which the image data source is located may also be directly joined by manual configuration.

In some exemplary embodiments, the PIM join message periodically sent to other network ports in the local area network (x, G) includes:

and in the case that the multicast routing table item (G) comprises the personnel characteristic value, periodically sending PIM addition messages (G) to other network ports in the local area network.

and periodically sending (G) PIM addition messages to other network ports in the local area network according to the first time period.

It will be appreciated that in accordance with the above steps, some exemplary embodiments provide for the device to join a multicast group in the local area network; after receiving the image data from the image data source, the switch forwards the received image data to the device according to the multicast routing table entry (G) of the switch.

In some exemplary embodiments, the initial multicast routing table entry (G) does not include a personal characteristic value.

In some exemplary embodiments, the egress port list of multicast routing entries (x, G) that do not contain personal characteristic values is empty.

In some exemplary embodiments, the joining the multicast group where the image data source is located further includes:

and under the condition that the personnel acquisition information accords with the set image output condition, storing the personnel characteristic value corresponding to the personnel acquisition information, and initially marking the leaving state of the personnel characteristic value as 'not leaving'.

And under the condition that the personnel acquisition information accords with the set image output condition, adding the personnel characteristic value corresponding to the personnel acquisition information into the multicast routing table item.

In some exemplary embodiments, when the person identification result corresponding to the person acquisition information includes at least one person of a set type, the person feature value corresponding to the person acquisition information is added to the multicast routing table entry.

In some exemplary embodiments, when the person identification result corresponding to the person acquisition information includes at least one set person, the person feature value corresponding to the person acquisition information is added to the multicast routing table entry.

In some exemplary embodiments, the personnel acquisition information includes: a face image or a photographed image containing a person.

In some exemplary embodiments, the acquiring personnel acquisition information includes: personnel acquisition information is acquired through a camera.

In some exemplary embodiments, the person acquisition information is face information, and the corresponding person feature value is a face feature vector; namely, the face feature vector is added into the multicast routing table entry. For example, a face feature vector is added to the multicast routing table entry (G).

In some exemplary embodiments, the person acquisition information is a face image, and the corresponding face feature vector is generated through a neural network.

For example, the table entry (G) added to the face feature vector is as follows:

in some exemplary embodiments, the leaving status of the newly added person feature value in the multicast routing table entry is initially marked as "not leaving".

In some exemplary embodiments, when the person identification result corresponding to the person acquisition information includes at least one person of a setting type, and the display switch state of at least one person of the setting type is empty or on, determining that the set image output condition is met; the case (1) is described.

In some exemplary embodiments, the person identification result corresponding to the person acquisition information includes at least one set person, and when the display switch state of at least one set person is empty or on, it is determined that the set image output condition is met; the case (2) is described.

In some exemplary embodiments, the setting personnel information corresponds to personnel characteristic values in the multicast routing table entry. That is, the setting personnel information is stored in the multicast routing table entry. In some exemplary embodiments, when the face feature vector corresponding to the person identification result can be matched with at least one face feature vector in the multicast routing table entry, it indicates that the person identification result corresponding to the person acquisition information includes at least one set person.

In some exemplary embodiments, the method further comprises:

and 150, acquiring and storing display switch state setting information of the identified personnel.

When each person to be identified does not set the respective display switch state in step 150, the corresponding display switch state is empty.

In some exemplary embodiments, the display switch state corresponding to the person feature value of the identified person in the multicast routing table entry is marked according to the display switch state of the identified person. And the personnel characteristic values stored in the multicast routing table entries respectively mark the display switch states corresponding to the personnel.

In some exemplary embodiments, the method further comprises:

step 160, acquiring personnel tracking information;

and 180, deleting the video port connected with the display device from the outlet port list of the multicast routing table item under the condition that the set stop image output condition is met according to the personnel tracking information.

In some exemplary embodiments, the person tracking information includes: personnel gather information and departure indication information.

In some exemplary embodiments, the camera has a person tracking function that generates person tracking information when the tracking detects that a person leaves the screen. The personnel acquisition information is an image cached by the camera.

In some exemplary embodiments, the camera has only a shooting function, and the set-top box or other device performs a person tracking function, generating person tracking information when the tracking detects that a person leaves the screen.

In some exemplary embodiments, the acquiring person tracking information includes:

personnel tracking information is acquired by a camera.

In some exemplary embodiments, the camera supports the determination of face detection and the type of person set in the picture. Correspondingly, the camera generates face images of preset types and caches the face images.

In some exemplary embodiments, in a case where it is determined that the person whose display switch is empty or whose display switch is on among the persons of the at least one setting type is all away from the person based on the person tracking information, the set stop image output condition is met, which is denoted as case (a).

Alternatively, in some exemplary embodiments, in a case where it is determined that the person whose display switch is empty or whose display switch is on among the at least one person is all away from the person based on the person tracking information, the set stop image output condition is met, which is denoted as the case (b).

In some exemplary embodiments, the display switch state of the corresponding mark of each personnel characteristic value is obtained from the multicast routing table entry.

In some exemplary embodiments, the leaving status of each personnel characteristic value corresponding to the mark is obtained from the multicast routing table entry.

In some exemplary embodiments, after step 160, the method further comprises:

step 170, marking the leaving state of the personnel characteristic value corresponding to the personnel acquisition information in the personnel tracking information as leaving according to the personnel tracking information; or deleting the personnel characteristic value corresponding to the personnel acquisition information in the saved personnel tracking information according to the personnel tracking information.

In some exemplary embodiments, the leaving status corresponding to the personnel feature value is stored in the multicast routing table entry, and accordingly, the leaving status of the corresponding personnel feature value in the multicast routing table entry is marked as "leaving" in step 170; or deleting the corresponding personnel characteristic value in the multicast routing table entry.

In some exemplary embodiments, the method further comprises:

and step 190, forwarding the image data to a network port included in the outlet port list of the multicast routing table item.

It can be understood that, in the case that the egress port list of the multicast routing table entry further includes a network port, the image data is directly forwarded through the network port, without decoding the image data.

In some exemplary embodiments, the method is applied to a set top box, and the display device is a television.

In some exemplary embodiments, the image data source is a smart video doorbell. The intelligent visual doorbell, the network camera and the set top box belong to the same local area network, are interconnected through a network switch, the television is associated with the network camera, and the set top box is connected with the television through a video port.

An embodiment of the present disclosure provides an image output control method, as shown in fig. 2, including:

step 210, acquiring personnel acquisition information;

step 220, determining corresponding display equipment according to the personnel acquisition information;

and step 230, acquiring image data and outputting the image data to the display equipment for display under the condition that the personnel acquisition information meets the set image output conditions.

In some exemplary embodiments, the step 220 includes:

In some exemplary embodiments, the person of the set type includes one or more of the following types: old people and children.

For example, if the setting type is old people and the person collecting information is a face image, in step 210, the face image is obtained, in step 230, the face image is identified, the identification result includes the old people, and if the display switch state of at least one old people is empty or on, it is determined that the situation (1) belongs, and the set image output condition is met.

Optionally, in some exemplary embodiments, the person identification result corresponding to the person acquisition information includes at least one set person, and if a display switch state of at least one set person is empty or on, it is determined that the set image output condition is met; the case (2) is described.

In some exemplary embodiments, the person of the set type includes one or more of the following types: doctor, security, etc. The person types to be identified are set as required according to different application scenes, and are not limited to aspects of examples of the embodiments of the present disclosure.

In some exemplary embodiments, the setup personnel includes one or more.

For example, setting a person to be Zhang San, wherein the person acquisition information is a face image, acquiring the face image in step 210, identifying the face image in step 230, and determining that the person belongs to the condition (2) if the identification result comprises Zhang San and the display switch state of Zhang San is empty and meets the set image output condition; if Zhang San is not included, then it does not belong to case (2).

For example, setting the person to be Zhang three and Liu four, wherein the person acquisition information is a face image, acquiring the face image in step 210, identifying the face image in step 230, wherein the identification result comprises Zhang three and/or Liu four, and determining that the person belongs to the condition (2) and accords with the set image output condition if the identified switch state of at least one person is empty or on; if neither Zhang Sannor Lifour is included, then it is not the case (2).

It should be noted that, identifying the person collected information to determine whether the person of the set type is included or not is only required to be executed according to the related person identifying scheme, and the method is not limited to a specific scheme, and specific aspects are not discussed in the embodiments of the present application. Identifying the personnel acquisition information to determine whether the personnel acquisition information contains a set personnel or not, and executing according to a related personnel identification scheme without being limited to a specific scheme; judging whether a certain setting person is included or not, and storing the characteristic value of the setting person in advance; specific aspects are not discussed in the embodiments of the present application.

For example, if the type is the old people, the face image is identified according to the related face recognition scheme to judge whether the old people are included. And if the person is the user A, identifying the face image according to a related face recognition scheme to judge whether the user A is contained or not, and pre-storing the face characteristic value or the face picture of the user A.

In some exemplary embodiments, prior to step 210, the method further comprises:

and 200, configuring the corresponding relation between the acquisition equipment and the display equipment.

I.e. step 200 is performed to pre-configure the correspondence between the acquisition device and the display device for searching in step 220.

In some exemplary embodiments, the method is applied to a set top box, and the display device is a television;

accordingly, the step 230 of acquiring image data and outputting the image data to the display device for display includes:

and the set top box acquires the image data, superimposes the image data on the original decoded image connected with the video port of the television, and outputs the image data to the television for display.

the set top box joins a multicast group where an image data source is located, and acquires the image data;

and adding a video port connected with the television into an output port list of the multicast routing list item of the set top box, and outputting the image data to the display equipment according to the output port list.

In some exemplary embodiments, the set top box joins a multicast group where the image data source is located, comprising:

periodically sending PIM joining messages (Join messages) of multicast routing list items (G) to other network ports in the local area network by the set top box;

In some exemplary embodiments, the initial multicast routing table entry (G) of the set-top box does not include a personal characteristic value.

In some exemplary embodiments, step 230 further comprises:

and under the condition that the personnel acquisition information accords with the set image output condition, adding the personnel characteristic value corresponding to the personnel acquisition information into a multicast routing table item (G) of the set top box.

In some exemplary embodiments, the set-top box periodically sends (, G) PIM join messages to other ports in the local area network, including:

and in the case that the multicast routing table item (x, G) comprises a personnel characteristic value, the set top box periodically transmits (x, G) PIM join messages to other network ports in the local area network.

and the set top box periodically sends (G) PIM joining messages to other network ports in the local area network periodically in a first duration.

It will be appreciated that in accordance with the above steps, some exemplary embodiments set-top boxes join a multicast group in a local area network; and after receiving the image data from the image data source, the switch forwards the received image data to the set top box according to the multicast routing table item (G) of the switch.

It should be noted that, the multicast routing table entry (x, G) on the switch may be aged, and after the aging time (Agingtime, also referred to as aging duration) arrives, the port corresponding to the set-top box is deleted from the egress port list of the multicast routing table entry (x, G), so that the multicast data is not forwarded from the port any more. In some exemplary embodiments, the aging time is 3 times the first duration, or other set duration.

In some exemplary embodiments, the multicast routing table entry is (S, G).

It should be noted that, in the embodiments of the present disclosure, the embodiments of the PIM (Protocol Independent Multicast ) multicast routing protocol are taken as examples to exemplify the related aspects. In addition to PIM protocols, other multicast routing protocols may be employed to enable forwarding or acquisition of related image data in a multicast group, such as: MBGP (Multicast Border Gateway Protocol, network multicast border gateway protocol) protocol, IGMP (Internet Group Management Protocol ) protocol, etc., are not limited to a particular aspect, and the relevant steps may be correspondingly adjusted.

In some exemplary embodiments, step 210 includes: personnel acquisition information is acquired through a camera.

In some exemplary embodiments, the camera only supports capturing of captured images, based on face detection and/or further face recognition of the captured images, performed by other devices; correspondingly, the other devices generate face images of preset types and cache the face images. In some exemplary embodiments, further face detection and/or face detection is performed by the set top box. The function of the related equipment is flexibly determined according to the scheme, and the method is not limited to a specific mode. In some exemplary embodiments, the camera is a webcam.

In some exemplary embodiments, when the person identification result corresponding to the person acquisition information includes at least one person of a set type, and the display switches of all the identified persons are in a closed state, determining that the set image output condition is met; the case (1) is described.

In some exemplary embodiments, when at least one set person is included in the person identification result corresponding to the person acquisition information, and the display switches of all the identified persons are not in the off state, determining that the set image output condition is met; the case (2) is described.

It will be appreciated that in some exemplary embodiments, the identified person may be provided with a corresponding display switch, the on state indicating: the person correspondingly allows the display device to display the image data, and the off state represents: the person is not allowed to display the image data in response to the display device.

In some exemplary embodiments, the identification result corresponding to the person acquired information includes a plurality of persons of a set type, and if the display switches of the plurality of persons are all in a closed state, the set image output conditions are not met; indicating that all the display switches corresponding to the currently identified person are in the off state, and displaying the image data on the display equipment in a newly added mode or in a transferred mode is not needed. If not all the display switches of all the people are in the off state, the set image output conditions are met. That is, if the display switch of at least one person is in an on state or the display switch state of at least one person is empty, the set image output condition is satisfied; indicating that the display switches corresponding to the currently identified persons are not all closed, and at least one person is required to display the image data in a newly added or transferred mode on the display equipment.

For example, user a, user B, user C, and user D, the corresponding display switch states are: on, off, NULL, i.e., user a allows display, user B does not allow display, user C does not allow display, and user D is not set. If the identification result comprises: the corresponding display switch states of the user A and the user B are as follows: opening and closing, wherein the condition (1) is that the set image output condition is determined; if the identification result includes the user B and the user C, the corresponding display switch states are: closing, namely determining that the set image output conditions are not met under the condition (1); if the identification result comprises: the user D, the corresponding display switch state is: NULL, not the off state, is determined to be in accordance with the set image output condition in case (1).

In some exemplary embodiments, the method further comprises:

step 221, the display switch setting information of the personnel is acquired and stored.

In some exemplary embodiments, in the case where the person acquisition information meets the set image output condition, the display switch setting information of the identified person is acquired and saved.

In some exemplary embodiments, display switch setting information of a setting person is acquired and saved. It will be appreciated that in this embodiment, the display switch setting information of the setting person may be acquired in advance without considering whether the set image output condition is met. For example, the APP collects the face image of the third person, and sets the display switch corresponding to the third person to be in the off state.

It will be appreciated that the display switch status of each identified person may be flexibly set as desired. In some exemplary embodiments, the settings may be set by APP software, or by a television remote control, the specific setting scheme not being discussed in detail herein.

For example, the person collects the face image of the king milk, and executes step 221, the king milk sets the display switch corresponding to the king milk to be in the off state through the remote controller, that is, the king milk considers that the person is watching television, the image data does not need to be displayed through the television, and after the image data is stored, the display switch corresponding to the king milk is in the off state. Accordingly, step 230 is executed, where the identification result includes only the king milk, and the display switch corresponding to the king milk is turned off, so as to determine that the set image output condition is not met.

In some exemplary embodiments, the method further comprises:

and under the condition that the personnel acquisition information accords with the set image output condition, saving the personnel characteristic value of the identified personnel.

In some exemplary embodiments, the departure status of the saved person feature value is marked as "not left" in the case that the person acquisition information meets the set image output condition.

In some exemplary embodiments, the method further comprises:

and under the condition that the personnel acquisition information accords with the set image output condition, adding the personnel characteristic value of the identified personnel into the multicast routing table entry.

In some exemplary embodiments, the leaving status of the personnel feature value added in the multicast routing table entry is marked as "not leaving" in case the personnel acquisition information meets a set image output condition.

In some exemplary embodiments, the person feature value is a face feature vector, and the identified face feature vector is added to the multicast routing table entry (x, G) accordingly.

In some exemplary embodiments, the initial multicast routing table entry (G) of the set-top box does not include a face feature vector.

In some exemplary embodiments, in the case that the multicast routing table entry (x, G) does not include a face feature vector, the set top box does not send a PIM join message to the other portal (x, G).

In some exemplary embodiments, the image data source is a video acquisition system, such as an intelligent video doorbell; or, an indoor and outdoor monitoring system. In some exemplary embodiments, the image data source is a video data receiving, transmitting and/or playing system, such as a video telephony system; or small video, etc. In some exemplary embodiments, the image data source is various types of application systems having user-operated display interfaces. And is not limited in particular aspects.

In some exemplary embodiments, as shown in fig. 3, the method further comprises:

step 240, acquiring personnel tracking information;

and step 260, stopping outputting the image data to the display device when the set stopping image output condition is determined to be met according to the personnel tracking information.

In some exemplary embodiments, the camera has a person tracking function that generates person tracking information when the tracking detects that a person leaves the screen. Wherein, the personnel image is the personnel image cached in step 210.

In some exemplary embodiments, the camera has only a shooting function, and the set-top box or other device performs a person tracking function, generating person tracking information when the tracking detects that a person leaves the screen. Wherein, the personnel image is the personnel image cached in step 210 correspondingly.

In some exemplary embodiments, the method further comprises:

step 250, marking the leaving state of the personnel characteristic value corresponding to the personnel acquisition information in the personnel tracking information as leaving according to the personnel tracking information; or deleting the personnel characteristic value corresponding to the personnel acquisition information in the saved personnel tracking information according to the personnel tracking information.

In some exemplary embodiments, in step 230, when it is determined that the person identification result includes at least one person of a setting type, and the display switch state of at least one person of the setting type is null or on, the image data is output to the display device, and in step 260, when it is determined that all the persons leave, the output of the image data to the display device is stopped.

For example, when the set type is old people, when at least one old people appears in front of the television and the display switch is on or empty, the image data of the intelligent visual doorbell is output to the television for display, and when all the old people leave, the output of the image data of the intelligent visual doorbell to the television is stopped.

In some exemplary embodiments, in step 230, when it is determined that the person identification result includes at least one setting person, and the display switch state of at least one setting person is null or on, the image data is output to the display device, and in step 260, when it is determined that all the persons leave, the output of the image data to the display device is stopped.

For example, the person is set as the user a, when the user a appears in front of the television and the display switch is turned on, the image data of the intelligent visual doorbell is output to the television for display, and when the user a leaves, the output of the image data of the intelligent visual doorbell to the television is stopped. For another example, if the user is the user A, B, C and the user a and the user B are both present in front of the television and the display switch is empty, the image data of the smart video doorbell is outputted to the television to be displayed, and if the user a is away, the image data of the smart video doorbell is not stopped from being outputted to the television because the user B is not away, and if the user B is away, the image data of the smart video doorbell is stopped from being outputted to the television. For another example, user B is not a setup person, and when user B is present in front of the television, the image data of the smart video doorbell is not output to the television for display, and further, when user B leaves, stopping is not required.

In some exemplary embodiments, according to the person tracking information, determining that the person with the display switch in the at least one setting type is empty or the person with the display switch in the on state is all away includes:

and determining whether all people with the display switch in the at least one set type of people in the empty state or the display switch in the on state leave according to the leaving state of the personnel characteristic value corresponding to the personnel acquisition information in the tracking information.

In some exemplary embodiments, according to the personnel tracking information, determining that the display switch is empty or the personnel with the display switch in the on state is all left in the at least one setting personnel includes:

and determining whether all people leaving in which the display switch is empty or the display switch is in an on state in the at least one set person leave or not according to the leaving state of the personnel characteristic value corresponding to the personnel acquisition information in the tracking information.

In some exemplary embodiments, the stopping outputting the image data to the display device includes:

and deleting the video port connected with the television from an outgoing interface list of the (+G) table item of the set top box.

It will be appreciated that the set-top box no longer superimposes the image data decoding after deletion onto the television program of the television.

The embodiment of the disclosure also provides an image output control method, which is applied to a set top box, an application scene is shown in fig. 4, wherein the display device is a television, the image data source is an intelligent visual doorbell, the image data is a video stream of the intelligent visual doorbell, and the acquisition device for personnel acquisition information comprises a network camera 1, and the set top box, the intelligent visual doorbell and the network camera 1 are positioned in the same local area network; presetting a corresponding relation (binding) between the network camera 1 and the television 1; the set top box is connected with the television 1 through the video port 1. The personnel acquisition information is face information, and the personnel of a set type are old people; the personnel identification result corresponding to the personnel acquisition information comprises at least one old man, and the condition that the switch display state of the old man is empty accords with the set image output condition; when all the old people leave, the preset stop image output condition is met.

As shown in fig. 5, the method includes:

step 510, the set top box receives the face image sent by the network camera 1;

step 520, determining that the display device corresponding to the network camera 1 is the television 1 according to the preset correspondence;

Step 530, when the face image meets the set image output condition, acquiring a video stream from the intelligent visual doorbell and outputting the video stream to the television 1 for display.

It is understood that none of the identified elderly people have a display switch status set.

In some exemplary embodiments, the person recognition result corresponding to the face image includes at least one person of the set type, and in case that the display switch status is empty, it is determined in step 530 that the set image output condition is met. Wherein the set type is old people.

The network camera 1 has functions of face detection and face age recognition, and the network camera 1 detects and recognizes faces appearing in a screen, and when determining that an old man appears in the screen (the display switch state is empty), sends a face image to the set top box.

In some exemplary embodiments, the network camera 1 transmits a face image to the set-top box only once for the same face to reduce the processing load of the set-top box.

In some exemplary embodiments, the webcam saves the face image.

In some exemplary embodiments, step 530 further comprises:

under the condition that the face image accords with the set image output condition, the set top box periodically sends PIM adding information (G) to other network ports in the local area network;

The video port connected to the television 1 is added to the output port list of the multicast routing table entry (G) of the set-top box.

In some exemplary embodiments, step 530 further comprises:

under the condition that the face image accords with the set image output condition, adding the face feature vector corresponding to the face image into a multicast routing table item (G) of the set top box;

the departure status of the face feature vector is marked as "not left".

In some exemplary embodiments, the capturing the video stream from the smart visual doorbell and outputting the video stream to the television 1 for display in step 530 includes:

the set top box receives a video stream coded by H.264 or H.265 of a multicast group G as a destination address from the intelligent visual doorbell;

and the set top box sends the video stream out of the outlet port list of the (G) table item.

Under the condition that the output port in the output port list of the (G) table entry includes a video port connected to the television 1, decoding the video stream, and then, overlapping the decoded image into the decoded television program image, and sending out the decoded image, so that the effect of picture-in-picture is presented on the picture of the television 1.

In some exemplary embodiments, the webcam is a clip-on webcam, and is mounted on the television 1.

In some exemplary embodiments, the television 1 is placed in a living room.

It can be seen that in some exemplary embodiments, when the pacifier sits in front of the television in the living room to watch the television, the network camera determines that a person of a set type (the old) is detected according to the face detection and recognition function of the network camera, and then sends a face image of the pacifier to the set top box; the set top box acquires a face image, determines that the face image meets the set image output condition, periodically transmits (G) PIM (PIM) adding information to other network ports in the local area network, and adds the video port 1 connected with the television 1 into an output port list of a multicast routing table item (G) of the set top box. When someone presses the doorbell outside the door, the video camera carried by the doorbell sends the video stream to the home network, and the destination address is the multicast group address G. Correspondingly, the set top box receives the video stream, matches with the self multicast routing table entry (G), directly forwards the video stream for the network port in the outlet interface list in the G table entry, decodes the video stream for the video port 1 in the outlet interface list, and superimposes the decoded image in a picture-in-picture mode into the image of the television set in the living room of the same outlet port, so that the king milk can see the scene outside the door on the television set.

In some exemplary embodiments, the method further comprises:

step 540, acquiring personnel tracking information;

and 550, stopping outputting the video stream from the intelligent visual doorbell to the television 1 for display under the condition that all the identified old people leave the picture according to the personnel tracking information.

It can be seen that in some exemplary embodiments, when the king milk leaves the television 1, the video stream is no longer displayed by the television 1 when the television is not being viewed in front of the television 1.

In some exemplary embodiments, step 540 includes:

when the network camera detects that the old people leave in the picture, personnel tracking information is sent to the set top box; the person tracking information includes: face images and departure indication information;

the set top box updates (G) the leaving state of the face feature vector in the table item to leave according to the personnel tracking information; or the set top box deletes the corresponding face feature vector in the table item (G) according to the personnel tracking information.

In some exemplary embodiments, step 550 includes: and deleting the video port 1 connected with the television 1 from the output port list of the multicast routing table item (G) of the set top box under the condition that all the identified old people leave the picture is determined. It will be appreciated that after video port 1 is deleted from the output port list, the set top box will not be sending the received multicast video stream to video port 1.

In some exemplary embodiments, step 550 includes: and stopping periodically sending (G) PIM joining messages (Join messages) to other network ports in the local area network when all the identified old people leave the picture. It can be appreciated that after the PIM join message is stopped, after the multicast routing forwarding table entry (x, G) on the switch is aged, the set-top box will no longer receive the multicast video stream with destination address G.

It will be appreciated that when the king milk leaves the television 1, it is no longer necessary to display the video stream from the doorbell on the television 1. If the king milk continues to watch television in front of the television set 1, it is still displayed on the television set 1 when the multicast video stream from the doorbell is received.

The embodiment of the disclosure also provides an image output control method, which is applied to a set top box, an application scene is shown in fig. 6, wherein the display device is a television, the image data source is an intelligent visual doorbell, the image data is a video stream of the intelligent visual doorbell, and the acquisition device for personnel acquisition information comprises a network camera 1 and a network camera 2, and the set top box, the intelligent visual doorbell, the network camera 1 and the network camera 2 are positioned in the same local area network; presetting a corresponding (binding) relation: the network camera 1 corresponds to the television 1 (living room), and the network camera 2 corresponds to the television 2 (main sleeping); the set top box is connected with the television 1 through the video port 1; the set top box is connected with the television 2 through the video port 2; the multicast routing table item (G) of the set-top box initially does not contain a face feature vector, and its outgoing interface list is empty. The personnel acquisition information is face information, and the personnel of a set type are old people; the personnel identification result corresponding to the personnel acquisition information comprises at least one old man, and the display switches of all the identified old man are in a closed state, so that the preset image output conditions are met; and if all the identified old people have left the display switch in the empty state or the display switch in the on state, the preset stop image output condition is met.

As shown in fig. 7, the method includes:

step 710, the set top box receives the face image sent by the network camera 2;

step 720, determining that the display device corresponding to the network camera 2 is the television 2 according to the preset corresponding relation;

step 730, adding the face feature vector of the identified person with the set type into the multicast routing table item (x, G) of the set top box and marking the leaving state of the face feature vector as "not leaving" when the person identification result corresponding to the face information is the set type;

step 740, obtaining the video stream from the intelligent visual doorbell and outputting the video stream to the television 2 for display.

For example, if the king milk and the grandpa watch on the main recumbent watch television, the face image received by the set-top box comes from the web camera 2, and the display device for the king milk and the grandpa is determined to be the television 2. And correspondingly adding the facial feature vectors of the king milk and the grandma into a multicast routing table item (G) of the set top box, wherein the leaving state of the facial feature vectors of the king milk and the grandma is 'not leaving' when the facial feature vectors are added.

In some exemplary embodiments, the method further comprises:

step 750, the display switch state setting information of the identified person is obtained and stored.

For example, the display switch of the vintage is set to the off state.

In some exemplary embodiments, as in fig. 8, the method further comprises:

step 760, determining that the set image output conditions are met when the person identification result corresponding to the face information is old people and not all the identified old people have their display switches in the off state;

step 770, obtaining the video stream from the intelligent visual doorbell and outputting the video stream to the television set 2 for display.

For example, the king milk is not provided with a display switch, and the vintage has set the display switch to an off state.

Step 740 or 770 includes:

when a person sitting in front of the television 2 accords with the set image output conditions, and when the intelligent visual doorbell outputs video streams and multicasts, the set top box acquires the video streams and outputs the video streams to the television 2 for display;

when the king milk and the grandma both sit in front of the television 2 and accord with the set image output conditions, when the intelligent visual doorbell has video stream output and multicast, the set top box acquires the video stream and outputs the video stream to the television 2 for display;

when the grandpa person sits in front of the television 2 and does not meet the set image output conditions, the set top box does not need to acquire the video stream or output the video stream to the television 2 for display when the intelligent visual doorbell outputs the video stream and multicasts the video stream.

In some exemplary embodiments, the method further comprises:

step 780, acquiring personnel tracking information;

and step 790, stopping outputting the video stream from the intelligent visual doorbell to the television 2 for display under the condition that the set stop image output condition is determined to be met according to the personnel tracking information.

Step 790 comprises:

when a person sitting in front of the television 2 leaves, the person meets the set image output stopping condition, and the video stream from the intelligent visual doorbell is stopped from being continuously output to the television 2 for display;

when the king milk and the grandma are both seated in front of the television 2, if the king milk leaves, the set condition for stopping image output is met, and the video stream from the intelligent visual doorbell is stopped from being continuously output to the television 2 for display;

when the king milk and the grandma are both seated in front of the television 2, if the grandma leaves, the set stop image output condition is not met, and the video stream from the intelligent visual doorbell is continuously output to the television 2 for display.

An embodiment of the present disclosure provides an image output control system, as shown in fig. 9, including:

a display device 910, an image data source device 930, and an output control device 940;

the output control device 940 is configured to join a multicast group where the image data source device 930 is located, and receive the multicast image data;

the output control device 940 is further configured to add a video port connected to the display device 910 to an egress port list of a multicast routing table entry; and according to the outlet port list of the multicast routing table item, the image data is decoded and then sent to a video port of the display device 910 for display.

In some exemplary embodiments, the output control device 940 is further configured to, after decoding the image data, superimpose the decoded image data on a video stream decoded image of the video port of the display device 910, and send the decoded image data to the video port of the display device 910 for display.

In some exemplary embodiments, the display device 910 is a television.

In some exemplary embodiments, the output control device 940 is a set top box.

In some exemplary embodiments, the image data source device 930 is a smart video doorbell.

In some exemplary embodiments, the system further comprises a personnel information gathering device 920.

In some exemplary embodiments, the personnel information gathering device 920 is a camera.

An embodiment of the present disclosure provides an image output control system, as shown in fig. 10, including:

a display device 1010, a person information acquisition device 1020, an image data source device 1030, and an output control device 1040;

the personnel information acquisition device 1020 is configured to acquire personnel acquisition information;

the output control device 1040 is configured to determine a corresponding display device 1010 according to the person acquisition information; in the case that the person acquisition information meets the set image output condition, image data is acquired from the image data source device and output to the display device 1010 for display.

In some exemplary embodiments, the display device 1010, the personnel information collection device 1020, the image data source device 1030, and the output control device 1040 are located on the same local area network.

In some exemplary embodiments, the display device 1010 is a television;

the personnel information acquisition device 1020 is a camera;

the output control device 1040 is a set top box.

In some exemplary embodiments, the image data source device 1030 is a smart video doorbell.

In some exemplary embodiments, the set-top box includes at least one video port for connecting to a television.

In some exemplary embodiments, the set top box includes at least two portals.

In some exemplary embodiments, the system includes one or more display devices.

In some exemplary embodiments, the system includes one or more personnel information gathering devices.

In some exemplary embodiments, the display devices and the personnel information acquisition devices in the system are in one-to-one correspondence.

In some exemplary embodiments, one of the set-top boxes is connected to one or more televisions.

The embodiment of the disclosure also provides an electronic device, including:

one or more processors;

storage means for storing one or more programs,

The embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a computer program that is executed by a processor to implement the image output control method according to any of the embodiments of the present disclosure.

It can be seen that, based on the image output control scheme provided by the embodiment of the disclosure, the image video is obtained in a multicast mode and is output to the video port of the display device after being decoded, so that the existing more preferable display device in the local area network environment can be fully utilized, a more convenient watching mode is provided for a user, and the user experience of related images or video applications is greatly improved. In some embodiments, according to the acquired information of the viewer, a corresponding display device is determined, and the image data is newly added or transferred to be output to the display device, so that the intelligent degree of the image data output scheme is improved. In some embodiments, the existing set top box and television in the home network system are utilized to newly add or transfer other image data sources possibly inconvenient to watch to the television more convenient to watch for display according to the condition that the user watches the television, so that the intelligent degree of the home system is further improved.

Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the invention, and all equivalent structural changes made by the specification and drawings of the present invention or direct/indirect application in other related technical fields are included in the scope of the present invention.

Claims

1. An image output control method, characterized by comprising:

according to the personnel acquisition information, determining display equipment watched by the watched personnel;

under the condition that the personnel acquisition information accords with the set image output condition, adding the video port of the display equipment into an outlet port list of a multicast routing table item;

according to the outlet port list of the multicast routing table item, the image data is decoded and then sent to a video port of the display equipment for display;

wherein the display device includes: a television set; the personnel acquisition information is the acquisition information of the watching personnel.

2. The method of claim 1, wherein,

after decoding the image data, sending the decoded image data to a video port of the display device for display, including:

3. The method of claim 1, wherein the determining the display device viewed by the viewer based on the person-gathering information comprises:

searching a preset corresponding relation between the source acquisition equipment and the display equipment according to the source acquisition equipment for acquiring information of the person, and determining the display equipment watched by the watched person.

4. The method of claim 1, wherein,

before receiving the multicast image data of the multicast group where the image data source is located, the method further includes: adding the multicast group where the image data source is;

wherein the joining the multicast group where the image data source is located includes:

5. The method of claim 4, wherein,

the joining the multicast group where the image data source is located further includes:

6. The method of claim 1 or 4, wherein,

determining that the set image output conditions are met under the condition that the personnel identification result corresponding to the personnel acquisition information comprises at least one personnel of a set type and the display switch state of at least one personnel of the set type is empty or on;

or,

and determining that the set image output conditions are met under the condition that at least one set person is included in the person identification result corresponding to the person acquisition information and the display switch state of at least one set person is empty or on.

7. The method of any one of claim 1 to 5,

the method further comprises the steps of: acquiring personnel tracking information;

under the condition that the preset stop image output condition is met according to the personnel tracking information, deleting the video port of the display device from the outlet port list of the multicast routing table item;

the personnel tracking information is the tracking information of the watching personnel.

8. An image output control system, comprising:

a display device, an image data source device, and an output control device;

the output control equipment is set to determine the display equipment watched by the watched person according to the personnel acquisition information; receiving multicast image data of a multicast group in which the image data source equipment is positioned;

the output control device is further configured to add the video port of the display device to an output port list of a multicast routing table item when the personnel acquisition information meets a set image output condition; according to the outlet port list of the multicast routing table item, the image data is decoded and then sent to a video port of the display equipment for display;

9. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the image output control method of any of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, which when executed by a processor implements the image output control method according to any one of claims 1 to 7.