WO2022262134A1 - Image display method, apparatus and device, and storage medium - Google Patents

Abstract

Disclosed in the present application are an image display method, apparatus and device, and a storage medium. In the present application, a target image display mode is determined by means of a conference request instruction that is input by a user, and different lens directions are adjusted according to different target image display modes, so as to be suitable for image presentation in a plurality of scenarios. An appropriate target image display strategy is selected according to a combination of the lens directions and the current conference participation target information; and different image display manners can be selected according to different pieces of conference participation target information, thereby avoiding the problem of a product only being able to be used in a single application scenario during video conferencing, and different display manners being unable to be selected and used according to the number of conference participants and conference modes.

Description

Image display method, device, equipment and storage medium

This application claims priority to a Chinese patent application with application number 202111047995.3 filed on September 7, 2021, the entire contents of which are hereby incorporated by reference.

technical field

The present application relates to the technical field of video display, and in particular to an image display method, device, equipment and storage medium.

Background technique

With the development of science and technology, telecommuting and teleconferencing are becoming more and more popular, and communication has surpassed the limitations of time and space. People have more and more functional requirements for conference communication products, and the performance requirements of products are also getting higher and higher. Therefore, many audio and video conference office products have been born, including fixed-focus video products, auto-focus video products, and rotating pan-tilt video products. Often these products are relatively simple, and they are just a shooting device. When conducting a video session, it can only be used in a single application scenario, and different display methods cannot be selected according to the number of participants and the conference mode, and the user experience is poor.

The above content is only used to assist in understanding the technical solution of the present application, and does not mean that the above content is admitted as prior art.

technical problem

The main purpose of this application is to provide an image display method, device, device, and storage medium, aiming to solve the problem that the existing technology can only be used in a single application scenario when conducting a video conversation, and cannot be used according to the number of participants and the meeting Mode selection uses different display methods, technical issues with poor user experience.

technical solution

In order to achieve the above object, the application provides an image display method, the method comprising the following steps:

Receiving the meeting request instruction, determining the target image display mode according to the meeting request instruction, and obtaining the current meeting target information;

determining the lens orientation according to the target image display mode, and adjusting the lens orientation according to the lens orientation;

determining a target image display strategy based on the lens orientation and the current meeting target information;

In response to the camera turning to the camera orientation, the preset private image or the current target image captured by the camera is displayed according to the target image display strategy.

Optionally, the target image display strategy includes a target image panorama display strategy;

Responding to the fact that the camera orientation is facing upwards, obtain the number of participants and the degree of participation in the meeting in the information about the current meeting participants;

Determine a target image panorama display strategy according to the lens orientation, the number of participants in the meeting, and the degree of participation in the meeting;

In response to the lens turning to the lens orientation, displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy, further comprising:

In response to the camera turning to the camera orientation, the current target image acquired by the camera is displayed according to the target image panorama display strategy.

Optionally, the target image display strategy includes a target image wide-angle display strategy;

In response to the camera orientation being forward facing, determine a target image wide-angle display strategy according to the lens orientation and the current conference participant information;

In response to the lens turning to the lens orientation, displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy, further comprising:

In response to the lens turning to the lens orientation, the current target image acquired by the camera is displayed according to the target image wide-angle display strategy.

Optionally, the target image display policy includes a target image privacy display policy;

Responsive to the camera orientation being downward, determining a target image privacy display policy according to the lens orientation;

In response to the lens turning to the lens orientation, displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy, further comprising:

In response to the camera turning to the camera orientation, displaying a preset privacy image according to the target image privacy display policy, and turning off the microphone.

Optionally, before displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy in response to the lens orientation turning to the lens orientation, the method includes:

Obtain the image of the current participant target through the camera, and turn on the microphone;

Segmenting the image of the current participant target through a preset image segmentation model to obtain a segmented image;

Perform image processing on the segmented image to obtain the current target image.

Optionally, performing image segmentation on the current conference target image through a preset image segmentation model to obtain a segmented image includes:

Determining the initial segmentation point of the current participant target image based on the camera orientation;

Segmenting the current meeting target image by using a preset image segmentation model based on the initial segmentation point and the preset direction to obtain a segmented image.

Optionally, after displaying the preset privacy image or the current target image captured by the camera according to the target image display strategy in response to the lens orientation turning to the lens orientation, the method further includes:

performing mouth shape detection on the current target image through a preset mouth shape detection model to obtain an initial speaker image;

Obtaining a current sound signal through a microphone, and determining speaker information according to the current sound signal;

Determine a target speaker image according to the speaker information and the initial speaker image, and mark the target speaker image.

In addition, in order to achieve the above purpose, the present application also proposes an image display device, the image display device includes:

The instruction receiving module is configured to receive the meeting request instruction, determine the target image display mode according to the meeting request instruction, and acquire the current meeting target information;

The lens adjustment module is configured to determine the orientation of the lens according to the target image display mode, and adjust the orientation of the lens according to the orientation of the lens;

A strategy confirmation module, configured to determine a target image display strategy based on the camera orientation and the current meeting target information;

The image display module is configured to display a preset private image or a current target image captured by a camera according to the target image display policy in response to the lens turning to the lens orientation.

In addition, in order to achieve the above purpose, the present application also proposes an image display device, which includes: a memory, a processor, and an image display program stored in the memory and operable on the processor. The above-mentioned image display program is configured to realize the steps of the above-mentioned image display method.

In addition, in order to achieve the above purpose, the present application also proposes a storage medium, on which an image display program is stored, and when the image display program is executed by a processor, the steps of the above-mentioned image display method are realized.

Beneficial effect

The present application determines the target image display mode according to the conference request command when receiving the conference request command, and obtains the current meeting target information, determines the lens orientation according to the target image display mode, and adjusts the lens according to the lens orientation Orientation: Determine the target image display strategy based on the camera orientation and the current meeting target information. When the lens orientation is rotated to the lens orientation, the preset privacy image or the current target captured by the camera will be displayed according to the target image display strategy. image for display. Compared with the prior art, this application determines the target image display mode through the meeting request command input by the user, and adjusts different lens orientations according to different target image display modes, and is suitable for image display in multiple scenarios. According to the lens orientation and the current Combining the target information of the participants to select the appropriate target image display strategy, it is possible to select different image display methods according to the target information of the participants, avoiding that the video session can only be used in a single application scenario, and cannot be used according to the participants. The number of people, the conference mode selection uses different display methods, and technical issues such as poor user experience.

Description of drawings

FIG. 1 is a schematic structural diagram of an image display device in a hardware operating environment involved in an embodiment of the present application;

FIG. 2 is a schematic flow chart of the first embodiment of the image display method of the present application;

FIG. 3 is a schematic diagram of a video conferencing device in an embodiment of an image display method of the present application;

FIG. 4 is a schematic flow chart of the second embodiment of the image display method of the present application;

FIG. 5 is a schematic diagram of a VIP image display strategy of an embodiment of the image display method of the present application;

FIG. 6 is a schematic diagram of a moderator image display strategy in an embodiment of the image display method of the present application;

FIG. 7 is a schematic diagram of a multi-person conversation image display strategy in an embodiment of the image display method of the present application;

FIG. 8 is a schematic diagram of an alternate speech image display strategy in an embodiment of the image display method of the present application;

FIG. 9 is a schematic flowchart of a third embodiment of the image display method of the present application;

FIG. 10 is a schematic diagram of a wide-angle mode scene of an embodiment of the image display method of the present application;

FIG. 11 is a schematic diagram of a target image wide-angle display strategy in an embodiment of the image display method of the present application;

FIG. 12 is a schematic flowchart of a fourth embodiment of the image display method of the present application;

FIG. 13 is a structural block diagram of the first embodiment of the image display device of the present application.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Embodiments of the present invention

It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

Referring to FIG. 1 , FIG. 1 is a schematic structural diagram of an image display device in a hardware operating environment involved in the solution of the embodiment of the present application.

As shown in FIG. 1 , the image display device may include: a processor 1001 , such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Wherein, the communication bus 1002 is configured to realize connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). Memory 1005 can be a high-speed random access memory (Random Access Memory, RAM), can also be a stable non-volatile memory (Non-Volatile Memory, NVM), such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation on the image display device, and may include more or less components than those shown in the illustration, or combine some components, or arrange different components.

As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image display program.

In the image display device shown in Figure 1, the network interface 1004 is mainly configured to communicate data with the network server; the user interface 1003 is mainly configured to perform data interaction with the user; the processor 1001, memory 1005 may be set in the image display device, and the image display device calls the image display program stored in the memory 1005 through the processor 1001, and executes the image display method provided in the embodiment of the present application.

An embodiment of the present application provides an image display method. Referring to FIG. 2 , FIG. 2 is a schematic flowchart of a first embodiment of an image display method of the present application.

In this embodiment, the image display method includes the following steps:

Step S10: receiving a meeting request instruction, determining a target image display mode according to the meeting request instruction, and acquiring current meeting target information.

It should be noted that the execution subject of this embodiment may be an image display device, wherein the image display device may be a controller of a video conferencing device, such as a personal computer, a control chip, etc., or other devices capable of video conferencing. device, which is not specifically limited in this embodiment.

It can be understood that the meeting request command may be a request command input by the user through the button of the video conference device, the remote control or the mobile APP to control the operation of the video conference device, and the meeting request command may include the start of the conference initial mode signal, in this embodiment, for the selection of the video conferencing device, refer to FIG. 3 , and describe the video conferencing device shown in FIG. 3 as an example.

It should be noted that, in this embodiment, the video conferencing device has five modules: 1. Microphone array module; 2. Motor and drive module; 3. Lens module; 4. Sensor module; 5. Controller; The microphone array module is configured to collect the sound signal of the user during the video conference, and send the sound signal to the controller to detect the sound direction. In order to make the collected sound signal more complete and clear, the microphone array can It is a multi-microphone array, for example: 4-microphone array, 6-microphone array, and 8-microphone array, etc., which is not specifically limited in this embodiment.

Wherein, the motor and the driving module are configured to rotate according to the signal sent by the controller, so that the lens can be rotated to realize the adjustment of the lens orientation. During the rotation of the motor and the driving module, the direction of rotation is not limited, that is, in When adjusting the orientation of the lens, the motor and the driving module may rotate clockwise or counterclockwise, which is not specifically limited in this embodiment.

In this embodiment, the selection of the lens in the lens module can be an image acquisition lens that can realize image acquisition at an angle of 220°, or other lenses that can achieve the same or similar functions; the sensor module is configured to detect motors and drive module control The lens is rotated, and the orientation of the lens is detected to complete the rotation operation.

It should be noted that the target image display mode may be panorama mode, wide-angle mode, privacy mode, etc., and according to the conference request instruction input by the user, the video conferencing device controller may activate the corresponding target image display mode.

It is understandable that the current conference participation target information may be information such as the number of people currently participating in the conference, face information, and conference participation.

Step S20: Determine the orientation of the lens according to the target image display mode, and adjust the orientation of the lens according to the orientation of the lens.

It should be noted that different target image display modes correspond to different lens orientations. In this embodiment, refer to FIG. 3: 1 indicates a panoramic mode, and the lens orientation corresponding to the panoramic mode is upward; 2 indicates a wide-angle mode , the lens orientation corresponding to the wide-angle mode is facing forward; 3 indicates the privacy mode, and the lens orientation corresponding to the privacy mode is facing downward.

In a specific implementation, after the lens orientation is determined according to the target image display mode, the lens is controlled to rotate according to the motor and the driving module, so that the lens orientation is rotated to the lens orientation corresponding to the target image display mode determined according to the conference request input by the user.

Step S30: Determine a target image display strategy based on the camera orientation and the current meeting target information.

It should be noted that, in this embodiment, the image display strategy may be the multi-person conversation image display strategy in the panorama mode, the host image display strategy, the VIP image display strategy, and the alternate speech image display strategy, etc., and may also be the wide-angle mode In this embodiment, the specific target image display strategy is determined according to the camera orientation and the current meeting target information.

It can be understood that before determining the target image display strategy, the lens orientation is determined by the image display mode. Therefore, in this embodiment, according to three different target image display modes, the lens of the video conferencing device has three orientations, which are: lens Up, Camera Forward, and Camera Down.

Step S40: In response to the camera turning to the camera orientation, display the preset private image or the current target image captured by the camera according to the target image display policy.

It should be noted that the current target image may be a conference image captured by a camera when the video conference device is in a panorama mode or a wide-angle mode.

Further, in order to make the image collected by the camera more suitable for display, before step S40, it also includes:

Obtain the image of the current participant target through the camera, and turn on the microphone;

Segmenting the image of the current participant target through a preset image segmentation model to obtain a segmented image;

Perform image processing on the segmented image to obtain the current target image.

It should be noted that the preset image segmentation model can be configured to perform image segmentation on the target image captured by the camera, extract the person image in the target image, and mark it as a segmented image.

It can be understood that the current target image is an expanded view obtained after expanding the target image collected by the camera. During the expansion process, the expansion point needs to be determined first. In this embodiment, the determination of the expansion point position can be Determining the segmentation initial point of the current conference target image based on the camera orientation; segmenting the current conference target image through a preset image segmentation model based on the segmentation initial point and preset direction to obtain a segmented image.

In the specific implementation, in the process of segmenting the current meeting target image collected by the camera, it is also necessary to detect the current meeting target image according to the preset face detection model and human figure detection model, so as to avoid the complete The person is cut in half, resulting in incompleteness of the target image.

In addition, after displaying the preset privacy image or the current target image captured by the camera according to the target image display policy, the participants who speak during the video conference can also be marked, so that the user can more intuitively understand who is Speaking, can have a better experience.

Further, in order to mark the participant target of speaking, after step S40, it also includes:

performing mouth shape detection on the current target image through a preset mouth shape detection model to obtain an initial speaker image;

Obtaining a current sound signal through a microphone, and determining speaker information according to the current sound signal;

Determine a target speaker image according to the speaker information and the initial speaker image, and mark the target speaker image.

It should be noted that the preset mouth shape detection model is configured to detect the mouth movement of the participant in the image of the current participant. When the participant speaks, the mouth will move, thereby judging the participant who is speaking At the same time, since the target may not emit a sound signal, but detects the presence of mouth movement, the initial speaker information can be obtained through the preset mouth detection model.

It can be understood that the accurate target speaker can be determined by using the sound signal obtained by the microphone to predict the direction through the preset direction prediction model and the initial speaker information, so as to mark the image of the target speaker.

It is worth noting that the marking of the target speaker can be by increasing the brightness of the image of the target speaker in the display image, marking different colors, etc., or by other marking methods with the same or similar marking functions, which are not covered in this embodiment. Specific restrictions.

In this embodiment, when a meeting request instruction is received, the target image display mode is determined according to the meeting request instruction, and the current meeting target information is obtained, the lens orientation is determined according to the target image display mode, and the camera orientation is adjusted according to the lens orientation. Camera orientation, based on the lens orientation and the current meeting target information to determine the target image display strategy, when the lens orientation is rotated to the lens orientation, according to the target image display strategy, the preset privacy image or the current image captured by the camera The target image is displayed. In this embodiment, the target image display mode is determined through the meeting request command input by the user, and different camera orientations are adjusted according to different target image display modes. Select the appropriate target image display strategy, and may choose different image display methods according to the different target information of the participants, avoiding that it can only be used in a single application scenario during a video session, and cannot be used according to the number of participants and the conference mode Choose to use different display methods, technical issues with poor user experience.

Referring to FIG. 4 , FIG. 4 is a schematic flowchart of a second embodiment of an image display method of the present application.

Based on the first embodiment above, in this embodiment, the step S30 includes:

Step S301: Responding to the fact that the camera orientation is upward, acquire the number of conference participants and the degree of conference participation in the current conference participant information.

It is worth noting that the degree of meeting participation can be calculated by judging the total number of sound signals collected by the microphone array of the video conferencing equipment, the source of the sound signals, etc., and then processing the sound signals to obtain the degree of meeting participation can be the sound signal The greater the total number of , the higher the degree of conference participation, and the greater the number of sources of sound signals, the higher the degree of conference participation.

Step S302: Determine a target image panorama display strategy according to the camera orientation, the number of conference participants and the conference participation degree.

It should be noted that when the camera is facing upwards, the target image panorama display strategy includes four target image display strategies: 1. VIP image display strategy; 2. Moderator image display strategy; 3. Multi-person conversation image display strategy; 4. , Alternate speech image display strategy; it is necessary to select an appropriate target image display strategy according to the specific number of participants and the degree of participation in the meeting.

Further, due to the different number of meeting participants and different meeting participation degrees, when selecting an image display strategy, it is necessary to select a different image display strategy, that is, step S301, including:

When the camera orientation is facing upward, obtain the number of participants and the degree of participation in the meeting in the current meeting target information;

Judging whether there is host information in the current meeting target information;

In response to host information being present in the current conference participation target information, comparing the conference participation degree with a first preset value, and comparing the number of conference participation targets with a second preset value;

When the degree of meeting participation exceeds a first preset value, and when the number of meeting participants is greater than a second preset value, determine according to the meeting request instruction, the number of meeting participants and the meeting participation degree VIP image display strategy;

When the degree of meeting participation does not exceed a first preset value, and when the number of meeting participants is less than a second preset value, according to the meeting request instruction, the number of meeting participants and the meeting participation degree Determine moderator image display strategy;

It is understandable that the moderator information may be a long-term display target image, and the moderator information may be the conference moderator, the organizer, or a person who needs to be permanently displayed, or the person directly in front of the video conferencing device As the default moderator, in actual operation, the moderator information can be manually switched by adjusting the placement of the video conferencing equipment, or can be switched through the buttons on the video conference equipment, remote control, mobile APP, etc. This implementation Examples are not specifically limited.

Secondly, when there is moderator information, different image display strategies are selected according to the degree of meeting participation and the number of participants. In actual operation, when the degree of meeting participation exceeds the first preset value, and when the participation target If the number is greater than the second preset value, the target image display strategy is the VIP image display strategy; when the conference participation does not exceed the first preset value, and when the number of participants is less than the second preset value, the target image The display strategy is the moderator's image display strategy, and the first preset value and the second preset value may be a value preset by the user, which is not specifically limited in this embodiment.

It is worth noting that in the VIP image display strategy, a VIP position will be set. Generally, the VIP position will be fixed in the upper left corner, and other numbers and positions may not be fixed. In this embodiment, the number of non-VIP positions is 5 , when it is detected that someone is speaking, the target image corresponding to the speaking participant target is extracted to several other positions through detection, and will be clockwise in the five frames of the non-VIP position starting from the VIP position.

It should be understood that in the moderator's image display strategy, all acquired target images of participants will be integrated to obtain a panorama containing all images of target participants, and the panorama will be fixed at the lower end of the display image. And fix the moderator image on the top, there may be a situation of multiple moderator information, so multiple moderator images can be placed on the top of the display image.

In addition, if the degree of participation in the meeting and the number of participants change during the video conference, it can also be detected through the preset humanoid detection model and the preset face detection model, and then re-according to different meeting participation and the number of participants Change target image display strategy.

In a specific implementation, there are currently six people A, B, C, D, E, and F. When the video conferencing device detects the information of the host, it confirms that A is the host. When the degree of participation in the meeting exceeds the first preset value, And when the number of participants is greater than the second preset value, the target image display strategy is the VIP image display strategy. Therefore, the video image display positions of six people are as shown in FIG. 5 .

Wherein, if A is the moderator, and when the degree of participation in the meeting does not exceed the first preset value, and when the number of participants in the meeting is less than the second preset value, the target image display strategy is the moderator image display strategy, so , the display positions of the video images of the six people are shown in Figure 6.

Furthermore, during the video conference, there may be no moderator, so step S302 also includes:

In response to the fact that there is no moderator information in the current conference target information, comparing the number of conference participation targets with a second preset value;

When it is determined that the number of participants in the meeting does not exceed a second preset value, determine a multi-person session image display strategy according to the meeting request instruction, the number of participants in the meeting, and the degree of participation in the meeting;

When it is determined that the number of conference participants exceeds a second preset value, an alternate speaking image display strategy is determined according to the conference request instruction, the number of conference participants, and the degree of conference participation.

It is worth noting that in the multi-person session image display strategy, the obtained target images of the participants will be integrated to obtain a panorama containing all the target images of the participants, and the panorama will be fixed on the display image. At the lower end, there will be three image frames fixed on the upper end of the display image. These three image frames will be adjusted to display the target image when it is detected that there are participants speaking. If the number of participants in the panorama is not enough for three people, On the upper part of the display image, only the image frame corresponding to the number of people is displayed, and the part with insufficient number of people displays a black frame. In addition, the order of these three image frames is fixed, for example: currently there are A, B, C, D, E, F six people, then display which image frame is on the top of the image in the following 10 order: ABC, ABD, ABE, ABF, ACD, ACE, ACF, ADE, ADF, AEF.

It should be understood that in the alternate speech image display strategy, there will not be a fixed number of image frames, and the display images will be cut according to the number of participants. The cutting rule is:

When there is only one person, the screen displaying the image has only one portrait frame, showing one person;

When there are only 2 people, the screen displaying the image is divided into left and right, and 2 people are displayed;

When there are only 3 people, the screen displaying the image is divided into 4 equal parts according to top, bottom, left and right, and 3 people are displayed, but the screen in the lower right corner is displayed on a black screen;

When there are only 4 people, the screen displaying the image is divided into 4 equal parts according to top, bottom, left and right, and 4 people are displayed;

When there are only 5 people, divide it into 6 equal parts as shown in Figure 12, and display 5 people, but the screen in the lower right corner is displayed on a black screen;

When there are only 6 people, perform 6 equal divisions as shown in Figure 12, showing 6 people;

When there are only 7 people, the screen displaying the image is divided into 9 equal parts according to the top, bottom, left and right, and 7 people are displayed, but the 2 screens in the lower right corner are displayed on a black screen;

When there are only 8 people, the screen displaying the image is divided into 9 equal parts according to the top, bottom, left and right, and 8 people are displayed, but the screen in the lower right corner is displayed on a black screen;

When there are only 9 people, the screen displaying the image is divided into 9 equal parts according to top, bottom, left and right, and 9 people are displayed;

When the number of people is more than 9, the screen shows only 9 people, and the initially displayed people are displayed clockwise according to the first person who spoke, and when the 10th person speaks, replace the last person who spoke;

It is understandable that when the video conferencing device does not detect the moderator information, it judges the quantitative relationship between the number of conference participants and the second preset value, and when the number of conference participants does not exceed the second preset value, the target The image display strategy is a multi-person conversation image display strategy; when the number of participants in the meeting exceeds a second preset value, the target image display strategy is an alternate speaking image display strategy.

In a specific implementation, there are currently six people A, B, C, D, E, and F. When the number of participants in the meeting does not exceed the second preset value, the target image display strategy is a multi-person conversation image display strategy, showing The image is shown in Figure 7. In this embodiment, if the upper end of the previously displayed image is A, D, and F, and it is detected that C is speaking at this time, then according to the switching rule, the upper end of the displayed image will be switched to A, D, and F. C, D three target images of participants.

Wherein, when the number of participating targets exceeds the second preset value, the target image display strategy is an alternate speech image display strategy, and the images are displayed according to the cutting rule as shown in FIG. 8 .

In this embodiment, when a meeting request instruction is received, the target image display mode is determined according to the meeting request instruction, and the current meeting target information is obtained, the lens orientation is determined according to the target image display mode, and the camera orientation is adjusted according to the lens orientation. Camera orientation, if the camera orientation is upward, then obtain the number of participants and the degree of participation in the meeting in the current meeting target information, according to the camera orientation, the number of participants and the conference participation The target image panorama display strategy is determined according to the degree, and when the lens orientation is rotated to the lens orientation, the preset privacy image or the current target image captured by the camera is displayed according to the target image display strategy. In this embodiment, the target image display mode is determined through the meeting request command input by the user, and different camera orientations are adjusted according to different target image display modes, which is applicable to image display in multiple scenarios. The combination of the number and the degree of meeting participation selects an appropriate target image panorama display strategy, and may select different image display methods according to the different target information of the participants, so as to avoid that it can only be used in a single application scenario when conducting a video session. It is impossible to choose different display methods according to the number of participants and the conference mode, and the technical problem is that the user experience is poor.

Referring to FIG. 9 , FIG. 9 is a schematic flowchart of a third embodiment of an image display method of the present application.

Based on the first embodiment above, in this embodiment, the step S30 includes:

Step S301A: In response to the camera orientation being forward facing, determine a target image wide-angle display strategy according to the camera orientation and the current conference participant information.

It can be understood that if the target display mode is the wide-angle mode, that is, when the camera orientation is forward, then the target image wide-angle display strategy is determined according to the lens orientation and the current conference participant information.

The description of the value is that in this embodiment, when the lens is facing forward, the lens of the video conferencing device adopts an image that can capture a wide-angle range of 220°, and the larger the angle on both sides of the lens, the greater the distortion of the image , in order to have a good image display effect, the angles on both sides of the obtained 220° image can be cut off by 20° to achieve the effect of a 180° image.

In the specific implementation, with reference to FIG. 10 , the existing four people A, B, C, and D are divided into four people located in front of the video conferencing equipment. D and the two people will be farther away from the lens, so the portraits of B and C displayed on the monitor are larger than those of A and D, so as to distinguish the distance relationship between the participants and the video conferencing equipment, and the final display image Refer to Figure 11.

Among them, if the three people in BCD leave and only A is left sitting in the original position, the portrait of A will be enlarged accordingly and placed in the center; when the three people in ACD leave, only B is left sitting in the original position, because B itself is at the side , so when zooming in, it will give priority to keeping B in the display frame, and finally the position of B will be zoomed in to a certain extent, and will be a little to the left; The same, so the image of AD is enlarged in the same proportion and placed in the center; when there are only BC on the entire conference table, the portraits of B and C are detected and enlarged in the same proportion, but because the sitting positions of BC and BC are different Move to the side, so the magnification ratio of BC is smaller than that of AD; when there are only two BDs on the entire conference table, after detecting B and D, the portrait and other scenes in the screen will be enlarged in the same ratio, because B is closer and closer to the lens Move to the side, so when zooming in on the same scale, the position of B is detected at the same time to prevent B's body from exceeding the display screen.

In this embodiment, when a meeting request instruction is received, the target image display mode is determined according to the meeting request instruction, and the current meeting target information is obtained, the lens orientation is determined according to the target image display mode, and the camera orientation is adjusted according to the lens orientation. The lens orientation, if the lens orientation is the lens facing forward, then determine the wide-angle display strategy of the target image according to the lens orientation and the current participant information, and when the lens orientation rotates to the lens orientation, set the target image wide-angle display strategy according to the target image. Set a private image or the current target image acquired by the camera for display. In this embodiment, the wide-angle display mode of the target image is determined through the meeting request command input by the user, and the camera orientation is adjusted to the front of the camera according to the wide-angle display mode of the target image, which is suitable for image display in multiple scenarios. It is possible to choose a different image display method according to the different target information of the participants, so as to avoid that the video session can only be used in a single application scenario, and the conference cannot be used according to the number of participants. Mode selection uses different display methods, technical issues with poor user experience.

Referring to FIG. 12 , FIG. 12 is a schematic flowchart of a fourth embodiment of an image display method of the present application.

Based on the first embodiment above, in this embodiment, the step S30 includes:

Step S301B: In response to the camera orientation being downward, determine a target image privacy display policy according to the lens orientation.

It can be understood that if the target display mode is the privacy mode, that is, when the camera orientation is the lens downward, then the target image privacy display policy is determined according to the lens orientation.

It should be noted that this mode is mainly to protect the privacy of the conference. When there is something to discuss in the local area and you don’t want the other party to see the local screen and hear the local discussion sound, without closing the ongoing video call conference, It can be implemented in this manner. In addition, the privacy mode can also be entered into the privacy mode from the panoramic mode or the wide-angle mode.

In a specific implementation, when the current target image display mode is panoramic mode, one party may need to conduct internal discussions and temporarily turn off the camera. When the device controller receives the display prohibition instruction, it controls the camera to rotate according to the display prohibition instruction, and controls the sensor to detect the current lens orientation; microphone.

Wherein, when the current target image display mode is the wide-angle mode, the user sends an instruction by pressing a button of the video conferencing device or a remote control, etc., and when the video conferencing device controller receives the display prohibition instruction, it controls the display according to the display prohibition instruction. The camera is rotated and the sensor is controlled to detect the current lens orientation. If the current lens orientation is downward, a preset privacy image is displayed and the microphone is turned off.

It should be noted that after the discussion is completed, the video conference needs to be continued. At this time, the user sends an instruction by pressing a button of the video conference device or a remote control, and the controller of the video conference device receives the display start instruction. The display opening command controls the rotation of the camera, and when the lens restoration position is rotated to the lens position corresponding to the target image display strategy, the microphone is turned on, and the target image of the current participant is obtained back, and the target image of the current participant is passed through the preset image The segmentation model performs image segmentation to obtain multiple segmented images.

It can be understood that the preset privacy image may be one or more images preset by the user, and the preset privacy image is configured to block the current video image and remind other video users that the video is not interrupted.

In this embodiment, when a meeting request instruction is received, the target image display mode is determined according to the meeting request instruction, and the current meeting target information is obtained, the lens orientation is determined according to the target image display mode, and the camera orientation is adjusted according to the lens orientation. Camera orientation, if the lens orientation is facing downward, then determine the target image privacy display policy according to the lens orientation, and when the lens orientation rotates to the lens orientation, set the preset privacy image according to the target image privacy display policy to show. In this embodiment, the target image privacy display mode is determined through the meeting request command input by the user, and the camera orientation is adjusted to the lens downward according to the target image privacy display mode, which is suitable for image display in multiple scenarios, according to the camera orientation and current meeting target information It is possible to choose a different image display method according to the different target information of the participants, so as to avoid that the video session can only be used in a single application scenario, and the conference cannot be used according to the number of participants. Mode selection uses different display methods, technical issues with poor user experience.

In addition, the embodiment of the present application also proposes a storage medium, on which an image display program is stored, and when the image display program is executed by a processor, the steps of the above-mentioned image display method are realized.

Since the storage medium adopts all the technical solutions of all the above-mentioned embodiments, it at least has all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

Referring to FIG. 13 , FIG. 13 is a structural block diagram of the first embodiment of the image display device of the present application.

As shown in Figure 13, the image display device proposed in the embodiment of the present application includes:

The instruction receiving module 10 is configured to receive a meeting request instruction, determine a target image display mode according to the meeting request instruction, and acquire current meeting target information.

The lens adjustment module 20 is configured to determine the orientation of the lens according to the target image display mode, and adjust the orientation of the lens according to the orientation of the lens.

The policy confirmation module 30 is configured to determine a target image display policy based on the camera orientation and the current meeting target information.

The image display module 40 is configured to display a preset private image or a current target image captured by a camera according to the target image display policy in response to the camera turning to the lens orientation.

In an embodiment, the policy confirmation module 30 is further configured to obtain the number of conference participants and the degree of conference participation in the current conference participant information in response to the camera orientation being upward; according to the The target image panorama display strategy is determined by the lens orientation, the number of participants in the meeting and the degree of participation in the meeting; when the lens orientation is rotated to the lens orientation, the preset privacy image or the current image acquired by the camera is determined according to the target image display strategy. The displaying of the target image further includes: displaying the current target image acquired by the camera according to the target image panorama display strategy when the lens orientation is rotated to the lens orientation.

In one embodiment, the policy confirmation module 30 is further configured to determine the wide-angle display strategy of the target image according to the lens orientation and the current participant information in response to the lens orientation being the lens facing forward; When the lens is in the orientation, displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy also includes: when the lens orientation is rotated to the lens orientation, displaying the camera according to the target image wide-angle display strategy The obtained current target image is displayed.

In an embodiment, the image display module 30 is further configured to determine a target image privacy display strategy according to the lens orientation in response to the lens orientation being downward; when the lens orientation is rotated to the lens orientation , displaying a preset privacy image or a current target image acquired by a camera according to the target image display strategy, and further comprising: displaying a preset privacy image according to the target image privacy display strategy when the lens orientation is rotated to the lens orientation , and turn off the microphone.

In one embodiment, the image display module 40 is further configured to obtain the image of the current conference target through the camera, and turn on the microphone; perform image segmentation on the current conference target image through a preset image segmentation model to obtain the Segmenting the image; performing image processing on the segmented image to obtain the current target image.

In one embodiment, the policy confirmation module 40 is further configured to determine the initial segmentation point of the current conference target image based on the camera orientation; The image is segmented through a preset image segmentation model to obtain a segmented image.

In an embodiment, the image display module 40 is further configured to perform mouth shape detection on the current target image through a preset mouth shape detection model to obtain an initial speaker image; obtain the current sound signal through a microphone, and Determine speaker information based on the current sound signal; determine a target speaker image according to the speaker information and the initial speaker image, and mark the target speaker image.

In this embodiment, when a meeting request instruction is received, the target image display mode is determined according to the meeting request instruction, and the current meeting target information is obtained, the lens orientation is determined according to the target image display mode, and the camera orientation is adjusted according to the lens orientation. Camera orientation, based on the lens orientation and the current meeting target information to determine the target image display strategy, when the lens orientation is rotated to the lens orientation, according to the target image display strategy, the preset privacy image or the current image captured by the camera The target image is displayed. In this embodiment, the target image display mode is determined through the meeting request command input by the user, and different camera orientations are adjusted according to different target image display modes. Select the appropriate target image display strategy, and may choose different image display methods according to the different target information of the participants, avoiding that it can only be used in a single application scenario during a video session, and cannot be used according to the number of participants and the conference mode Choose to use different display methods, technical issues with poor user experience.

It should be understood that the above is only an example, and does not constitute any limitation to the technical solution of the present application. In a specific application, those skilled in the art can make settings according to needs, and the present application does not limit this.

It should be noted that the workflow described above is only illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select part or all of them to implement according to actual needs. The purpose of the scheme of this embodiment is not limited here.

In addition, for technical details not exhaustively described in this embodiment, reference may be made to the image display method provided in any embodiment of the present application, which will not be repeated here.

Furthermore, it should be noted that in this document, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or system comprising a set of elements includes not only those elements, but also other elements not expressly listed, or elements inherent in such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system comprising that element.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as a read-only memory (Read Only Memory) , ROM)/RAM, magnetic disk, optical disk), including several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The above are only optional embodiments of the application, and are not intended to limit the patent scope of the application. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the application, or directly or indirectly used in other related technologies fields, are all included in the scope of patent protection of this application in the same way.

Claims (10)

1. An image display method, comprising:

   Receiving the meeting request instruction, determining the target image display mode according to the meeting request instruction, and obtaining the current meeting target information;

   determining the lens orientation according to the target image display mode, and adjusting the lens orientation according to the lens orientation;

   determining a target image display strategy based on the lens orientation and the current meeting target information;

   In response to the camera turning to the camera orientation, the preset private image or the current target image captured by the camera is displayed according to the target image display strategy.
2. The image display method according to claim 1, wherein the target image display strategy comprises a target image panorama display strategy;

   The determining target image display strategy based on the lens orientation and the current meeting target information includes:

   Responding to the fact that the camera orientation is facing upwards, obtain the number of participants and the degree of participation in the meeting in the information about the current meeting participants;

   Determine a target image panorama display strategy according to the lens orientation, the number of participants in the meeting, and the degree of participation in the meeting;

   In response to the lens turning to the lens orientation, displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy, further comprising:

   In response to the camera turning to the camera orientation, the current target image acquired by the camera is displayed according to the target image panorama display strategy.
3. The image display method according to claim 1, wherein the target image display strategy comprises a target image wide-angle display strategy;

   The determining target image display strategy based on the lens orientation and the current meeting target information includes:

   In response to the camera orientation being forward facing, determine a target image wide-angle display strategy according to the lens orientation and the current conference participant information;

   In response to the lens turning to the lens orientation, displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy, further comprising:

   In response to the lens turning to the lens orientation, the current target image acquired by the camera is displayed according to the target image wide-angle display strategy.
4. The image display method according to any one of claims 1-3, wherein the target image display strategy includes a target image privacy display strategy;

   The determining target image display strategy based on the lens orientation and the current meeting target information includes:

   Responsive to the camera orientation being downward, determining a target image privacy display policy according to the lens orientation;

   In response to the lens turning to the lens orientation, displaying the preset privacy image or the current target image acquired by the camera according to the target image display strategy, further comprising:

   In response to the camera turning to the camera orientation, displaying a preset privacy image according to the target image privacy display policy, and turning off the microphone.
5. The image display method according to any one of claims 1-3, wherein when the camera orientation is rotated to the camera orientation, the preset privacy image or the current target image acquired by the camera is displayed according to the target image display strategy Before making a presentation, include:

   Obtain the image of the current participant target through the camera, and turn on the microphone;

   Segmenting the image of the current participant target through a preset image segmentation model to obtain a segmented image;

   performing image processing on the segmented image to obtain the current target image.
6. The image display method according to claim 5, wherein said performing image segmentation on said current meeting target image through a preset image segmentation model to obtain a segmented image comprises:

   Determining the initial segmentation point of the current participant target image based on the camera orientation;

   Segmenting the current meeting target image by using a preset image segmentation model based on the initial segmentation point and the preset direction to obtain a segmented image.
7. The image display method according to any one of claims 1-3, wherein when the camera orientation is rotated to the camera orientation, the preset privacy image or the current target image acquired by the camera is displayed according to the target image display strategy After the presentation, also include:

   performing mouth shape detection on the current target image through a preset mouth shape detection model to obtain an initial speaker image;

   Obtaining a current sound signal through a microphone, and determining speaker information according to the current sound signal;

   Determine a target speaker image according to the speaker information and the initial speaker image, and mark the target speaker image.
8. An image display device, comprising:

   The instruction receiving module is configured to receive the meeting request instruction, determine the target image display mode according to the meeting request instruction, and acquire the current meeting target information;

   The lens adjustment module is configured to determine the orientation of the lens according to the target image display mode, and adjust the orientation of the lens according to the orientation of the lens;

   A strategy confirmation module, configured to determine a target image display strategy based on the camera orientation and the current meeting target information;

   The image display module is configured to display a preset private image or a current target image captured by a camera according to the target image display policy in response to the lens turning to the lens orientation.
9. An image display device, the image display device comprising: a memory, a processor, and an image display program stored in the memory and operable on the processor, when the image display program is executed by the processor Realizing the image display method as claimed in any one of claims 1 to 7.
10. A storage medium, on which an image display program is stored, and when the image display program is executed by a processor, the image display method according to any one of claims 1 to 7 is realized.