JP2023082816A - Image processing device, information processing device, video conference server, and video conference system - Google Patents

Abstract

To reduce dimensional irregularity in the faces of attendants placed in a common background.SOLUTION: An image processing section 20 comprises: an image acquisition unit 21 which acquires image data; a face area identifying unit 22 which identifies the face area of a person included in the image data; a dimension detection unit 23 which detects the dimension of a horizontal width at a predetermined height position of the identified face area, as a horizontal width dimension; a reference value identifying unit 27 which identifies a horizontal width reference value corresponding to the age of the person from reference information associating an age and the horizontal width reference value of the face area with each other; an image adjustment unit 28 which adjusts the image data so as to make the horizontal width dimension close to the identified horizontal width reference value; and an output unit 29 which outputs the adjusted image data.SELECTED DRAWING: Figure 4

Description

The present invention relates to an image processing device, an information processing device, a videoconference server, and a videoconference system.

In recent years, video conferences have become more frequent, and along with this, various application tools related to video conferences have been proposed. One of them is that by placing the participants in the video conference under a common background, participants can feel as if they are gathering in a common space and listening to the meeting or seminar. Techniques are suggested that can be given to participants. For example, in the "immersive view" provided by Zoom Video Communications, Inc., the host can select a favorite background from among multiple prepared backgrounds, and the participant's image is displayed in the selected background. can be placed manually.

However, the size of the body including the face of each participant, the size of the face, and the height position of the face in the image transmitted from each participant of the video conference vary. Therefore, if the image data received from each participant is arranged as it is, as shown in FIG. 14, the positions and heights of the faces of the participants may become uneven, and the image as a whole may appear unnatural. was there.

The present invention has been made in view of such circumstances, and provides an image processing apparatus, an information processing apparatus, and a video conference that can reduce variations in face size of participants placed on a common background. The purpose is to provide a server and a video conference system.

A first aspect of the present invention comprises a processor and a memory storing a program configured to be executed by the processor, the program acquiring image data and displaying images of a person included in the image data. A face area is identified, a width dimension at a predetermined height position of the identified face area is detected as a width dimension, and a width corresponding to the age of the person is detected from reference information in which the age and the width reference value of the face area are associated. The image processing apparatus includes instructions for specifying a reference value, adjusting the image data so that the width dimension approaches the specified width reference value, and outputting the adjusted image.

A second aspect of the present invention includes an image acquisition unit that acquires image data, a face region specifying unit that specifies a face region of a person included in the image data, and a horizontal width at a predetermined height position of the specified face region. a dimension detection unit that detects the dimension as a width dimension; a reference value identification unit that identifies a width reference value corresponding to the age of the person from reference information in which the age and the width reference value of the face area are associated; The image processing apparatus includes an image adjustment unit that adjusts the image data so that the width dimension approaches a width reference value, and an output unit that outputs the adjusted image data.

A third aspect of the present invention is a program for causing a computer to function as the image processing apparatus.

A fourth aspect of the present invention is an information processing apparatus including the above image processing apparatus.

A fifth aspect of the present invention comprises a step of obtaining image data, a step of identifying a face region of a person included in the image data, and a width dimension at a predetermined height position of the identified face region as a width dimension. identifying a width reference value corresponding to the age of the person from reference information in which the age and the width reference value of the face area are associated; and bringing the width dimension closer to the identified width reference value. and a step of outputting the adjusted image data by a computer.

A sixth aspect of the present invention includes a plurality of information processing devices, and a video conference server that receives the adjusted image data from the plurality of information processing devices, each of the information processing devices performing the image processing described above. device, wherein the videoconference server is a videoconferencing system comprising an image synthesizing unit that arranges the image data received from each of the information processing devices under a common background.

A seventh aspect of the present invention is a receiving unit for receiving image data of a plurality of participants in a video conference, a face area specifying unit for specifying face areas of the participants included in each of the image data, and the specified faces. A dimension detection unit that detects a width dimension at a predetermined height position of an area as a width dimension, and a width reference value that serves as a reference is calculated by statistically processing a plurality of width dimensions detected in each of the image data. and an image adjustment unit for adjusting each of the image data so that each of the width dimensions of each of the image data approaches the width reference value, and an image synthesizing unit for arranging each of the image data on a common virtual background. video conferencing server.

ADVANTAGE OF THE INVENTION According to this invention, it is effective in the ability to reduce the dispersion|variation in the size of the face of the participant arranged on a common background.

1 is a system configuration diagram schematically showing the system configuration of a video conference system according to a first embodiment of the present invention; FIG. 1 is a schematic external view as an example of an information processing apparatus according to a first embodiment of the present invention; FIG. 1 is a schematic configuration diagram showing an example of a hardware configuration of an information processing apparatus according to a first embodiment of the present invention; FIG. 2 is a functional block diagram showing an example of image processing functions provided in the information processing apparatus according to the first embodiment of the present invention; FIG. It is a figure for demonstrating the width dimension and height dimension which are detected by the dimension detection part which concerns on 1st Embodiment of this invention. FIG. 10 is a diagram showing growth curves for height, iliac spine height, right leg length, and head circumference from 1 year old to 20 years old. FIG. 4 is a diagram for explaining adjustment processing by an image adjustment unit according to the first embodiment of the present invention; 4 is a flow chart showing an example of a processing procedure of an image processing method according to the first embodiment of the present invention; 1 is a schematic configuration diagram showing an example of a hardware configuration of a videoconference server according to a first embodiment of the present invention; FIG. 3 is a functional block diagram showing an example of functions provided in the videoconference server according to the first embodiment of the present invention; FIG. FIG. 4 is a diagram showing an example of an image displayed on the display of each information processing device in the video conference system according to the first embodiment of the present invention; FIG. 9 is a functional block diagram showing an example of functions provided in a videoconference server according to the second embodiment of the present invention; FIG. 11 is a functional block diagram showing an example of functions provided in a videoconference server according to the third embodiment of the present invention; It is a figure for demonstrating the subject of this invention.

[First embodiment]
A first embodiment of an image processing apparatus, an information processing apparatus, a videoconference server, and a videoconference system according to the present invention will be described below with reference to the drawings.

FIG. 1 is a system configuration diagram schematically showing the system configuration of a video conference system 1 according to the first embodiment of the invention. As shown in FIG. 1, a network 8 is connected with a plurality of information processing apparatuses 10 and a videoconference server 50 . Examples of the information processing device 10 include a notebook PC, a tablet terminal, a smartphone, and the like. For convenience of explanation, a notebook PC will be described as an example of the information processing apparatus 10 .
Although three information processing apparatuses 10 are illustrated in the example shown in FIG. 1, the number of connected information processing apparatuses is not limited to this.

FIG. 2 is a schematic external view as an example of the information processing apparatus 10 according to the first embodiment of the invention. As shown in FIG. 2, the information processing apparatus 10 includes a body-side housing 2 and a display-side housing 3, both of which are substantially rectangular parallelepipeds. The body-side housing 2 has an input device 4 . The input device 4 is a user interface for the user to perform input operations, and includes a keyboard composed of various keys for inputting characters, commands, etc., and for moving the cursor on the screen and selecting various menus. It has a touch pad.

The display-side housing 3 includes a display 5 for displaying images. In this embodiment, the display 5 is an LCD (Liquid Crystal Display). The LCD converts input display data into a video signal, and displays various information according to the converted video signal on the display screen.

The body-side housing 2 and the display-side housing 3 are connected by a pair of connecting portions 6 at their respective ends. The connecting portion 6 is a hinge, and supports the main body side housing 2 and the display side housing 3 so that they can be opened and closed.

The display-side housing 3 also includes a camera 7 for acquiring an image. The camera 7 is arranged, for example, in the central portion above the display screen provided in the display-side housing 3 (the side portion opposite to the side portion connected by the connecting portion 6), and a person in front (for example, a user person's face) can be imaged.

FIG. 3 is a schematic configuration diagram showing an example of the hardware configuration of the information processing apparatus 10 according to this embodiment. As shown in FIG. 3, the information processing apparatus 10 includes, in addition to the input device 4, the display 5, and the camera 7 described above, a CPU (processor) 11, a main memory 12, a storage section 13, an external interface 14, a communication interface 15, A speaker 16, a microphone 17, and the like are provided. These units are connected to each other directly or indirectly via a bus, and cooperate with each other to perform various processes.

The CPU 11 controls the entire information processing apparatus 10 by, for example, an OS (Operating System) stored in a storage unit 13 connected via a bus, and executes various programs stored in the storage unit 13. Executes various processing. A plurality of CPUs 11 may be provided and cooperate with each other to realize processing.

The main memory 12 is composed of a writable memory such as a cache memory and a RAM (Random Access Memory), and is used as a working area for reading the execution program of the CPU 11 and writing processing data by the execution program.

The storage unit 13 is a non-transitory computer readable storage medium. Examples of the storage unit 13 include ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, and the like. The storage unit 13 stores, for example, an OS such as Windows (registered trademark), iOS (registered trademark), and Android (registered trademark) for controlling the entire information processing apparatus 10, BIOS (Basic Input/Output System), and peripheral devices. It stores various device drivers, various application software, and various data and files for operating the hardware. The storage unit 13 also stores programs for implementing various processes and various data required for implementing various processes. A plurality of storage units 13 may be provided, and data as described above may be divided and stored in each storage unit 13 .

The external interface 14 is an interface for connecting with an external device. Examples of external devices include an external monitor, a USB memory, an external HDD, an external camera, and the like. Although only one external interface is shown in the example shown in FIG. 1, a plurality of external interfaces may be provided.

The communication interface 15 functions as an interface for connecting to a network, communicating with other devices, and transmitting and receiving information. For example, the communication interface 15 communicates with other devices by wire or wirelessly. Wireless communication includes communication through lines such as Bluetooth (registered trademark), Wi-Fi, 3G, 4G, 5G, LTE, and wireless LAN. An example of wired communication is communication through a line such as a wired LAN (Local Area Network).

Since the speaker 16 and the microphone 17 have well-known configurations, detailed description thereof will be omitted here.

The videoconference server 50 is a so-called computer, and includes a CPU, a main memory, a storage unit, a communication interface, and the like, like the information processing apparatus 10 described above. Details of the video conference server 50 will be described later.

Next, an example of the functions of the information processing apparatus 10 according to this embodiment will be described with reference to the drawings. FIG. 4 is a functional block diagram showing an example of image processing functions provided in the information processing apparatus 10 according to this embodiment.

A series of processes for realizing various functions to be described later are stored in the storage unit 13 provided in each information processing apparatus 10 in the form of a program, for example. Various functions are realized by reading out the information and executing processing and arithmetic processing of the information. The program may be pre-installed in the storage unit 13, may be provided in a state stored in another computer-readable storage medium, or may be distributed via wired or wireless communication means. may apply. Computer-readable storage media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, and the like.

As shown in FIG. 4, the image processing unit (image processing device) 20 of the information processing device 10 includes an image acquisition unit 21, a face region identification unit 22, a dimension detection unit 23, an attribute estimation unit 24, a parameter storage unit 25, a reference An information storage unit 26 , a reference value identification unit 27 , an image adjustment unit 28 and an output unit 29 are provided.

The image acquisition unit 21 acquires image data captured by the camera 7, for example. Specifically, the image acquisition unit 21 successively acquires image data captured by the camera 7 at a predetermined frame rate.

The face area identifying unit 22 identifies the face area of a person (for example, a participant in a video conference) included in the input image data. Identification of a person's face area can be realized by appropriately adopting a known technique. For example, the face region is specified by extracting the feature amount of the face in the image. Further, the face area specifying section 22 may be configured to be able to specify eyebrows, eyes, nose, mouth, etc. in the face area.

For example, as shown in FIG. 5, the dimension detection unit 23 detects the width dimension at a predetermined height position of the face area identified by the face area identification unit 22 as the width dimension L1. For example, in this embodiment, as shown in FIG. 5, the position of a predetermined portion of the ear (for example, trugs) is used as the predetermined height position, but the present invention is not limited to this example. For example, the position of the ear, the position of the eyebrows, the position of the pupil, etc. may be used as the height position.

5, the dimension detection unit 23 detects the height dimension of the person as the height dimension L2 based on the information of the face area identified by the face area identification unit 22. FIG. For example, in the present embodiment, a position of a predetermined portion of the ear (for example, tragus) is used as the predetermined height position, but the position is not limited to this example. For example, the position of the ear, the position of the eyebrows, the position of the pupil, etc. may be used as the height position.

Here, it has been published in academic papers and the like that the width of the face and the circumference of the head show less variation depending on age and sex than other parts of the body. For example, ”Static adult human physical characteristics of the adult head, from pages 72-75 of Poston, Alan. (April 2000) Department of Defense Human Factors Engineering Technical Advisory Group (DOD HFE TAG)” (https://de.wikipedia .org/wiki/Datei:HeadAnthropometry.JPG), as a result of a certain statistic, the average face width for men is 14.5 cm, and the average face width for women is 13.3 cm. It is disclosed that the difference is only about 0.8 cm. In addition, Jissen Women's University, Faculty of Life Science, Department of Life Environment, "20th Child Growth and Clothing Keiko Takabe" "https://www.jissen.ac.jp/kankyo/lib-lec20.html" The Percentage Growth Curves (Male, 1978-81) of Kogyo Giken data shows growth curves for height, iliac spine height, right leg length, and head circumference from 1 to 20 years of age ( For example, see FIG. 6).

From the statistical results described above, it can be seen that variations in the width of the face and head circumference due to age and gender are extremely small compared to other parts. That is, according to the growth curve shown in FIG. 6, the head circumference already shows a value corresponding to about 80% of the head circumference of an adult at the age of 1, and there is almost no difference in the value due to age after the age of 13. I understand. Based on such new findings, the inventors found that if the width of the face in the image data input from the camera 7 is adjusted to a value according to the age, the faces of the participants participating in the video conference can be improved. I got the idea that the size can be adjusted to a natural size.

The attribute estimation unit 24 estimates the age and gender of the person based on the image of the facial area identified by the facial area identification unit 22 . As for the method of estimating the age and sex of a person, it is possible to adopt a known method as appropriate. For example, the feature amount of the face image is extracted, and the similarity is calculated from the pre-registered feature amount for identifying age (age) and gender, and based on the similarity value, age and gender to estimate As an example of a technique for estimating age and sex, it is possible to employ the technique described in Japanese Patent No. 5287333, for example.

The parameter storage unit 25 stores the width dimension and the estimated age and sex detected in the same image data as one data set. For example, the parameter storage unit 25 is a buffer memory that stores a predetermined number (predetermined number≧2) of data sets in order of newest. The parameter storage unit 25 is implemented by, for example, a FIFO (First In First Out) memory. As a result, the parameter storage unit 25 stores a predetermined number of data sets detected most recently.

The reference information storage unit 26 stores reference information in which the age, the width reference value L1_ref of the face region, and the height reference value L2_ref of the person are associated with each other. This reference information is provided for each gender. Ages may be divided into ages such as teens, twenties, and thirties, and each age may be associated with a width reference value and a height reference value.

The reference value identification unit 27 acquires age information from a predetermined number of data sets stored in the parameter storage unit 25, and statistically processes the information to identify a representative age. In other words, the reference value identifying unit 27 calculates a representative age by statistically processing a plurality of ages estimated in a plurality of time-series image data. Examples of typical ages include the average age, the 50% age of the age distribution, and the like.

Further, the reference value specifying unit 27 acquires gender information from a predetermined number of data sets stored in the parameter storage unit 25, and statistically processes the information to specify representative gender. For example, the reference value identification unit 27 identifies the sex with the highest number among the predetermined number of sexes read from the parameter storage unit 25 as the representative sex.
The reference value specifying unit 27 specifies the width reference value L1_ref and the height reference value L2_ref corresponding to the representative age using the reference information corresponding to the specified sex.

The image adjusting unit 28 adjusts the image data so that the horizontal width dimension L1 of the image data approaches the horizontal width reference value L1_ref specified by the reference value specifying unit 27 .
For example, the image adjustment unit 28 acquires the width dimension from a predetermined number of data sets stored in the parameter storage unit 25 . In other words, the image adjuster 28 statistically processes a plurality of width dimensions detected in a plurality of time-series image data to calculate a representative width dimension. Examples of typical width dimensions include an average value, a 50% percentile value of the width dimensions, and the like. Then, the image adjustment unit 28 calculates an adjustment ratio (enlargement/reduction ratio) such that the representative width dimension becomes the width reference value, and uses the calculated adjustment ratio to reduce or enlarge the image data. In this way, by calculating the adjustment ratio using a predetermined number of the most recently detected width dimensions, it is possible to smooth the change in the adjustment ratio.

Further, the image adjusting unit 28 adjusts the height position of the person in the image data so that the height L2 of the person in the image data approaches the height reference value L2_ref specified by the reference value specifying unit 27. For example, the image adjuster 28 clips the height dimension L2 of the person in the image data so that it matches the height reference value L2_ref.
As a result, for example, in the image data shown in FIG. 5, as shown in FIG. 7, the width of the person is adjusted to the width reference value L1_ref, and the height of the person is adjusted to the height reference value L2_ref. The height position is adjusted so that

Further, as shown in FIG. 7, the image adjustment unit 28 adjusts the height position of the person, and if there is a shortage of pixels Px in the height direction in the image data after adjustment, the lack of pixels Px occurs. The pixel Px is estimated from surrounding pixel information and supplemented.

The output section 29 outputs the image data adjusted by the image adjusting section 28 . The output unit 29 , for example, transmits the adjusted image data to the videoconference server 50 via the network 8 .

Next, an image processing method executed by the information processing apparatus 10 will be described with reference to FIG. FIG. 8 is a flow chart showing an example of the processing procedure of the image processing method. The following series of processes are executed by CPU (processor) 11 reading a program stored in storage unit 13 to main memory 12 and processing and arithmetic processing of information.

For example, when the information processing apparatus 10 is connected to the videoconference server 50 via the network 8 and the camera function is turned on, image data is acquired by the camera 7 and the acquired image data is output as needed.
When the image processing unit (image processing device) 20 acquires the image data output from the camera 7 (SA1), the image processing unit (image processing device) 20 specifies a person's face area included in the acquired image data (SA2). Subsequently, the width dimension at a predetermined height position of the specified face region, for example, the position of the trugs, is detected as the width dimension L1, and the height dimension L2 of the person included in the image data is detected ( SA3).

Subsequently, the age (generation) and sex of the person are estimated based on the feature amount of the person's face area in the image data (SA4). Subsequently, the width dimension L1 and height dimension L2 detected in step SA3 and the age and sex of the person detected in step SA4 are stored as one data set in the parameter storage unit 25 (SA5). As a result, the oldest data set already stored in the parameter storage unit 25 is deleted, and the latest data set described above is stored in the parameter storage unit 25 .

Next, age information is acquired from a predetermined number of data sets stored in the parameter storage unit 25, and representative ages are specified by statistically processing these. Similarly, gender information is obtained from a predetermined number of data sets stored in the parameter storage unit 25, and is statistically processed to specify representative gender (SA6).

Subsequently, the width reference value L1_ref and the height reference value L2_ref corresponding to the representative age are specified using the reference information corresponding to the representative sex (SA7).

Next, the width dimension L1 is acquired from a predetermined number of data sets stored in the parameter storage unit 25, and a representative width dimension is calculated by statistically processing these. Similarly, the height dimension L2 is acquired from a predetermined number of data sets stored in the parameter storage unit 25, and a representative height dimension is calculated by statistically processing these (SA8).

Subsequently, an adjustment ratio (enlargement/reduction ratio) is calculated so that the representative width dimension becomes the width reference value L1_ref (SA9), and based on the calculated adjustment ratio and the height reference value L2_ref, which was input in step SA1 Image data is adjusted (SA10). Specifically, the image data is enlarged or reduced based on the calculated adjustment ratio, and the height position of the person in the image data is adjusted so that the height of the person in the image data approaches the height reference value L2_ref. As a result, for example, the image data shown in FIG. 5 is enlarged or reduced so that the width dimension L1 of the person's face becomes the width reference value L1_ref as shown in FIG. is the height reference value L2_ref. Further, as shown in FIG. 7, if there is a missing pixel Px in the height direction in the adjusted image data, the missing pixel Px is estimated from surrounding pixel information and supplemented.

When image adjustment is completed in this way, the image data after adjustment is output (SA11). The adjusted image data is sent to the videoconference server 50 (see FIG. 1).
By performing the above-described processing each time image data is acquired, image data in which the size and height position of the person are adjusted are continuously transmitted to the videoconference server 50 .

When the videoconference server 50 (see FIG. 1) receives image data from each information processing device 10, it arranges the received image data under a common virtual background. Here, since the image data received from each information processing device 10 is an image in which the width and height position of the face of each person (participant) are adjusted, when arranged on a common background, the image data can be unified. It is possible to create a synthetic image that gives a feeling and does not give a sense of incongruity.

The videoconference server 50 will be described below with reference to the drawings.
FIG. 9 is a schematic configuration diagram showing an example of the hardware configuration of the videoconference server 50 according to this embodiment. As shown in FIG. 9, the videoconference server 50 is a computer, and includes a CPU (processor) 51, a main memory 52, a storage section 53, an external interface 54, a communication interface 55, and the like. The videoconference server 50 may also include input devices and displays.
The units described above are connected to each other directly or indirectly via a bus, and cooperate with each other to perform various processes. Since each of these configurations is the same as that of the information processing apparatus 10 described above, detailed description thereof will be omitted here.

FIG. 10 is a functional block diagram showing an example of the functions of the videoconference server 50. As shown in FIG. A series of processes for realizing various functions to be described later are stored, for example, in the form of a program in the storage unit 53 provided in the video conference server 50 , and the program is stored in the main memory 52 by the CPU (processor) 51 . Various functions are realized by reading out and processing and arithmetic processing of information. The program may be pre-installed in the storage unit 53, may be provided in a state stored in another computer-readable storage medium, or may be distributed via wired or wireless communication means. may apply. Computer-readable storage media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, and the like.

As shown in FIG. 10 , the videoconference server 50 includes a receiver 61 , a luminance calculator 62 , a luminance adjuster 63 , an image synthesizer 64 and a transmitter 65 .

The receiving unit 61 receives image data transmitted from each information processing device 10 .
The luminance calculation unit 62 calculates a luminance histogram for each image data received from each information processing device 10 .
The brightness adjustment unit 63 adjusts the brightness of each image data so that variations in the brightness histograms of a plurality of image data are reduced. For example, the brightness of each image data is adjusted so that the brightness at the 50% percentile of the brightness histogram of each image data matches. This makes it possible to reduce variations in luminance between image data.

The image synthesizing unit 64 arranges each image data after brightness adjustment under a common virtual background to create a synthesized image. The process of arranging the image data on the common virtual background may be performed automatically according to a predetermined algorithm, or may be arranged based on an input command from the videoconference host (one of the information processing devices 10). You can do it.
The transmission unit 65 transmits the composite image to each information processing device 10 .

Next, the operation of the video conference system 1 according to this embodiment will be briefly described.
For example, when each user (participant) operates the input device 4 of each information processing device 10 to start a video conference and turn on the camera function, the user's image data is acquired by the camera 7, It is input to the image processing section 20 . The image processing unit 20 executes the image processing described above on image data input from the camera 7 . As a result, the width and height dimensions of the face in the image data are adjusted according to the age of the user, and the adjusted image data is transmitted to the videoconference server 50 .

When the videoconference server 50 receives the image data from each of the information processing devices 10, the videoconference server 50 adjusts the brightness of the received image data, and arranges the image data after the brightness adjustment on a common virtual background to create a composite image. create. Then, the created composite image is transmitted to each information processing apparatus 10 . As a result, on the display 5 of each information processing device 10, for example, as shown in FIG. 11, an image in which each participant is arranged on a common virtual background is displayed. At this time, the size of each participant's face is adjusted to the width according to age, and the brightness of each participant's image is adjusted to reduce variation. As a result, each participant can be provided with an immersive view with a sense of unity as a whole.

As described above, the image processing unit (image processing device) 20, the information processing device 10, the video conference server 50, and the video conference system 1 according to the present embodiment have the following effects.

The image processing unit 20 identifies the face area of the person included in the image data captured by the camera 7, detects the width dimension L1 and the height dimension L2 of the person at a predetermined height position of the identified face area, and calculates the age. The width reference value L1_ref and the height reference value L2_ref are obtained according to the width reference value L1_ref and the height reference value L2_ref, and the image data is adjusted so that the width dimension and the height dimension are close to the obtained width reference value L1_ref and height reference value L2_ref. Further, if there is a missing pixel in the image data after adjustment, the missing pixel is estimated from the surrounding pixel information and supplemented. This makes it possible to output an image in which the size of the face and the height of the person are standardized according to age.

In addition, when adjusting the image, not only the width dimension detected this time, but also a predetermined number of recently detected width dimensions are statistically processed to calculate a representative width dimension, The adjustment ratio is calculated so that the width dimension becomes the width reference value, and the image data is enlarged or reduced using the calculated adjustment ratio. In this way, by calculating the adjustment ratio in consideration of the most recently detected width dimensions, it is possible to smooth the change in the adjustment ratio and suppress the change in the image on the time axis. It becomes possible. Examples of the statistical processing include averaging processing and normalization processing.

In addition, since the age of the person is estimated from the image data and the width reference value and the height reference value are specified based on the estimated age, it is possible to save the user from having to input the age.

In addition, by statistically processing the age estimated this time and the specified number of ages most recently estimated, the representative age is calculated, and the width standard value and height standard value corresponding to the representative age are used as standards. Identify from information. This makes it possible to smoothly change the width reference value.

Since the reference information is provided for each gender, it is possible to adjust the image data so that the size of the person is natural considering the gender.

In addition, in the embodiment described above, both the width of the face and the height of the person are adjusted, but the present invention is not limited to this. For example, only the width of the face may be adjusted, and the height position of the person may not be adjusted.

In this embodiment, a representative width dimension is calculated by statistically processing a predetermined number of width dimensions L1, and an adjustment ratio is calculated using the calculated width dimension and the width reference value L1_ref. It is not limited to this example. For example, the parameter storage unit 25 may be configured to store only one data set, and the adjustment ratio may be calculated using the latest width dimension L1 and width reference value L1_ref.

Similarly, for the height L2 of the person, the image data may be adjusted using the latest height L2 and the height reference value L2_ref.
In this embodiment, a representative age is calculated by statistically processing a predetermined number of ages, and the width reference value and height reference value are specified from the calculated age, but the present invention is not limited to this example. For example, the width reference value and height reference value may be specified based on the latest age.

In this embodiment, the attribute estimating unit 24 estimates the age and sex of the person from the image data, but the present invention is not limited to this example. For example, instead of estimating the age and gender from the image data, the user's age information and gender information registered as user information in the storage unit 13 included in the information processing apparatus 10 may be acquired.

In this embodiment, reference information is provided according to gender, but the present invention is not limited to this example. For example, the reference information is not provided according to gender, and the width reference value and the like may be specified using common reference information regardless of gender.

In this embodiment, the image processing section 20 may be integrated with the camera 7, or may be provided as a camera module in which the camera 7 and the image processing section 20 are integrated. In this case, since the size of the person included in the image data output from the camera module has already been standardized, the information processing apparatus 10 can transfer the image data output from the camera module to the videoconference server. 50.

[Second embodiment]
Next, an image processing device, an information processing device, a video conference server 50a, and a video conference system according to the second embodiment of the present invention will be described.
In the above-described embodiment, each information processing device 10 includes the image processing unit 20. However, in the present embodiment, the image processing unit 20 is provided not by the information processing device but by the videoconference server. Differs from one embodiment. In the following, the same reference numerals are given to the configurations that are common to the above-described first embodiment, and the description thereof will be omitted, and the different points will be mainly described.

FIG. 12 is a functional block diagram showing an example of the functions of the videoconference server 50a according to this embodiment. As shown in FIG. 12, the videoconference server 50a includes an image processing section 20 for adjusting the size and position of the person in the image data from each information processing device 10 received by the receiving section 61. FIG. An example of detailed functions and processing procedures of the image processing unit 20 is as described in the above embodiment.

The image data after adjustment by the image processing unit 20 is output to the luminance calculation unit 62, and processing for luminance adjustment is further performed. Here, either the brightness adjustment or the size adjustment may be performed first. For example, each image data whose brightness has been adjusted by the brightness adjusting section 63 may be input to the image processing section 20 .

Also, the brightness adjustment may be omitted. In this case, in FIG. 12, the configuration is such that the luminance calculation unit 62 and the luminance adjustment unit 63 are omitted. As a result, the image data adjusted by the image processing section 20 is input to the image synthesizing section 64, and the input image data is laid out on a common background.

[Third embodiment]
Next, an image processing device, an information processing device, a video conference server 50b, and a video conference system according to a third embodiment of the present invention will be described.
In the second embodiment described above, the videoconference server 50b uses the reference information to standardize the size and height position of the person in the image data received from each information processing device. The videoconference server 50b differs from the above-described second embodiment in the method of adjusting image data. Specifically, in this embodiment, the videoconference server 50b can acquire image data from each information processing device, and the horizontal width reference value is calculated based on a plurality of image data received from each information processing device. to derive
In the following, the same reference numerals are given to the configurations that are common to the above-described second embodiment, and the description thereof will be omitted, and the different points will be mainly described.

FIG. 13 is a functional block diagram showing an example of the functions of the videoconference server 50b according to this embodiment. As shown in FIG. 13, the videoconference server 50b includes a receiving unit 61, a face area identifying unit 71, a dimension detecting unit 72, an image adjusting unit 73, a brightness calculating unit 62, a brightness adjusting unit 63, an image synthesizing unit 64, and a transmitting unit. A portion 65 is provided.

The receiving unit 61 receives image data transmitted from each information processing device 10 .
The face area specifying unit 71 specifies the face area of the person (participant) included in each image data. Note that the facial area specifying unit 71 has the same function as the above-described facial area specifying unit 22, so details thereof will be omitted.
The dimension detection unit 72 detects the width dimension at a predetermined height position of the specified face area as the width dimension. Thereby, the width dimension of the face in each image data is detected.

The image adjuster 73 obtains a reference width reference value by statistically processing a plurality of width dimensions detected in each image data. For example, the image adjuster 73 acquires the average width dimension as the width reference value. In addition, the image adjustment unit 73 may obtain a predetermined percentile value (for example, 50%) of the distribution (variation) of the width dimension as the width reference value, or obtain the maximum value of the width dimension as the width reference value. You may

Subsequently, the image adjustment unit 73 adjusts each image data so that each width dimension of each image data approaches the width reference value. That is, an adjustment ratio is calculated from the width dimension of the person (participant) in each image data and the width reference value, and the image data is enlarged or reduced based on the calculated adjustment ratio. This makes it possible to reduce variation in the size of a person's face in each image data received from each information processing device 10 .

The adjusted image data is arranged under a common virtual background by the image synthesizing section 64 after the brightness is adjusted by the brightness calculating section 62 and the brightness adjusting section 63 . The composite image is transmitted to each information processing device 10 by the transmission unit 65 .

In the above description, the case of adjusting the width of the face has been described, but not only the width of the face but also the height of the person may be adjusted by a similar method. That is, the height reference value that serves as a reference is acquired by detecting the height dimension of the person in each image data and statistically processing the detected height dimensions. For example, the average height dimension is acquired as the height reference value. Alternatively, a predetermined percentile (for example, 50%) value of the height dimension distribution (variation) may be obtained as the height reference value, or the maximum value of the height dimension may be obtained as the height reference value. good too.

Then, each image data is adjusted so that each height dimension in each image data approaches the height reference value. Specifically, the height position of the person in the image data is adjusted so that the height dimension of the person (participant) in each image data matches the height reference value. This makes it possible to reduce variation in the height position of the person in each image data received from each information processing device 10 .

Note that the video conference server 50b according to the third embodiment can be used in combination with the information processing apparatus 10 according to the first embodiment described above. In this case, already standardized image data is received from each information processing apparatus 10 . Further, by performing the above-described image adjustment in the video conference server 50b, it is possible to unify the horizontal width and height position of the person in each image data.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be made to the above-described embodiments without departing from the gist of the invention, and forms with such changes or improvements are also included in the technical scope of the present invention.
Further, the flow of processing described in the above embodiment is also an example, and unnecessary steps may be deleted, new steps added, or the processing order changed without departing from the scope of the present invention. good.

For example, in each of the embodiments described above, a videoconference server is provided, but all or part of the functions of the videoconference server may be provided in the information processing apparatus 10 serving as the host of the videoconference.

1: video conference system 4: input device 5: display 7: camera 8: network 10: information processing device 11: CPU
12: main memory 13: storage unit 14: external interface 15: communication interface 16: speaker 17: microphone 20: image processing unit 21: image acquisition unit 22: face area identification unit 23: dimension detection unit 24: attribute estimation unit 25: Parameter storage unit 26 : Reference information storage unit 27 : Reference value identification unit 28 : Image adjustment unit 29 : Output unit 50 : Video conference server 50a : Video conference server 50b : Video conference server 51 : CPU
52 : Main memory 53 : Storage unit 54 : External interface 55 : Communication interface 61 : Reception unit 62 : Luminance calculation unit 63 : Luminance adjustment unit 64 : Image composition unit 65 : Transmission unit 71 : Face area identification unit 72 : Dimension detection unit 73: Image adjustment unit L1: Width dimension L1_ref: Width reference value L2: Height dimension L2_ref: Height reference value

Claims (15)

a processor;
a memory storing a program configured to be executed by the processor;
with
The program
get the image data,
identifying a person's face area included in the image data;
detecting a width dimension at a predetermined height position of the identified face area as a width dimension;
identifying a width reference value corresponding to the age of the person from reference information in which the age and the width reference value of the face area are associated;
adjusting the image data so that the width dimension approaches the specified width reference value;
An image processing device including instructions for outputting the adjusted image data. an image acquisition unit that acquires image data;
a face area identification unit that identifies a person's face area included in the image data;
a dimension detection unit that detects the width dimension at a predetermined height position of the identified face area as the width dimension;
a reference value identification unit that identifies a width reference value corresponding to the age of the person from reference information in which the age and the width reference value of the face area are associated;
an image adjustment unit that adjusts the image data so that the width dimension approaches the specified width reference value;
and an output unit that outputs the adjusted image data. The image adjustment unit calculates a representative width dimension by statistically processing the width dimension detected this time and a predetermined number of the width dimensions detected most recently, and the representative width dimension is the 3. The image processing apparatus according to claim 2, wherein an adjustment ratio that provides a horizontal width reference value is calculated, and the image data is enlarged or reduced using the calculated adjustment ratio. an attribute estimation unit that estimates the age of the person from the image data;
4. The image processing apparatus according to claim 2, wherein the reference value specifying unit specifies the width reference value corresponding to the estimated age from the reference information. The reference value specifying unit calculates a representative age by statistically processing the currently estimated age and a predetermined number of ages most recently estimated, and calculates the width reference value corresponding to the representative age. is specified from the reference information. 4. The image processing apparatus according to claim 2, wherein the reference information is provided according to gender. An attribute estimation unit that estimates the age and gender of the person from the image data,
7. The image processing apparatus according to claim 6, wherein the reference value specifying unit specifies the width reference value corresponding to the estimated age and sex from the reference information. The dimension detection unit detects a height dimension of a person in the image data as a height dimension,
The reference information is associated with an age, a width reference value, and a person's height reference value,
The reference value specifying unit specifies a width reference value and a height reference value corresponding to the age of the person from the reference information,
The image processing apparatus according to any one of claims 2 to 7, wherein the image adjustment unit adjusts the image data so that the height dimension approaches the specified height reference value. 9. The image processing according to claim 8, wherein, in the image data after adjustment, if there is a shortage of pixels in the height direction, the image adjustment unit estimates and replenishes the missing pixels from surrounding pixel information. Device. A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 9. An information processing apparatus comprising the image processing apparatus according to claim 1 . obtaining image data;
identifying a person's face area included in the image data;
a step of detecting a width dimension at a predetermined height position of the identified face area as the width dimension;
identifying a width reference value corresponding to the age of the person from reference information in which the age and the width reference value of the face area are associated;
adjusting the image data so that the width dimension approaches the specified width reference value;
and outputting the adjusted image data. a plurality of information processing devices;
a video conference server that receives the adjusted image data from the plurality of information processing devices;
Each information processing device comprises the image processing device according to any one of claims 1 to 9,
A video conference system in which the video conference server includes an image synthesizing unit that arranges the image data received from each of the information processing devices under a common background. a receiver for receiving image data of a plurality of participants in a video conference;
a facial area identifying unit that identifies the facial area of the participant included in each of the image data;
a dimension detection unit that detects the width dimension at a predetermined height position of the identified face area as the width dimension;
By statistically processing a plurality of width dimensions detected in each of the image data, a width reference value that serves as a reference is calculated, and each of the width dimensions in each of the image data is brought closer to the width reference value, an image adjusting unit that adjusts each of the image data;
an image synthesizer for placing each of said image data on a common virtual background. a luminance calculation unit for calculating a luminance histogram of each image data;
a luminance adjustment unit that adjusts the luminance of each of the image data so that variations in the luminance histogram between the plurality of image data are reduced;
15. The videoconference server according to claim 14, wherein the image synthesizing unit arranges each of the image data after brightness adjustment under the common background.

Images