JP2012182688A - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of reducing processing load on a communication apparatus when the communication apparatus transmits data for a plurality of images captured by a plurality of imaging means to execute a teleconference, and an image processing method and an image processing program.SOLUTION: A communications system comprises: a PC which transmits data via a network; and a plurality of teleconference terminals connected to the PC in series. Each one of the teleconference terminals acquires the number of the other teleconference terminals connected to the PC in a row. A teleconference terminal acquires image data for the terminal itself from a camera which captures an image (S60), and acquires image data for other devices from other teleconference terminals connected downstream (S61). The acquired image data for the terminal itself and that for the other devices are combined to generate composite image data (S62). The teleconference terminal outputs the generated composite data to the PC (S65).

Description

  The present invention relates to an image processing device, an image processing method, and an image processing program for processing image data transmitted from a communication device via a network to execute a video conference.

  2. Description of the Related Art Conventionally, a technique for reducing the processing burden on a data output destination device when outputting image data captured by an imaging unit to another device is known. For example, the imaging apparatus described in Patent Document 1 acquires memory size information from a printer when outputting image data to a printer that prints an image. Based on the acquired memory size, the size of the image data that can be output to the printer is calculated. If the size of the image data to be output is larger than the size that can be output, the imaging apparatus recompresses the compressed image data and outputs the compressed image data to the printer. As a result, processing for printing an image is appropriately performed.

JP 2006-101220 A

  In a communication system for executing a video conference, image data is transmitted and received between a plurality of communication devices, so that images of a plurality of bases are shared in the system. When a single communication device transmits data of a plurality of images captured by a plurality of imaging means, the communication device needs to combine a plurality of images and transmit the combined image data. Therefore, the processing load on the communication device is increased, and the image displayed during the video conference may be disturbed. In the prior art, it has not been possible to reduce the processing load on the communication device when transmitting data of a plurality of images captured by a plurality of imaging means.

  The present invention relates to an image processing apparatus and an image processing method capable of reducing the processing burden on a communication apparatus when the communication apparatus transmits data of a plurality of images captured by a plurality of image capturing means to execute a video conference. And an image processing program.

  An image processing apparatus according to a first aspect of the present invention is used in a communication system including a communication device that transmits data via a network and a plurality of image processing devices connected in series to the communication device. An image processing apparatus capable of outputting image data of an image to be displayed at another base to the communication apparatus, and obtaining the number of other image processing apparatuses connected to the communication apparatus. Acquisition means; first image data acquisition means for acquiring first image data from an imaging means for capturing an image; and second image data for acquiring second image data from another image processing apparatus connected to the apparatus. An acquisition unit, the first image data acquired by the first image data acquisition unit and the second image data acquired by the second image data acquisition unit are transferred to the number acquisition unit. Generating means for generating composite image data by combining them according to the number obtained, and outputting the composite image data generated by the generation means to another connected image processing apparatus or the communication apparatus Output means.

  The image processing apparatus according to the first aspect generates composite image data by combining image data of images picked up by a plurality of image pickup means. The generated composite image data can be output to another image processing apparatus or communication apparatus. The communication device that has input the composite image data from the image processing device does not need to perform processing for combining the image data. Therefore, the image processing apparatus can reduce the processing load on the communication apparatus, and can reduce the possibility of communication disruption during a video conference.

  The image processing apparatus includes: a connection order information acquisition unit that acquires a connection order of the own apparatus with respect to the communication apparatus among the own apparatus connected to the communication apparatus and another image processing apparatus; There may be further provided table acquisition means for acquiring, from the storage means, a table that associates the coordinate information indicating the combination position of the first image data at the coordinates of the first image data and the connection order of the own apparatus. In the table acquired by the table acquisition unit, the generation unit combines the first image data at the combination position indicated by the coordinate information corresponding to the connection order of the own apparatus acquired by the connection order information acquisition unit. Also good. The image processing apparatus can synthesize the first image data at an appropriate synthesis position according to the connection order of the own apparatus with respect to the communication apparatus. The communication apparatus does not need to instruct the image processing apparatus to synthesize the first image data. Therefore, the image processing apparatus can further reduce the processing load on the communication apparatus.

  The image processing device is information set when a user inputs an operation to a template displayed on a display unit, and a first image of each image processing device specified by the user at coordinates on the template You may further provide the setting information acquisition means which acquires the coordinate information which shows the synthetic | combination position of data. The generating unit may combine the first image data at a combining position indicated by the coordinate information acquired by the setting information acquiring unit. In this case, the user can freely determine the layout of the image captured by each imaging unit.

  The second image data acquisition unit may acquire the second image data subjected to transform coding and quantization from the other connected image processing apparatus. The image processing apparatus determines whether the own apparatus is connected to the communication apparatus without going through another image processing apparatus, and the communication means without using the other image processing apparatus by the judging means. A compression processing unit that performs a compression process including a predictive coding process on the synthesized image data generated by the generation unit when it is determined that the apparatus is connected to the apparatus may be further provided. The output means may output the composite image data processed by the compression processing means to the communication device. Since the image processing apparatus acquires the second image data that has been subjected to transform coding and quantization, it can prevent the processing load from increasing by acquiring the second image data having a large amount of data. Furthermore, the image processing apparatus performs a compression process including a predictive encoding process only when outputting the composite image data to the communication apparatus. Therefore, it is not necessary to decode the second image data when generating the composite image data, and the processing can be performed efficiently. Since the communication device inputs composite image data that has already undergone compression processing, the processing load on the communication device is further reduced.

  The image processing device includes: output information acquisition means for acquiring the resolution and frame rate of the composite image data output to the communication device from the communication device; and the resolution and frame rate acquired by the output information acquisition means for the communication Notification means for notifying other image processing apparatuses connected to the apparatus may be further provided. The generation unit may generate composite image data according to the resolution and frame rate acquired by the output information acquisition unit, or the resolution and frame rate notified by a notification unit of another image processing apparatus. In this case, the image processing apparatus can share the resolution and frame rate required by the communication apparatus with other image processing apparatuses, and can output appropriate composite image data to the communication apparatus.

  An image processing method according to a second aspect of the present invention is used in a communication system including a communication device that transmits data via a network and a plurality of image processing devices connected in series to the communication device. An image processing method executed by an image processing apparatus capable of outputting image data of an image to be displayed at another base to the communication apparatus, wherein the other image processing is connected to the communication apparatus. A number acquisition step for acquiring the number of devices, a first image data acquisition step for acquiring first image data from an imaging means for capturing an image, and second image data from another image processing device connected to the device itself In the second image data acquisition step, the first image data acquired in the first image data acquisition step, and the second image data acquisition step. Connecting the acquired second image data to the generation step of generating composite image data by combining the acquired second image data according to the number of units acquired in the number acquisition step, and the composite image data generated in the generation step; Output to the other image processing apparatus or the communication apparatus.

  According to the image processing method according to the second aspect, the image processing apparatus generates combined image data by combining image data of images captured by a plurality of imaging units. The generated composite image data can be output to another image processing apparatus or communication apparatus. The communication device that has input the composite image data from the image processing device does not need to perform processing for combining the image data. Therefore, the image processing apparatus can reduce the processing load on the communication apparatus, and can reduce the possibility of communication disruption during a video conference.

  An image processing program according to a third aspect of the present invention is used in a communication system including a communication device that transmits data via a network and a plurality of image processing devices connected in series to the communication device. An image processing program used in an image processing apparatus capable of outputting image data of an image to be displayed at another base to the communication apparatus, and the other image processing apparatus connected to the communication apparatus A number acquisition step for acquiring the number of images, a first image data acquisition step for acquiring first image data from an imaging means for capturing an image, and second image data from another image processing device connected to the device itself Second image data acquisition step to be acquired, first image data acquired in the first image data acquisition step, and second image data acquisition step The second image data acquired in step S2 is combined with the number acquired in the number acquisition step to generate combined image data, and the combined image data generated in the generation step is An instruction for causing the controller of the image processing apparatus to execute an output step of outputting to another connected image processing apparatus or the communication apparatus.

  According to the image processing program according to the third aspect, the image processing apparatus generates composite image data by combining image data of images captured by a plurality of imaging units. The generated composite image data can be output to another image processing apparatus or communication apparatus. The communication device that has input the composite image data from the image processing device does not need to perform processing for combining the image data. Therefore, the image processing apparatus can reduce the processing load on the communication apparatus, and can reduce the possibility of communication disruption during a video conference.

1 is a diagram showing a system configuration of a communication system 100. FIG. It is a block diagram which shows the electrical structure of the video conference terminal 1 and PC3. It is a figure which shows an example of the image displayed on the display apparatus 46 during a video conference. It is a flowchart of the PC conference process which PC3 performs. It is a figure which shows an example of the display apparatus 46 which is displaying the template 51 for image arrangement input. It is explanatory drawing for demonstrating the data structure of layout information. It is a flowchart of the terminal conference process which the video conference terminal 1 performs. It is a flowchart of the highest terminal process performed during a terminal conference process. It is explanatory drawing for demonstrating the data structure of the synthetic | combination position table memorize | stored in the flash memory 13 of the video conference terminal 1. FIG. It is a flowchart of the low-order terminal process performed during a terminal conference process.

  Hereinafter, a video conference terminal 1, which is an embodiment of an image processing apparatus according to the present invention, will be described with reference to the drawings. The drawings to be referred to are used for explaining technical features that can be adopted by the present invention. The configuration of the apparatus, the flowcharts of various processes, and the like described in the drawings are not intended to be limited to these, but are merely illustrative examples.

  A schematic configuration of a communication system 100 including a video conference terminal 1 will be described with reference to FIG. The communication system 100 includes a plurality of communication devices that can transmit and receive data via the network 8. In this embodiment, a personal computer (hereinafter referred to as “PC”) 3 is used as a communication device, but a device dedicated to video conferencing that can transmit and receive data may be used as a communication device.

  The video conference terminal 1 can be connected to the PC 3. The video conference terminal 1 inputs the voice of the installed base from the microphone 26 (see FIG. 2) and inputs the image from the camera 28 (see FIG. 2). The input voice and image data is output to the PC 3. Further, the video conference terminal 1 can generate the sound of another base from the speaker 27 (see FIG. 2) based on the sound data input from the PC 3.

  A plurality of video conference terminals 1 can be connected to the PC 3. Therefore, as shown in FIG. 1, when a plurality of users gather at one base and perform a video conference, each of the plurality of video conference terminals 1 can be arranged in the vicinity of each user. As a result, a clear image of each user is displayed at the other base, and the speech voice of each user is output accurately at the other base.

  In the present embodiment, when a plurality of video conference terminals 1 are connected to one PC 3, the plurality of video conference terminals 1 are connected to the PC 3 in series. In the example shown in FIG. 1, a video conference terminal 1 </ b> A is connected to the PC 3. A video conference terminal 1B is connected to the video conference terminal 1A, and a video conference terminal 1C is further connected to the video conference terminal 1B. Accordingly, data input from the video conference terminal 1C from its own microphone 26 and camera 28 is output to the PC 3 via the video conference terminal 1B and the video conference terminal 1A. Similarly, audio and image data input by the video conference terminal 1B is output to the PC 3 via the video conference terminal 1A. The PC 3 transmits data input from the plurality of video conference terminals 1 </ b> A to 1 </ b> C to the PC 3 at another base via the network 8. As a result, the voices and images of users A, B, and C are output at other locations.

  As described above, by connecting a plurality of video conference terminals 1 to one PC 3, accurate speech sounds and clear images of each of a plurality of users are output at other bases. On the other hand, in the conventional technique, the PC 3 has to synthesize a plurality of image data input from the plurality of video conference terminals 1. As a result, the processing burden on the PC 3 has increased. When the processing load on the PC 3 increases, data communication, audio output, image display, and the like may be disturbed, and the video conference may not be performed smoothly. The video conference terminal 1 according to the present embodiment can smoothly execute a video conference by reducing the processing load on the PC 3.

  In the following description, among the plurality of video conference terminals 1 connected in series to the PC 3, a terminal closer to the PC 3 is referred to as “upper” and a terminal farther from the PC 3 is referred to as “lower”. A video conference terminal 1 (for example, the video conference terminal 1A in FIG. 1) directly connected to the PC 3 without passing through another video conference terminal 1 is referred to as a “top-level terminal”. The video conference terminal 1 farthest from the PC 3 (for example, the video conference terminal 1C in FIG. 1) is referred to as a “lowest terminal”. The video conference terminal 1 (for example, the video conference terminal 1B in FIG. 1) connected to the upper and lower two video conference terminals 1 is referred to as an “intermediate terminal”.

  The electrical configuration of the video conference terminal 1 and the PC 3 will be described with reference to FIG. The video conference terminal 1 includes a CPU 10 that controls the video conference terminal 1. A ROM 11, a RAM 12, a flash memory 13, and an input / output interface 14 are connected to the CPU 10 via a bus 19.

  The ROM 11 stores a program for operating the video conference terminal 1, an initial value, and the like. A program for executing a terminal conference process (see FIG. 7 and the like) described later is stored in the ROM 11. The RAM 12 temporarily stores various information used in the control program. The flash memory 13 is a nonvolatile storage device. A composite position table (see FIG. 9) to be described later is stored in the flash memory 13. Instead of the flash memory 13, a storage device such as an EEPROM or a memory card may be used.

  A USB device I / F 21, an audio input processing unit 22, an audio output processing unit 23, a video input processing unit 24, a USB host I / F 25, and an operation unit 29 are connected to the input / output interface 14. The USB device I / F 21 is connected to the USB host I / F 43 of the PC 3 or the USB host I / F 25 of another video conference terminal 1, and outputs the voice and image data of the local site to the connected device. When the video conference terminal 1 is connected to another device by the USB device I / F 21, the other connected device takes the lead in the USB communication protocol. On the other hand, the USB host I / F 25 is connected to the USB device I / F 21 of the lower video conference terminal 1 and inputs the voice and image data of its own site from the connected lower video conference terminal 1. When connected to another device by the USB host I / F 25, the own device takes the lead in the USB communication protocol. Needless to say, various signals can be input / output to / from other devices by the USB device I / F 21 and the USB host I / F 25. The voice input processing unit 22 processes voice data input from the microphone 26. The audio output processing unit 23 processes the operation of the speaker 27. The video input processing unit 24 processes input of image data from the camera 28. The operation unit 29 includes a power button, a volume control button, a microphone mute button, and the like, and accepts various operation inputs from the user.

  The PC 3 includes a CPU 30 that controls the PC 3. A ROM 31, a RAM 32, a hard disk drive (hereinafter referred to as “HDD”) 33, and an input / output interface 34 are connected to the CPU 30 via a bus 39. The ROM 31 stores a program for operating the PC 3, initial values, and the like. The RAM 12 temporarily stores various information. The HDD 13 is a non-volatile storage device.

  A video output processing unit 41, an external communication I / F 42, a USB host I / F 43, and an operation unit 47 are connected to the input / output interface 34. The video output processing unit 41 processes the operation of the display device 46 that displays video. The external communication I / F 42 connects the PC 3 to the network 8. The USB host I / F 43 connects the PC 3 to the video conference terminal 1 (top-level terminal) via a USB cable. The operation unit 47 includes a keyboard and receives various operation inputs.

  With reference to FIG. 3, an example of an image displayed on the display device 46 of the PC 3 while the video conference is executed in the communication system 100 will be described. In the example shown in FIG. 3, three users A, B, and C gather at the base X, and one user D exists at the base Y. Three video conference terminals 1 are connected to the PC 3 at the base X, and each video conference terminal 1 images each of the three users A, B, and C. The PC 3 at the site X displays on the display device 46 a composite image 48 in which the three images captured by the three video conference terminals 1 are combined. Further, the image 49 of the user D received from the PC 3 at the site Y is displayed on the display device 46. The PC 3 at the site X transmits the image data of the composite image 48 to the PC 3 at the site Y. Therefore, an image similar to the image shown in FIG. 3 is also displayed on the display device 46 of the PC 3 at the site Y. Note that the PC 3 may project the image illustrated in FIG. 3 on a projector to execute a video conference, or may display the image on another display device connected to the PC 3.

  The video conference terminal 1 according to the present embodiment can generate a composite image data by combining a plurality of images captured by the plurality of video conference terminals 1, and can output the generated composite image data to the PC 3. Therefore, the video conference terminal 1 can reduce the processing load on the PC 3. In addition, the plurality of video conference terminals 1 can share coordinate information indicating the synthesis position in the synthesized image data of the image captured by each video conference terminal 1 and synthesize the images at appropriate positions. Furthermore, the video conference terminal 1 can output suitable composite image data corresponding to the resolution and frame rate requested from the PC 3 to the PC 3. Details of the processing will be described below.

  The PC conference process executed by the PC 3 will be described with reference to FIGS. An application program for executing a video conference is installed in the HDD 33 of the PC 3. When the CPU 30 of the PC 3 inputs an instruction for starting an application from the operation unit 47, the PC conference process shown in FIG. 4 is executed.

  As shown in FIG. 4, when the PC conference process is started, a number survey command is output to the connected highest terminal (S1). The number survey command is a command for instructing the number of video conference terminals 1 connected to the PC 3 to be surveyed. Although details will be described later, when the video conference terminal 1 inputs a number survey command from the PC 3, the number of the video conference terminals 1 is surveyed and notified to the PC 3. The CPU 10 of the PC 3 acquires the number from the highest terminal and stores it in the RAM 32 (S2).

  Next, it is determined whether or not a layout designation command has been input to the operation unit 47 (S3). The layout designation command is input by the user when the user himself designates the layout of the composite image. If the layout designation command has not been input (S3: NO), it is determined whether or not an instruction to start the conference using the default layout has been input (S4). If not input (S4: NO), the process returns to the determination of S3.

  When the layout designation command is input (S3: YES), the image layout input template is activated and displayed on the display device 46 (S6). FIG. 5 shows an example of the display device 46 that displays the image layout input template 51. The image layout input template is a template for accepting designation of the layout of the composite image from the user. In S6, a template corresponding to the number acquired in S2 and the resolution of the image adopted in the video conference is activated. The image layout input template 51 illustrated in FIG. 5 is a template when the number of the video conference terminals 1 is three and the adopted resolution is VGA (640 × 480). In this case, “position of image 1” 52 indicating the position of the captured image of the highest terminal, “position of image 2” 53 indicating the position of the captured image of the intermediate terminal, and “image” indicating the position of the captured image of the lowest terminal. 3 position "54 is displayed. Furthermore, the numerical value of the coordinates with the upper left corner as the origin is displayed. The user designates the position and size of each image by operating a pointer (not shown) displayed on the display device 46 with the operation unit 47. When confirming the designated layout, the confirm button 57 is operated. The hatched portion 55 is a blank portion where a captured image is not displayed.

  The image layout input template is continuously activated until the confirmation button 57 is operated (S7: NO). When the confirmation button 57 is operated and the designation of the layout is completed (S7: YES), layout information is set and output to the highest terminal (S8). As shown in FIG. 6, the layout information indicates a composite position on the composite image data of each image captured by each video conference terminal 1. Specifically, a coordinate value indicating a composite position of each image at coordinates on the composite image data is set and output as layout information.

  When the output of layout information (S8) is completed, or when an instruction to start a conference using the default layout is input (S4: YES), the conference start instruction is output to the highest terminal (S10). ). The resolution and frame rate of the image used in the video conference are output to the highest terminal (S11). The PC conference process ends. In addition, CPU30 of PC3 will start the process which transmits / receives the image and audio | speech data between other PC3, if PC meeting process is complete | finished. Since this process is a general process, description thereof is omitted.

  A terminal conference process executed by the video conference terminal 1 will be described with reference to FIGS. The ROM 11 of the video conference terminal 1 stores an image processing program for outputting image data to the PC 3 during the video conference. The CPU 10 of the video conference terminal 1 executes the terminal conference process shown in FIG. 7 when the power is turned on.

  As shown in FIG. 7, when the terminal conference process is started, it is determined whether or not a number survey command is input from the PC 3 (S21). If it has not been input (S21: NO), it is determined whether or not a number survey command has been input from the video conference terminal 1 connected to the host (S22). If not input (S22: NO), the determinations of S21 and S22 are repeated.

  When the own apparatus is the highest terminal and is connected to the PC 3 without passing through the other video conference terminals 1, a number survey command is output from the PC 3 at the start of the video conference. Therefore, when the number survey command is directly input from the PC 3 (S21: YES), the CPU 10 recognizes the own device as the highest terminal and sets the connection number of the own device to “1” (S23). The connection number is a number indicating the connection order of the own apparatus with respect to the PC 3 among the plurality of video conference terminals 1 connected in series with the PC 3. Although the details will be described later, the video conference terminal 1 recognizes the combined position of the data of the image captured by the camera 28 of its own device according to the connection number. In the present embodiment, the connection numbers are set so that the connection numbers increase in order from the higher-level video conference terminal 1. Specifically, in the case of direct connection to the PC 3, the connection order is “1” and the connection number is “1”. When connected to the PC 3 via one video conference terminal 1 (the highest terminal), the connection order is “2” and the connection number is “2”. When the number survey command is input from the video conference terminal 1 connected to the host (S22: YES), the process proceeds to S24 as it is.

  When the number survey command is input, the number survey command is output to the video conference terminal 1 connected to the lower level (S24). It is determined whether or not there is a reply of the number of lower terminals in response to the output number survey command (S26). When there is no answer (S26: NO), the video conference terminal 1 is not connected to the lower level of the own device. Accordingly, the own apparatus is recognized as the lowest terminal (S27). Further, it is output to the upper video conference terminal 1 that the number of lower terminals is only one of the lowest terminals (S28). Next, the process waits until the total number is input from the higher-level video conference terminal 1 (S33: NO). Although details will be described later, the total number is sequentially input from the highest terminal to the lower video conference terminal 1. When the total number is input (S33: YES), a value obtained by subtracting the number of terminals lower than the own device from the total number is set as the connection number of the own device (S34). That is, if the own device is the lowest terminal, the number of terminals lower than the own device is “0”, so the total number is equal to the connection number of the own device. Next, it is determined whether or not the own device is the lowest terminal (S39). If it is the lowest terminal (S39: YES), the process directly proceeds to the determination of S41.

  When there is an answer to the number survey command (S26: YES), the number of lower terminals input from the lower video conference terminal 1 is recognized and stored in the RAM 12 (S30). Next, it is determined whether or not the device is the highest terminal (S31). If the own apparatus is not the highest terminal (S31: NO), the number obtained by adding “1” to the number of lower terminals input from the lower video conference terminal 1 is the number of lower terminals in the higher video conference terminal 1. The video is output to the upper video conference terminal 1 (S32). When the total number is input from the video conference terminal 1 (S33: YES), a value obtained by subtracting the number of lower terminals of the own device from the total number is set as the connection number of the own device (S34). If the own apparatus is not the lowest terminal (S39: NO), the total number input from the upper video conference terminal 1 is directly output (transferred) to the lower video conference terminal 1 (S40). The process proceeds to the determination in S41.

  When the own apparatus is the highest terminal (S31: YES), the number obtained by adding “1” to the number of lower terminals input from the lower video conference terminal 1 is recognized as the total number (S36). The recognized total number is output to the PC 3 and the lower video conference terminal 1 connected to the own apparatus (S37). As described above, since the total number is notified to the lowest terminal in order, all of the video conference terminals 1 connected to the PC 3 can recognize the total number. The process proceeds to the determination in S41.

  Next, it is determined whether or not a conference start instruction is input from the PC 3 or the higher-level video conference terminal 1 (S41), and a standby state is entered until it is input (S41: NO). When a conference start instruction is input (S41: YES), it is determined whether or not the device is the highest terminal (S42). If it is the highest terminal (S42: YES), the highest terminal process is performed (S43), and the process ends. If the own apparatus is not the highest terminal (S42: NO), the lower terminal process is performed (S44), and the process ends.

  With reference to FIG. 8, the top terminal processing will be described. When the highest terminal process is started, the resolution and frame rate employed in the video conference are input from the PC 3 (S51). The resolution and the frame rate are output by the PC 3 together with the conference start instruction (see FIG. 4, S10 and S11). The input resolution, frame rate, and conference start instruction are transferred to the video conference terminal 1 connected to the lower level (S52).

  It is determined whether layout information (see FIG. 6) has been input from the PC 3 (S53). As described above, when the user designates the image layout using the image layout input template, the PC 3 outputs layout information indicating the composite position of each image on the composite image data to the highest terminal. When layout information is input (S53: YES), the input layout information is transferred to the lower video conference terminal 1 (S54). Among the composite positions indicated by the layout information, the composite position corresponding to the connection number “1” of the highest terminal is recognized as the composite position of the image captured by the own device (S55).

  When the layout information has not been input (S53: NO), the user has input an instruction to start the conference using the default layout. In this case, the composite position table stored in the flash memory 13 is acquired (S56). As shown in FIG. 9, in the composite position table, coordinate information indicating the composite position of each image at coordinates on the composite image data is associated with the connection order (connection number) of the video conference terminal 1. In the present embodiment, there are provided a table when the resolution is VGA and a table when the resolution is XGA. For each of the total number of video conference terminals 1 connected to the PC 3, coordinate information indicating the combined position of each terminal is associated with the connection order. The CPU 10 of the video conference terminal 1 acquires coordinate information corresponding to the total number, resolution, and connection order of the own device. The synthesis position indicated by the acquired coordinate information is recognized as a synthesis position for synthesizing image data captured by the own device (S57).

  Next, data of the image captured by the camera 28 of the own device (own device image data) is acquired (S60). In S60, the self image data of the resolution input from the PC 3 in S51 is acquired. Further, the own apparatus image data is acquired according to the frame rate input in S51. Next, image data (other device image data) is acquired from the video conference terminal 1 connected to the lower level (S61). The other apparatus image data acquired in S61 is image data of an image captured by the lower video conference terminal 1 if only one video conference terminal 1 is connected to the lower level. On the other hand, if there are a plurality of video conference terminals 1 connected to the lower level, a composite image of a plurality of images captured by the lower level video conference terminals 1 is obtained. Although details will be described later, the resolution and frame rate of the other apparatus image data are the same as the resolution and frame rate of the own apparatus image data acquired in S60. Further, DCT and quantization have already been performed on the other apparatus image data. Next, the own device image data is synthesized at the position recognized in S55 or S57 among the other device image data (S62). As a result, composite image data is generated.

  In the highest terminal, a predictive encoding process is performed on the generated composite image data (S63). Further, Huffman coding is performed (S64). Since the highest terminal performs the predictive coding and the Huffman coding to compress the data, the PC 3 does not need to perform these processes. In S64, the CPU 10 may perform entropy coding other than Huffman coding. Only either predictive coding or entropy coding may be performed. The encoded composite image data is output to the PC 3 connected to the own apparatus (S65). Until a conference end instruction is input from the operation unit 29 or the like (S66: NO), the processing of S60 to S66 is repeated. When the conference end instruction is input (S66: YES), the process ends.

  The lower terminal processing will be described with reference to FIG. When the lower terminal processing is started, the resolution and the frame rate are input from the video conference terminal 1 connected to the upper terminal (S71). If the video conference terminal 1 is connected to the lower level (the device itself is not the lowest level terminal), the input resolution / frame rate and the conference start instruction are transferred to the lower level video conference terminal 1 (S72). ).

  It is determined whether layout information (see FIG. 6) has been input from the video conference terminal 1 connected to the host (S73). When the layout information is input (S73: YES), if the own apparatus is not the lowest terminal, the layout information is transferred to the lower video conference terminal 1 (S74). Among the composite positions indicated by the layout information, the composite position corresponding to the connection number of the self apparatus is recognized as the composite position of the self apparatus image (S75). When the layout information is not input (S73: NO), the composite position table (see FIG. 9) is acquired from the flash memory 13 (S76). The coordinate information corresponding to the total number of units, the resolution, and the connection order of the own apparatus is acquired from the acquired combined position table. The position indicated by the acquired coordinate information is recognized as the composite position of the own apparatus image (S77).

  Next, own device image data is acquired from the camera 28 of the own device (S79). In S79, the apparatus image data is acquired according to the resolution and frame rate acquired in S71. That is, a common resolution and frame rate are used among the plurality of video conference terminals 1. If the own device is the lowest terminal (S80: YES), the own device image data is arranged at the position recognized in S75 or S77 (S81), and discrete cosine transform (DCT) and quantization are performed on this image data. Processing is performed (S85). The image data subjected to DCT and quantization is output to the video conference terminal 1 connected to the host (S86). By performing DCT and quantization before outputting image data, the processing burden on the upper video conference terminal 1 and the PC 3 can be reduced. Furthermore, in the present embodiment, predictive coding and Huffman coding are performed only at the highest terminal. Therefore, the intermediate terminal and the highest terminal can synthesize their own device image data with the other device image data without decoding the other device image data. In S85, other transform coding processing such as discrete Fourier transform (DFT) may be performed, or only transform coding or quantization may be performed.

  If the own apparatus is not the lowest terminal but an intermediate terminal (S80: NO), the other apparatus image data is acquired from the video conference terminal 1 connected to the lower terminal (S83). Among the acquired other device image data, the own device image data is synthesized at the position recognized in S75 or S77 (S84). As a result, composite image data is generated. The generated composite image data is subjected to DCT and quantization (S85), and the composite image data is output to the upper video conference terminal 1 (S86). Until a conference end instruction is input from the operation unit 29 or the like (S87: NO), the processes of S79 to S87 are repeated. When the conference end instruction is input (S87: YES), the process ends.

  As described above, the video conference terminal 1 according to the present embodiment generates composite image data by combining the image data of a plurality of images captured by the plurality of cameras 28. The generated composite image data can be output to the upper video conference terminal 1 or the PC 3. The PC 3 that has input the composite image data from the highest terminal does not need to perform a process of combining the image data. Therefore, the video conference terminal 1 can reduce the processing load of the PC 3, and can reduce the possibility that communication during the video conference is disturbed.

  The video conference terminal 1 can easily recognize the composite position of the image data (self apparatus image data) captured by the self apparatus according to the combination position table and the connection order of the self apparatus. The apparatus image data can be synthesized at the appropriate recognized position. The PC 3 does not need to instruct each video conference terminal 1 to synthesize the image data. Therefore, when the composite position table is used, the video conference terminal 1 can further reduce the processing load on the PC 3.

  The video conference terminal 1 can recognize the composite position of its own device image data based on the layout information set by the user operating the operation unit 47 of the PC 3. In this case, the user can freely determine the layout of the image captured by each video conference terminal 1. The video conference terminal 1 can synthesize the own device image data at an appropriate position indicated by the layout information.

  When the video conference terminal 1 operates as an intermediate terminal or the lowest terminal, the video conference terminal 1 outputs image data subjected to transform coding and quantization to a higher video conference terminal. Therefore, it is possible to prevent an upper video conference terminal from acquiring image data having a large amount of data and increasing the processing load. In addition, predictive coding and entropy coding are not performed on the other apparatus image data input by the higher-level video conference terminal 1. Therefore, the video conference terminal 1 does not need to decode the other device image data when generating the composite image data. Further, when the video conference terminal 1 operates as the highest terminal, the video conference terminal 1 outputs the composite image data subjected to the predictive coding and the entropy coding to the PC 3. Therefore, the processing burden on the PC 3 can be further reduced.

  When the video conference terminal 1 operates as the highest-level terminal, the video conference terminal 1 acquires the resolution and the frame rate from the PC 3 and notifies the lower video conference terminal 1. Each video conference terminal 1 can generate composite image data according to the resolution and frame rate acquired from the PC 3. Accordingly, the video conference terminal 1 can share appropriate resolution and frame rate with other video conference terminals 1 and output appropriate composite image data to the PC 3.

  In the above embodiment, the video conference terminal 1 corresponds to the “image processing apparatus” of the present invention. The PC 3 corresponds to the “communication device” of the present invention. The CPU 10 that acquires the total number in the terminal conference process of FIG. 7 functions as a “number acquisition unit”. The own apparatus image data corresponds to the “first image data” of the present invention. The CPU 10 that acquires the apparatus image data in S60 of FIG. 8 and S79 of FIG. 10 functions as a “first image data acquisition unit”. The other device image data corresponds to the “second image data” of the present invention. The CPU 10 that acquires the other apparatus image data in S61 of FIG. 8 and S83 of FIG. 10 functions as the “second image data acquisition unit”. The CPU 10 that generates the composite image data in S62 of FIG. 8 and S84 of FIG. 10 functions as a “generating unit”. The CPU 10 that outputs the composite image data in S65 of FIG. 8 and S86 of FIG. 10 functions as an “output unit”.

  The CPU 10 that acquires the connection number in S23 and S34 in FIG. 7 functions as a “connection order information acquisition unit”. The CPU 10 that acquires the combined position table in S56 of FIG. 8 and S76 of FIG. 10 functions as a “table acquisition unit”. The CPU 10 that acquires the layout information in S53 of FIG. 8 and S73 of FIG. 10 functions as a “setting information acquisition unit”. The CPU 10 that determines whether or not the device itself is directly connected to the PC 3 in S21 of FIG. 7 functions as a “determination unit”. The CPU 10 that performs the compression process including the predictive encoding process in S63 and S64 of FIG. 8 functions as a “compression processing unit”. The CPU 10 that acquires the resolution and the frame rate in S51 of FIG. 8 functions as an “output information acquisition unit”. The CPU 10 that transfers the resolution and the frame rate in S52 of FIG. 8 and S72 of FIG. 10 functions as “notification means”.

  The process of acquiring the total number in the terminal conference process of FIG. 7 corresponds to the “number acquisition step”. The process of acquiring the apparatus image data in S60 of FIG. 8 and S79 of FIG. 10 corresponds to the “first image data acquisition step”. The process of acquiring the other apparatus image data in S61 of FIG. 8 and S83 of FIG. 10 corresponds to a “second image data acquisition step”. The process of generating the composite image data in S62 of FIG. 8 and S84 of FIG. 10 corresponds to a “generation step”. The process of outputting the composite image data in S65 of FIG. 8 and S86 of FIG. 10 corresponds to an “output step”.

  It goes without saying that the present invention is not limited to the above-described embodiment, and various modifications are possible. For example, a device other than the video conference terminal 1 can be used as the “image processing apparatus” according to the present invention. For example, a video conference can be executed by connecting a plurality of PCs equipped with cameras in series to the PC 3 connected to the network 8. In this case, each of a plurality of PCs equipped with a camera corresponds to the “image processing apparatus” of the present invention. Further, the image processing apparatus itself does not need to have a camera. The image processing apparatus can also input its own image data from an external camera connected to the image processing apparatus. Similarly, the “communication device” according to the present invention need not be the PC 3. A dedicated communication device or the like for executing a video conference can also be used as the “communication device” according to the present invention.

  The connection between the PC 3 and the highest terminal and the connection between the plurality of video conference terminals 1 are not limited to the USB connection, and a connection method according to another protocol may be used. It goes without saying that wireless communication may be used.

  In the said embodiment, the some video conference terminal 1 is connected to PC3 in series. However, the present invention can be applied to various cases except when all of the plurality of video conference terminals 1 are connected to the PC 3 in parallel. For example, even when a plurality of video conference terminals 1 are connected in parallel to the PC 3, the present invention is applied if the video conference terminal 1 is further connected to each video conference terminal 1 connected in parallel. can do. In this case, the PC 3 needs to input and combine image data from a plurality of video conference terminals 1 connected in parallel to the PC 3. However, if the present invention is applied, the highest terminal directly connected to the PC 3 can synthesize a plurality of images input to the own device and output the synthesized image to the PC 3. Therefore, the processing amount of the process in which the PC 3 synthesizes an image is reduced as compared with the case where the present invention is not applied. Further, when a plurality of video conference terminals 1 are connected in parallel to the highest terminal, the highest terminal synthesizes all the image data and outputs them to the PC 3, thereby reducing the processing load on the PC 3. Can do.

  In the above embodiment, the PC 3 does not take an image by itself. However, the PC 3 may acquire its own device image data and synthesize the own device image data with the other device image data input from the highest terminal. In this case, it is desirable that the highest terminal outputs the composite image data to the PC 3 without performing predictive coding and entropy coding.

  In the above embodiment, when the user himself / herself designates the image layout, the layout information is set by the PC 3 receiving the user's operation input. However, the video conference terminal 1 may accept user operation input and set layout information.

1 Video conference terminal 3 PC
8 Network 10 CPU
11 ROM
13 Flash memory 21 USB device I / F
25 USB host I / F
28 Camera 51 Image layout input template 100 Communication system

Claims (7)

Used in a communication system including a communication device that transmits data via a network and a plurality of image processing devices connected in series to the communication device, and the image data of an image to be displayed at another base is the communication An image processing apparatus capable of outputting to an apparatus,
Number acquisition means for acquiring the number of other image processing devices connected to the communication device;
First image data acquisition means for acquiring first image data from an imaging means for capturing an image;
Second image data acquisition means for acquiring second image data from another image processing apparatus connected to the apparatus;
The first image data acquired by the first image data acquisition unit and the second image data acquired by the second image data acquisition unit are combined and synthesized according to the number of units acquired by the number acquisition unit. Generating means for generating image data;
An image processing apparatus comprising: output means for outputting the composite image data generated by the generation means to another connected image processing apparatus or the communication apparatus. A connection order information acquisition means for acquiring a connection order of the own apparatus with respect to the communication apparatus among the own apparatus connected to the communication apparatus and other image processing apparatuses;
A table acquisition means for acquiring from the storage means a table that associates the coordinate information indicating the synthesis position of the first image data in the coordinates on the composite image data and the connection order of the own device;
In the table acquired by the table acquisition unit, the generation unit combines the first image data at a combination position indicated by coordinate information corresponding to the connection order of the own apparatus acquired by the connection order information acquisition unit. The image processing apparatus according to claim 1. This is information set by the user inputting an operation on the template displayed on the display means, and indicates the composite position of the first image data of each image processing apparatus designated by the user at the coordinates on the template It further comprises setting information acquisition means for acquiring coordinate information,
The image processing apparatus according to claim 1, wherein the generation unit combines the first image data at a combination position indicated by the coordinate information acquired by the setting information acquisition unit. The second image data acquisition means acquires the second image data subjected to transform coding and quantization from the other image processing device connected thereto,
The image processing apparatus includes:
Determining means for determining whether or not the own apparatus is connected to the communication apparatus without passing through another image processing apparatus;
When it is determined by the determination means that the communication device is connected without going through another image processing device, the composite image data generated by the generation device is subjected to compression processing including predictive coding processing. Further comprising compression processing means for performing,
The image processing apparatus according to claim 1, wherein the output unit outputs the composite image data processed by the compression processing unit to the communication apparatus. Output information acquisition means for acquiring from the communication device the resolution and frame rate of the composite image data to be output to the communication device;
A notification means for notifying the other image processing apparatus connected to the communication apparatus of the resolution and frame rate acquired by the output information acquisition means;
The generation unit generates composite image data in accordance with the resolution and frame rate acquired by the output information acquisition unit or the resolution and frame rate notified by a notification unit of another image processing apparatus. The image processing apparatus according to any one of 1 to 4. Used in a communication system including a communication device that transmits data via a network and a plurality of image processing devices connected in series to the communication device, and the image data of an image to be displayed at another base is the communication An image processing method executed by an image processing apparatus capable of outputting to an apparatus,
Number acquisition step of acquiring the number of other image processing devices connected to the communication device; and
A first image data acquisition step of acquiring first image data from an imaging means for capturing an image;
A second image data acquisition step of acquiring second image data from another image processing device connected to the device;
The first image data acquired in the first image data acquisition step and the second image data acquired in the second image data acquisition step are combined and combined according to the number acquired in the number acquisition step. A generation step for generating image data;
An image processing method comprising: an output step of outputting the composite image data generated in the generation step to another connected image processing device or the communication device. Used in a communication system including a communication device that transmits data via a network and a plurality of image processing devices connected in series to the communication device, and the image data of an image to be displayed at another base is the communication An image processing program used in an image processing apparatus capable of outputting to an apparatus,
Number acquisition step of acquiring the number of other image processing devices connected to the communication device; and
A first image data acquisition step of acquiring first image data from an imaging means for capturing an image;
A second image data acquisition step of acquiring second image data from another image processing device connected to the device;
The first image data acquired in the first image data acquisition step and the second image data acquired in the second image data acquisition step are combined and combined according to the number acquired in the number acquisition step. A generation step for generating image data;
An image processing program including an instruction for causing the controller of the image processing apparatus to execute the output step of outputting the composite image data generated in the generating step to another connected image processing apparatus or the communication apparatus .

Images