JP7346915B2 - Image processing system and image processing method - Google Patents

Description

The present invention relates to an image processing device, an image processing system, an image processing method, and a program.

In recent years, real estate viewing services have been provided that allow users to preview real estate such as condominiums on their personal PCs without having to go to the site. In such services, users can create a virtual experience as if they are moving back and forth in the room by taking images of each room, stitching together multiple images, displaying them, and switching viewpoints. can be done.

Furthermore, in such a service, in order to display images that do not feel strange to the user, it is important to match the brightness and color tone among the plurality of displayed images. Therefore, there is a known technique for correcting the tint of images captured under different brightness and lighting conditions (for example, Patent Document 1).

However, in the conventional method, there is room for consideration in correcting the image to be corrected depending on the purpose of the service provided by the business operator or the like and the needs of the user.

The image processing system according to claim 1 includes a photographing device that is communicably connected via a communication network to an image processing device that performs color correction of image data to be corrected based on a plurality of image data, and that photographs a subject. An image processing system comprising: a photographing system including a photographing system, wherein the image processing apparatus includes a photographing system including a photographing system, the image processing apparatus comprising: a photographing system including a photographing system; representative color determining means for determining one representative color for each of the image data; and a second representative that indicates characteristics regarding colors representing the plurality of image data based on the plurality of determined first representative colors. A representative color calculating means for calculating a color, the first representative color corresponding to the image data to be corrected, and the second representative color to be used for the color correction of the image data to be corrected. a correction parameter calculation means for calculating a correction parameter; a correction means for performing the color correction of the image data to be corrected using the calculated correction parameter; and an image processing means for reducing the image size of the image data. , a first transmitting means for transmitting a plurality of preview image data obtained by performing the color correction on the reduced image data to the photographing system, the photographing system comprising: a receiving unit that receives a selection of specific preview image data from among the plurality of preview image data transmitted from the plurality of preview image data; and a second transmitting unit that transmits the accepted specific preview image data to the image processing device. and, the correction means performs the color correction using the correction parameter corresponding to the specific preview image data transmitted from the photographing system.

According to the present invention, it is possible to perform image correction according to the purpose of a service provided by a business operator or the like and the needs of a user.

FIG. 1 is a diagram illustrating an example of a system configuration of an image processing system according to a first embodiment. FIG. 2 is a diagram illustrating an example of a schematic process of the image processing system according to the first embodiment. FIG. 1 is a diagram illustrating an example of the hardware configuration of an image processing apparatus according to a first embodiment. It is a diagram showing an example of the hardware configuration of the imaging device according to the first embodiment. It is a diagram showing an example of the hardware configuration of a communication terminal according to the first embodiment. FIG. 2 is a diagram illustrating an example of the hardware configuration of a display terminal according to the first embodiment. FIG. 1 is a diagram illustrating an example of a functional configuration of an image processing system according to a first embodiment. FIG. 3 is a diagram illustrating an example of a detailed functional configuration of a calculation unit according to the first embodiment. FIG. 3 is a conceptual diagram showing an example of a representative color management table according to the first embodiment. FIG. 3 is a conceptual diagram showing an example of a comprehensive representative color management table according to the first embodiment. FIG. 3 is a conceptual diagram showing an example of a correction parameter management table according to the first embodiment. FIG. 2 is a sequence diagram illustrating an example of correction processing for captured images in the image processing system according to the first embodiment. 7 is a flowchart illustrating an example of a correction parameter calculation process in the image processing apparatus according to the first embodiment. 7 is a flowchart illustrating an example of representative color calculation processing in the image processing apparatus according to the first embodiment. FIG. 3 is a diagram showing an example of a photographed image according to the first embodiment. FIG. 3 is a diagram showing an example of an image obtained by performing cluster processing on a photographed image according to the first embodiment. 7 is a flowchart illustrating an example of comprehensive representative color calculation processing in the image processing apparatus according to the first embodiment. FIG. 3 is a diagram showing an example of a corrected image according to the first embodiment. FIG. 3 is a sequence diagram illustrating an example of preview image selection processing in the communication terminal according to the first embodiment. FIG. 3 is a diagram showing an example of a preview screen according to the first embodiment. FIG. 7 is a diagram illustrating an example of a schematic process of an image processing system according to a second embodiment. FIG. 7 is a diagram illustrating an example of a detailed functional configuration of a calculation unit according to a second embodiment. FIG. 7 is a conceptual diagram showing an example of a representative brightness value management table according to the second embodiment. 7 is a flowchart illustrating an example of correction parameter calculation processing in the image processing apparatus according to the second embodiment. FIG. 7 is a diagram showing an example of brightness correction parameters according to the second embodiment. It is a figure which shows an example of the preview screen based on 2nd Embodiment. It is a figure showing an example of detailed functional composition of a calculation part concerning modification 1 of a 2nd embodiment. FIG. 7 is a conceptual diagram showing an example of a comprehensive representative brightness value management table according to Modification 1 of the second embodiment. FIG. 7 is a conceptual diagram showing an example of a correction parameter management table according to Modification 1 of the second embodiment. 12 is a flowchart illustrating an example of a correction parameter calculation process in the image processing apparatus according to Modification 1 of the second embodiment. 12 is a flowchart illustrating an example of overall representative brightness value calculation processing in the image processing apparatus according to Modification 1 of the second embodiment. FIG. 7 is a diagram illustrating an example of brightness correction parameters according to Modification 1 of the second embodiment. FIG. 7 is a diagram illustrating an example of brightness correction gain according to Modification 1 of the second embodiment. It is a figure showing an example of detailed functional composition of a calculation part concerning a 3rd embodiment. It is a figure showing an example of detailed functional composition of a calculation part concerning modification 1 of a 3rd embodiment. FIG. 7 is a diagram illustrating an example of a system configuration of an image processing system according to another embodiment. FIG. 7 is a diagram illustrating an example of a functional configuration of an image processing system according to another embodiment. FIG. 7 is a diagram illustrating an example of a system configuration of an image processing system according to another embodiment. FIG. 7 is a diagram illustrating an example of a functional configuration of an image processing system according to another embodiment.

Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings. In addition, in the description of the drawings, the same elements are given the same reference numerals, and overlapping description will be omitted.

●First embodiment●
●System Configuration FIG. 1 is a diagram showing an example of the system configuration of an image processing system according to the first embodiment. The image processing system 1a shown in FIG. 1 allows a viewer Y to check the status of the base P by displaying an image of an arbitrary space of the base P captured by the imaging device 30 on a display terminal 90 located at a remote location. This is a system that allows you to

As shown in FIG. 1, the image processing system 1a according to the first embodiment includes an image processing device 10a, a photographing device 30, a communication terminal 50, a wireless router 9, and a display terminal 90.

Among these, the image processing device 10a is, for example, a server computer, and can perform data communication with the communication terminal 50 and the display terminal 90 via the communication network 5. The image processing device 10a performs color correction processing on the photographed image data 200 to be corrected, based on the plurality of photographed image data 200 transmitted from the communication terminal 50. The image processing device 10a also provides the display terminal 90 with corrected image data generated by the color correction process. Note that the image processing device 10a may be configured by a single server computer, or may be configured by a plurality of server computers.

Further, the photographing device 30 is a digital camera for obtaining a spherical (360°) panoramic image. Note that this photographing device 30 may be a general digital camera, and if the communication terminal 50 is equipped with a camera, the communication terminal 50 can be a digital camera. The communication terminal 50 is a terminal that transmits and receives data to and from the photographing device 30. Further, the communication terminal 50 can perform data communication with the photographing device 30 and can also perform data communication with the image processing device 10a via the wireless router 9 and the communication network 5. Note that the communication network 5 is, for example, the Internet. Further, the photographing device 30 and the communication terminal 50 constitute a photographing system 7. Note that the photographing system 7 configured by the photographing device 30 and the communication terminal 50 may be configured by one device or terminal having the functions of the photographing device 30 and the communication terminal 50.

Further, the display terminal 90 is, for example, a notebook PC (Personal Computer), and can perform data communication with the image processing device 10a via the communication network 5. The display terminal 90 is a terminal used by a user (viewer Y) who uses the service provided by the administrator X. Note that the display terminal 90 may be not only a notebook PC but also a smartphone, a tablet PC, a desktop PC, or the like.

The photographing device 30, the communication terminal 50, and the wireless router 9 are installed at a base P that is managed by the administrator X. The base P is, for example, a housing complex such as a condominium or an apartment, a detached house, a store, or the like. The image processing system 1a provides a user (viewer Y) with a predetermined service using a photographed image 800 related to photographed image data 200 of a space (subject) at the base P photographed by the photographing device 30. The display terminal 90 displays the image representing the situation of the base P sent via the image processing device 10a, so that the viewer Y can view the image representing the space (situation) of the base P. can.

●Overview Here, an overview of the processing of the image forming system according to the first embodiment will be described. Note that FIG. 2 briefly explains the outline of the image processing system according to the first embodiment, and details such as functions realized by the image processing system 1a will be explained using drawings etc. described later. .

FIG. 2 is a diagram schematically illustrating an example of processing of the image forming system according to the first embodiment. The image processing system 1a is a system in which the image processing device 10a performs color correction of the captured image data 200 to be corrected, based on a plurality of captured image data 200 captured and acquired by the imaging device 30. The image processing device 10a can perform correction to match the color tone on a photographed image 800 related to a plurality of photographed image data 200 in which different subjects are photographed.

First, the image processing device 10a acquires a plurality of photographed image data 200 (200a, 200b, 200c, 200d) in which different subjects are photographed by the photographing device 30. The image processing device 10a determines the representative color of the captured image 800 related to the acquired captured image data 200 for each captured image data 200. Here, the representative color is a pixel value that indicates a characteristic related to a color that represents the photographed image data 200 based on the pixel value of each pixel constituting the photographed image 800 related to the photographed image data 200.

Next, the image processing device 10a uses the plurality of representative colors determined by the representative color determination process to calculate a comprehensive representative color that indicates characteristics related to colors representing the plurality of photographed image data 200. Then, the image processing device 10a performs correction for performing color correction of the photographed image data 200 using the comprehensive representative color calculated by the comprehensive representative color calculation process and the representative color determined for each photographed image data 200. Calculate parameters.

Next, the image processing device 10a performs color correction processing for each photographed image data 200 using the correction parameters calculated by the correction parameter calculation processing. The image processing device 10a then transmits the corrected image data 250 (250a, 250b, 250c, 250d) on which the color correction process has been performed to the display terminal 90. Thereby, the display terminal 90 can allow the user to confirm the situation of the base P where the photographing device 30 is located by displaying the plurality of received corrected image data 250 in which different subjects are photographed.

In a tour display in a real estate viewing service to which the image processing system 1a is applied, it is necessary to ensure that the colors are consistent and the brightness is appropriate among multiple images of different subjects taken for each room of the real estate property. It is preferable. For example, if each image is viewed individually, the user will not feel uncomfortable, but if a plurality of images are viewed in series, the user viewing the images (for example, viewer Y in FIG. 1) may feel uncomfortable.

However, conventional color correction methods are methods for improving color reproducibility for the same subject, and do not take into account image processing that corrects brightness and color for images with different subjects. do not have. For example, even if a pixel to be corrected is extracted from image data taken under a certain light source, it is not possible to match the extraction area corresponding to the pixel to be corrected to another image data of a different subject. . Furthermore, since the conventional method is a method for reproducing the same colors as the ideal original (subject) under a predetermined viewing light source (illumination), it is possible to reproduce multiple images with different brightness and color tone. , I didn't know how to determine the brightness and color tone I was aiming for, and how to correct them.

Therefore, the image processing system 1a determines a representative color for each photographed image 800 from a plurality of photographed images 800 having different brightness and color, and further determines one overall representative color representing each of the determined representative colors. do. Then, the image processing system 1a calculates a correction parameter to be used for color correction using the representative color in each photographed image 800 and a general representative color common to each photographed image 800, and uses the calculated correction parameter to calculate a correction parameter to be used for color correction. , performs color correction on each captured image 800. As a result, the image processing system 1a can generate a corrected image 850 with uniform brightness and color tone for a plurality of captured images 800 that differ in brightness and color tone due to photographing different subjects. Image correction can be performed in accordance with the needs of service providers and users who wish to adjust the brightness and color of the plurality of captured images 800 to improve convenience for users who view the images.

●Hardware Configuration Next, the hardware configuration of the image processing device 10a, the photographing device 30, the communication terminal 50, and the display terminal 90 in the first embodiment will be described using FIGS. 3 to 5. Note that the hardware configurations shown in FIGS. 3 to 5 may have the same configuration in each embodiment, and components may be added or deleted as necessary.

Hardware Configuration of Image Processing Apparatus First, the hardware configuration of the image processing apparatus 10a will be described using FIG. 3. FIG. 3 is a diagram illustrating an example of the hardware configuration of the image processing apparatus according to the first embodiment.

The image processing device 10a is constructed by a computer, and as shown in FIG. , an HDD (Hard Disk Drive) controller 105, a recording medium 106, a media I/F 107, a network I/F 109, a DVD-RW (Digital Versatile Disk Rewritable) drive 114, and a bus line 110.

Among these, the CPU 101 controls the entire operation of the image processing apparatus 10a. The ROM 102 stores programs used to drive the CPU 101, such as IPL (Initial Program Loader). RAM 103 is used as a work area for CPU 101. The HD 104 stores various data such as programs. The HDD controller 105 controls reading and writing of various data to the HD 104 under the control of the CPU 101. The media I/F 107 controls reading or writing (storage) of data to a recording medium 106 such as a flash memory. The network I/F 109 is an interface for data communication using the communication network 5. The DVD-RW drive 114 controls reading and writing of various data to and from a DVD-RW 113, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may be DVD-R or the like. Further, the DVD-RW drive 114 may be a Blu-ray drive that controls reading and writing of various data on a Blu-ray Disc (registered trademark).

The image processing device 10a also includes a bus line 110. The bus line 110 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 101 shown in FIG. 3.

Hardware configuration of photographing device First, the hardware configuration of the photographing device 30 will be explained using FIG. 4. FIG. 4 is a diagram showing an example of the hardware configuration of the imaging device according to the first embodiment. In the following, the photographing device 30 is assumed to be a spherical (omnidirectional) photographing device using two image sensors, but the number of image sensors may be any number greater than two. Furthermore, the photographing device 30 does not necessarily have to be a device exclusively for omnidirectional photography, and by attaching an aftermarket omnidirectional imaging unit to a normal digital camera, smartphone, etc., it can have substantially the same functions as the photographing device 30. It may be made to have.

As shown in FIG. 4, the imaging device 30 includes an imaging unit 301, an image processing unit 304, an imaging control unit 305, a microphone 308, a sound processing unit 309, a CPU 311, a ROM 312, an SRAM 313, a DRAM 314, an operation unit 315, an input/output I/O It is composed of F316, short-range communication circuit 317, antenna 317a, and acceleration/direction sensor 318.

Among these, the imaging unit 301 includes wide-angle lenses (so-called fisheye lenses) 302a and 302b each having an angle of view of 180° or more for forming a hemispherical image, and two lenses provided corresponding to each wide-angle lens. It includes image sensors 303a and 303b. The image sensors 303a and 303b are image sensors such as CMOS (Complementary Metal Oxide Semiconductor) sensors and CCD (Charge Coupled Device) sensors that convert optical images formed by wide-angle lenses 302a and 302b into electrical signal image data and output the image data. It has a timing generation circuit that generates horizontal or vertical synchronization signals and pixel clocks for the sensor, and a group of registers in which various commands, parameters, etc. necessary for the operation of this image sensor are set.

The image sensors 303a and 303b are each connected to an image processing unit 304 via a parallel I/F bus. Further, the imaging elements 303a and 303b are each connected to an imaging control unit 305 via a serial I/F bus (such as an I2C bus). Image processing unit 304, imaging control unit 305, and sound processing unit 309 are connected to CPU 311 via bus line 310. Further, the bus line 310 is connected to a ROM 312, an SRAM 313, a DRAM 314, an operation unit 315, an input/output I/F 316, a short-range communication circuit 317, an acceleration/direction sensor 318, and the like.

The image processing unit 304 takes in image data output from the image sensors 303a and 303b through the parallel I/F bus. Then, the image processing unit 304 performs predetermined processing on each image data, and then synthesizes these image data to create data of an equirectangular projection image.

Generally, the imaging control unit 305 uses the I2C bus to set commands and the like in register groups of the imaging devices 303a and 303b, using the imaging control unit 305 as a master device and the imaging devices 303a and 303b as slave devices. Necessary commands and the like are received from the CPU 311. The imaging control unit 305 also uses the I2C bus to take in status data and the like of the register groups of the imaging elements 303a and 303b, and sends it to the CPU 311.

Further, the imaging control unit 305 instructs the imaging elements 303a and 303b to output image data at the timing when the shutter button of the operation unit 315 is pressed. The photographing device 30 may have a preview display function on a display (for example, the display 517 of the communication terminal 50), and a function corresponding to the display of still images and moving images. In the case of a moving image, image data is output continuously from the image sensors 303a and 303b at a predetermined frame rate (frames/minute).

The imaging control unit 305 also functions as a synchronization control unit that synchronizes the output timing of image data of the imaging elements 303a and 303b in cooperation with the CPU 311, as will be described later. Note that in this embodiment, the photographing device 30 is not provided with a display, but may be provided with a display section.

The microphones 308 each collect sound from the surrounding environment of the photographing device 30 and convert the collected sound into sound (signal) data. The sound processing unit 309 takes in sound data output from the microphone 308 through the I/F bus, and performs predetermined processing on the sound data. The microphones 308 constitute sound collection means for collecting sounds from the surrounding environment. Further, although FIG. 4 shows only a case where the microphone 308 is built into the photographing device 30, the microphone 308 may be externally attached to the photographing device 30. Further, the number of microphones 308 is not limited to one, and may include a plurality of microphones having a predetermined arrangement. Preferably, microphone 308 is an ambisonics microphone.

The CPU 311 controls the overall operation of the photographing device 30 and executes necessary processing. ROM312 stores various programs for CPU311. The SRAM 313 and DRAM 314 are work memories that store programs executed by the CPU 311, data being processed, and the like. In particular, the DRAM 314 stores image data that is currently being processed by the image processing unit 304 and data of processed equirectangular projection images.

The operation unit 315 is a general term for operation buttons such as the shutter button 315a. The user inputs various shooting modes, shooting conditions, etc. by operating the operation unit 315.

The input/output I/F 316 is a general term for an interface circuit (USB I/F, etc.) with external media such as an SD card or a personal computer. The input/output I/F 316 may be wireless or wired. The data of the equirectangular projection image stored in the DRAM 314 may be recorded on an external media via the input/output I/F 316, or may be sent to an external terminal such as the communication terminal 50 via the input/output I/F 316 as necessary. (device).

The short-range communication circuit 317 performs communication via an antenna 317a provided in the photographing device 30 by short-range wireless communication technology such as Wi-Fi (registered trademark), NFC (Near Field Communication), or Bluetooth (registered trademark). It communicates with an external terminal (device) such as the terminal 50. The short-range communication circuit 317 can transmit data of the equirectangular projection image to an external terminal (device) such as the communication terminal 50.

The acceleration/azimuth sensor 318 calculates the azimuth of the imaging device 30 from the earth's magnetism and outputs azimuth information. The orientation information is an example of related information (metadata) in accordance with Exif (Exchangeable image file format), and is used for image processing such as image correction of captured images. Note that the related information includes data such as the date and time when the image was taken and the data capacity of the image data.

Hardware Configuration of Communication Terminal Next, the hardware configuration of the communication terminal 50 will be described using FIG. 5. FIG. 5 is a diagram illustrating an example of the hardware configuration of the communication terminal according to the first embodiment.

The communication terminal 50 includes a CPU 501, a ROM 502, a RAM 503, an EEPROM (Electrically Erasable Programmable Read-Only Memory) 504, a CMOS sensor 505, an image sensor I/F 513a, an acceleration/direction sensor 506, a media I/F 508, and a GPS receiver 509. ing.

Among these, the CPU 501 controls the operation of the communication terminal 50 as a whole. The ROM 502 stores programs used to drive the CPU 501 such as IPL. RAM 503 is used as a work area for CPU 501. The EEPROM 504 reads or writes various data such as communication terminal programs under the control of the CPU 501.

The CMOS sensor 505 captures an image of a subject (mainly a self-portrait) under the control of the CPU 501 and obtains image data. The image sensor I/F 513a is a circuit that controls driving of the CMOS sensor 512. The acceleration/direction sensor 506 is a variety of sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect geomagnetism. A media I/F 508 controls reading or writing (storage) of data to a recording medium 507 such as a flash memory. GPS receiving section 509 receives GPS signals from GPS satellites.

The communication terminal 50 also includes a long-distance communication circuit 511, an antenna 511a, a CMOS sensor 512, an image sensor I/F 513b, a microphone 514, a speaker 515, a sound input/output I/F 516, a display 517, an external device connection I/F 518, a nearby It includes a distance communication circuit 519, an antenna 519a of the short distance communication circuit 519, and a touch panel 521.

The long-distance communication circuit 511 is a circuit that communicates with other devices via the communication network 5. The CMOS sensor 512 is a type of built-in imaging means that images a subject and obtains image data under the control of the CPU 501. The image sensor I/F 513b is a circuit that controls driving of the CMOS sensor 512. The microphone 514 is a type of built-in sound collecting means for inputting audio. The sound input/output I/F 516 is a circuit that processes input/output of sound signals between the microphone 514 and the speaker 515 under the control of the CPU 501.

The display 517 is a type of display means such as a liquid crystal or an organic EL display that displays an image of a subject, various icons, and the like. External device connection I/F 518 is an interface for connecting various external devices. The short-range communication circuit 519 is a communication circuit such as Wi-Fi, NFC, or Bluetooth. The touch panel 521 is a type of input means by which the user operates the communication terminal 50 by pressing the display 517. The communication terminal 50 also includes a bus line 510. The bus line 510 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501.

Hardware Configuration of Display Terminal Next, the hardware configuration of the display terminal 90 will be described using FIG. 6. FIG. 6 is a diagram illustrating an example of the hardware configuration of the display terminal according to the first embodiment.

The display terminal 90 is constructed by a computer, and as shown in FIG. , a mouse 912, a DVD-RW drive 914, and a bus line 910.

Of these, the CPU 901 controls the overall operation of the display terminal 90. The ROM 902 stores programs used to drive the CPU 901 such as IPL. RAM903 is used as a work area for CPU901. The HD 904 stores various data such as programs. The HDD controller 905 controls reading and writing of various data to the HD 904 under the control of the CPU 901. A media I/F 907 controls reading or writing (storage) of data to a recording medium 906 such as a flash memory. A display 908 displays various information such as a cursor, menu, window, characters, or images. Display 908 is an example of a display unit. The network I/F 909 is an interface for data communication using the communication network 5. The keyboard 911 is a type of input means that includes a plurality of keys for inputting characters, numbers, various instructions, and the like. The mouse 912 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 914 controls reading and writing of various data on a DVD-RW 913, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may be DVD-R or the like. Further, the DVD-RW drive 914 may be a Blu-ray drive that controls reading and writing of various data on a Blu-ray disc.

The display terminal 90 also includes a bus line 910. The bus line 910 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 901 shown in FIG. 6.

Note that recording media such as HDs and CD-ROMs in which the above programs are stored can be provided domestically or internationally as program products.

●Functional configuration Next, the functional configuration of the device and terminal according to the first embodiment will be described. FIG. 7 is a diagram illustrating an example of the functional configuration of the image processing system according to the first embodiment.

First, the functional configuration of the image processing device 10a will be explained. The functions realized by the image processing device 10a include a network communication section 11, an image processing section 12, a calculation section 13a, a correction section 14, a judgment section 15, a display screen generation section 16, a storage/reading section 19, and a storage section 1000. .

The network communication unit 11 has a function of transmitting and receiving various data (or information) to and from the communication terminal 50 or the display terminal 90 via the communication network 5. The network communication unit 11 receives, for example, photographed image data 200 transmitted from the photographing system 7 and acquired by photographing with the photographing device 30. Further, the network communication unit 11 transmits the corrected image data 250 generated by the correction unit 14 to the display terminal 90 via the communication network 5. The network communication unit 11 is mainly realized by the network I/F 109 shown in FIG. 3 and the processing of the CPU 101. The network communication unit 11 is an example of a first transmitting means.

The image processing unit 12 has a function of performing various processes on the captured image data 200 received by the network communication unit 11. The image processing unit 12, for example, acquires the captured image data 200 received by the network communication unit 11, and develops it as pixel data. The image processing unit 12 also converts the acquired captured image data 200 into an arbitrary data size (image size). Specifically, the image processing unit 12 reduces the data size of the acquired captured image data 200 as a preview image to be viewed by the administrator X (see FIG. 1). The image processing unit 12 is mainly realized by the processing of the CPU 101 shown in FIG. The image processing section 12 is an example of an image processing means.

The calculation unit 13a has a function of calculating correction parameters for correcting the brightness or tint of the captured image data 200 processed by the image processing unit 12. The calculation unit 13a calculates, for example, a correction parameter for color correction of the photographed image data 200 to be corrected by the correction unit 14, which will be described later, based on a plurality of pieces of photographed image data 200. A detailed explanation of the calculation unit 13a will be given later. The calculation unit 13a is mainly realized by the processing of the CPU 101 shown in FIG.

The correction unit 14 has a function of performing color correction on the captured image data 200 using the correction parameters calculated by the calculation unit 13a. The correction unit 14 generates corrected image data 250 by performing color correction on the captured image data 200, for example. The correction unit 14 is mainly realized by the processing of the CPU 101 shown in FIG. The correction unit 14 is an example of a correction means.

The judgment unit 15 is mainly realized by the processing of the CPU 101 shown in FIG. 3, and has a function of making various judgments.

The display screen generation unit 16 has a function of generating various display screens to be displayed on the communication terminal 50 or the display terminal 90. The display screen generation unit 16 generates a preview screen for display on the communication terminal 50, for example. The display screen generation unit 16 generates, for example, a display screen for displaying the corrected image data on the display terminal 90. The display screen generation unit 16 is mainly realized by the processing of the CPU 101 shown in FIG.

The storage/reading unit 19 has a function of storing various data in the storage unit 1000 or reading various data from the storage unit 1000. The storage/reading section 19 is mainly realized by the processing of the CPU 101 shown in FIG. The storage unit 1000 is mainly realized by the ROM 102, HD 104, and recording medium 106 shown in FIG. Furthermore, a pixel value management DB 1001 and a correction parameter management DB 1003 are constructed in the storage unit 1000. Of these, the pixel value management DB 1001 is composed of a representative color management table and a comprehensive representative color management table, which will be described later. The correction parameter management DB 1003 is composed of a correction parameter management table, which will be described later. Furthermore, the storage unit 1000 stores photographed image data 200 and corrected image data 250.

Here, the functional configuration of the calculation unit 13a will be described in detail using FIG. 8. FIG. 8 is a diagram illustrating an example of a detailed functional configuration of the calculation unit according to the first embodiment. The calculation unit 13a shown in FIG. 8 includes a representative pixel value calculation unit 131a, a representative color determination unit 132a, a comprehensive representative color calculation unit 133a, and an RGB gain calculation unit 134a.

The representative pixel value calculation unit 131a has a function of calculating a representative pixel value indicating a characteristic related to a color representing a cluster obtained by clustering processing of the pixel values of each pixel constituting the photographed image 800 related to the photographed image data 200. For example, the representative pixel value calculation unit 131 performs clustering processing on the pixel values of each pixel constituting the photographed image 800 related to the photographed image data 200 read by the image processing unit 12, and calculates k (k is a natural number) pixel values. >0) clusters are generated. Then, the representative pixel value calculation unit 131 calculates the center pixel of the generated cluster as the representative pixel value of the cluster. The center pixel of a cluster is a pixel value that is the center of gravity of a group of pixels forming the cluster. The representative pixel value calculation unit 131a is an example of representative pixel value calculation means.

The representative color determining unit 132a determines, for each piece of photographed image data 200, a representative color that indicates a characteristic related to a color that represents the photographed image data 200, based on the pixel value of each pixel constituting the photographed image 800 related to the photographed image data 200. It is a function to decide. For example, the representative color determining unit 132a determines, as the representative color, the representative pixel value corresponding to a specific cluster having the largest number of pixels included in the cluster, among the representative pixel values for each cluster calculated by the representative pixel value calculating unit 131a. decide. The representative color determined by the representative color determination unit 132a is an example of a first representative color. Further, the representative color determining unit 132a is an example of representative color determining means.

The overall representative color calculation unit 133a has a function of calculating an overall representative color that indicates characteristics related to colors that represent the plurality of captured image data 200, based on the representative colors determined by the representative color determination unit 132a. The comprehensive representative color calculated by the comprehensive representative color calculation unit 133a is an example of the second representative color. Further, the comprehensive representative color calculation unit 133a is an example of representative color calculation means.

The RGB gain calculation unit 134a calculates an RGB gain value to be a correction parameter based on the representative color determined by the representative color determination unit 132a and the overall representative color calculated by the overall representative color calculation unit 133a. The RGB gain calculation unit 134a is an example of a correction parameter calculation unit.

〇Representative Color Management Table FIG. 9 is a conceptual diagram showing an example of the representative color management table according to the first embodiment. In the storage unit 1000, a pixel value management DB 1001 is constructed, which is constituted by a representative color management table as shown in FIG. The representative color management table shown in FIG. 9 shows the results of clustering processing of the pixel values of each pixel constituting the photographed image 800 related to the photographed image data 200, for each image ID for identifying the photographed image data 200. In the representative color management table, the number of pixels included in the cluster and the center pixel of the cluster are stored and managed in association with each other for each cluster number for identifying divided clusters. Specific processing for obtaining the processing results shown in FIG. 9 will be described later.

Comprehensive Representative Color Management Table FIG. 10 is a conceptual diagram showing an example of the comprehensive representative color management table according to the first embodiment. In the storage unit 1000, a pixel value management DB 1001 is constructed, which is composed of a comprehensive representative color management table as shown in FIG. The comprehensive representative color management table shown in FIG. 10 shows the results of clustering processing of a plurality of representative colors determined by the representative color determining section 132a. In the general representative color management table, for each cluster number for identifying divided clusters, the cluster center pixel which is the center (center of gravity) of the cluster and the captured image data 200 corresponding to the representative color belonging to the cluster are identified. The image ID for the image is stored and managed in association with the image ID. In the example of FIG. 10, the cluster center pixel of the cluster No. "1" is (R, G, B) = (114, 108, 99), and the image IDs are "P01", "P02", "P04". ” belongs to each of the three photographed image data 200 that corresponds to the representative color. In addition, in the cluster No. "2", the cluster center pixel (R, G, B) = (127, 115, 86) corresponds to one photographed image data 200 identified by the image ID "P03". to which the representative color belongs.

- Correction Parameter Management Table FIG. 11 is a conceptual diagram showing an example of the correction parameter management table according to the first embodiment. In the storage unit 1000, a correction parameter management DB 1003 is constructed, which is composed of a correction parameter management table as shown in FIG. The correction parameter management table shown in FIG. 11 includes a representative color determined by the representative color determining unit 132a and a correction parameter calculated by the RGB gain calculating unit 134a for each image ID for identifying the captured image data 200. They are stored and managed in association with each other. Further, in the correction parameter management table shown in FIG. 11, the overall representative color calculated by the overall representative color calculation unit 133a is commonly stored and managed for a plurality of image IDs.

Next, the functional configuration of the photographing device 30 shown in FIG. 7 will be explained. The functions realized by the photographing device 30 include a receiving section 31, a photographing section 32, an image processing section 33, a transmitting/receiving section 34, a storage/reading section 39, and a storage section 3000.

The reception unit 31 has a function of accepting operation input from a user (for example, administrator X shown in FIG. 1). The reception section 31 is mainly realized by the processing of the operation section 315 and the CPU 311 shown in FIG.

The photographing unit 32 has a function of photographing a subject, a landscape image, etc., and obtaining photographed image data 200. The captured image data may be not only still images but also moving images. The imaging unit 32 is mainly realized by the processing of the imaging unit 301, image processing unit 304, imaging control unit 305, and CPU 311 shown in FIG.

The image processing section 33 has a function of performing various processes on the photographed image data obtained by the photographing section 32. For example, when the photographed image 800 related to the photographed image data is a spherical image, the image processing unit 33 generates an equirectangular cylinder based on two hemispherical image data obtained by each of the two image sensors 303a and 303b. Create projection image data. The image processing unit 33 is mainly realized by instructions from the CPU 311 shown in FIG.

The transmitting/receiving unit 34 has a function of communicating with the communication terminal 50 using short-range wireless communication technology such as Wi-Fi. The transmitting/receiving unit 34 is mainly realized by the short-range communication circuit 317 and antenna 317a shown in FIG. 4, and the processing of the CPU 311.

The storage/reading unit 39 has a function of storing various data in the storage unit 3000 or reading various data from the storage unit 3000. The storage/reading section 39 is mainly realized by the processing of the CPU 311 shown in FIG. The storage unit 3000 is mainly realized by the ROM 312, SRAM 313, and DRAM 314 shown in FIG.

Next, the functions realized by the communication terminal 50 shown in FIG. 7 include a network communication section 51, a transmission/reception section 52, a reception section 53, a display control section 54, a storage/reading section 59, and a storage section 5000.

The network communication unit 51 has a function of transmitting and receiving various data (or information) to and from the image processing device 10a via the communication network 5. The network communication unit 51 is mainly realized by the long distance communication circuit 511 and antenna 511a shown in FIG. 5, and the processing of the CPU 501. The network communication unit 51 is an example of second transmitting means.

The transmitting/receiving unit 52 has a function of communicating with the photographing device 30 using short-range wireless communication technology such as Wi-Fi. The transmitting/receiving unit 52 is mainly realized by the short-range communication circuit 519 and antenna 519a shown in FIG. 5, and the processing of the CPU 501.

The receiving unit 53 has a function of receiving user input through input means such as the touch panel 521 shown in FIG. 5. For example, the receiving unit 53 receives a selection of a specific image from among the display images shown on the preview screen 600a (see FIG. 20) displayed on the display 517. The reception unit 53 is mainly realized by a program executed by the CPU 501 shown in FIG. 5. The reception unit 53 is an example of reception means.

The display control unit 54 has a function of displaying various screen information on the display 517 shown in FIG. For example, the display control unit 54 causes the display 517 to display a preview screen 600a related to the preview image received by the network communication unit 51. The display control unit 54 is mainly realized by the display 517 shown in FIG. 5 and a program executed by the CPU 501.

The storage/reading unit 59 has a function of storing various data in the storage unit 5000 or reading various data from the storage unit 5000. The storage/reading unit 59 is mainly realized by the processing of the CPU 501 shown in FIG. The storage unit 5000 is mainly realized by the ROM 502, RAM 503, and EEPROM 504 shown in FIG.

Next, the functional configuration of the display terminal 90 shown in FIG. 7 will be explained. The functions realized by the display terminal 90 include a network communication section 91, a reception section 92, a display control section 93, a storage/reading section 99, and a storage section 9000.

The network communication unit 91 has a function of transmitting and receiving various data or information to and from other devices (for example, the image processing device 10a) via the communication network 5. The network communication unit 91 receives, for example, corrected image data processed by the image processing device 10a from the image processing device 10a. The network communication unit 91 is mainly realized by the processing of the network I/F 909 and CPU 901 shown in FIG.

The receiving unit 92 has a function of receiving user input through input means such as the keyboard 911 shown in FIG. 6. The reception unit 92 is mainly realized by the keyboard 911 shown in FIG. 6 and a program executed by the CPU 901.

The display control unit 93 has a function of displaying various screen information on the display 908 shown in FIG. For example, the display control unit 93 causes the display 908 to display an image related to the corrected image data received by the network communication unit 91. The display control unit 93 is mainly realized by a display 908 shown in FIG. 6 and a program executed by the CPU 901. The display control section 93 is an example of display control means.

The storage/reading unit 99 has a function of storing various data in the storage unit 9000 or reading various data from the storage unit 9000. The storage/reading unit 99 is mainly realized by the processing of the CPU 901 shown in FIG. The storage unit 9000 is mainly realized by the ROM 902, HD 904, and recording medium 906 shown in FIG.

●Processing or operation of the first embodiment Next, processing or operation in the image processing system according to the first embodiment will be described using FIGS. 12 to 20. First, the overall flow of processing or operation in the image processing system will be described using FIG. 12. FIG. 12 is a sequence diagram illustrating an example of correction processing for captured images in the image processing system according to the first embodiment.

First, in step S11, the photographing device 30 causes the photographing section 32 to perform photographing processing. The photographing unit 32 of the photographing device 30 acquires a plurality of photographed image data 200 photographed at the base P where the photographing device 30 is located. Here, the plurality of photographed image data 200 have different subjects for each photographed image data 200. Different subjects are objects, people, or spaces that exist within the base P and are photographed under different environmental conditions such as brightness and illumination. In step S12, the transmitting/receiving unit 34 of the photographing device 30 transmits the plurality of photographed image data 200 acquired by the photographing unit 32 to the communication terminal 50. Thereby, the transmitting/receiving unit 52 of the communication terminal 50 receives the plurality of photographed image data 200 transmitted from the photographing device 30.

Next, in step S13, the network communication unit 51 of the communication terminal 50 transmits the plurality of captured image data 200 received by the transmitting/receiving unit 52 to the image processing device 10a via the communication network 5. Thereby, the network communication unit 11 of the image processing device 10a receives the plurality of captured image data 200 transmitted from the communication terminal 50. Through the above processing, the photographed image data 200 acquired by the photographing device 30 is uploaded to the image processing device 10a. Note that although the configuration in which the photographing device 30 transmits the photographed image data 200 to the image processing device 10a via the communication terminal 50 has been described above, the photographing device 30 corresponds to the network communication section 51 of the communication terminal 50. When equipped with such a function, the photographing device 30 may be configured to directly transmit (distribute) the photographed image data 200 to the image processing device 10a without going through the communication terminal 50.

In step S14, the image processing device 10a uses the plurality of captured image data 200 received by the network communication unit 11 to calculate correction parameters used for color correction of the captured image data 200 to be corrected. The method for calculating the correction parameters will be described later.

In step S15, the correction unit 14 of the image processing device 10a performs color correction on the captured image data 200 using the correction parameters calculated in step S14. In this case, the correction unit 14 performs color correction on each of the plurality of photographed image data 200 using the correction parameters calculated for each photographed image data 200 in step S14.

In step S16, the network communication unit 11 of the image processing device 10a transmits the plurality of corrected image data 250 to the display terminal 90 via the communication network 5. Thereby, the network communication unit 91 of the display terminal 90 receives the plurality of corrected image data 250 transmitted from the image processing device 10a.

In step S17, the display control unit 93 of the display terminal 90 causes the display 908 to display a corrected image (see FIG. 18) for each of the plurality of corrected image data 250. Thereby, the image processing system 1a can display a plurality of corrected images with the same color tone generated by the image processing device 10a on the display terminal 90 used by the user (viewer Y).

- Correction Parameter Calculation Process Next, details of the correction parameter calculation process by the image processing device 10a will be described using FIGS. 13 to 20. FIG. 13 is a flowchart illustrating an example of a correction parameter calculation process in the image processing apparatus according to the first embodiment.

First, in step S101a, the image processing unit 12 of the image processing device 10a performs a process of reading the captured image data 200 received by the network communication unit 11. Specifically, the image processing unit 12 develops the captured image data 200 as pixel data and obtains RGB pixel values. For example, if the captured image data 200 is a JPEG (Joint Photographic Experts Group) image, the JPEG image is decoded by a JPEG decoder, and the decoded YCbCr format image data is converted into RGB format according to the conversion method specified as JPEG. Convert to image data. The image processing unit 12 also performs image size reduction processing of the converted image data. Here, a description will be given assuming that the image processing unit 12 resizes (reduces) the number of pixels of the converted image to 400 x 200 (horizontal x vertical). Note that the following processing may be performed depending on the image size of the captured image data 200 received by the network communication unit 11 without performing the image size reduction process by the image processing unit 12.

In step S102a, the calculation unit 13a of the image processing device 10a calculates a representative pixel value for the captured image data 200 read by the image processing unit 12. The representative pixel value is a pixel value indicating a characteristic related to a color representing a cluster obtained by clustering processing of the pixel values of each pixel constituting the photographed image 800 related to the photographed image data 200.

Here, details of the representative pixel value calculation process will be explained using FIG. 14. FIG. 14 is a flowchart illustrating an example of representative color calculation processing in the image processing apparatus according to the first embodiment. First, in step S102a-1, the representative pixel value calculation section 131a calculates the pixel value of each pixel constituting the photographed image 800 related to the photographed image data 200 read by the image processing section 12.

In step S102a-2, the representative pixel value calculation unit 131a divides the captured image data 200 into a plurality of clusters by clustering the calculated pixel values. The clustering process is performed, for example, by unsupervised learning using the calculated pixel values. Unsupervised learning is a method of classifying predetermined data without external criteria such as labeling. Unsupervised learning is, for example, a method such as K-means clustering (K-means method). The representative pixel value calculation unit 131a divides the pixel values of pixels constituting the captured image 800 related to the captured image data 200 into a pre-specified number of clusters so that areas belonging to the same cluster are expressed in similar colors. do. Here, the clustering process in step S102-2a will be explained using FIGS. 15 and 16. FIG. 15 is a diagram showing an example of a photographed image according to the first embodiment. FIG. 16 is a diagram showing an example of an image obtained by performing cluster processing on the photographed image according to the first embodiment. Processed images 810a to 810d shown in FIG. 16 show a state in which captured images 800a to 800d (see FIG. 15) related to different captured image data 200a to 200d are clustered into a plurality of clusters (regions) using pixel values as input data. . The representative pixel value calculation unit 131a groups the pixel values in each of the captured images 800a to 800d into similar ones based on a predetermined distance measure. As the distance measure here, the spatial distance of pixel values (RGB values) of each pixel is used.

Next, in step S102a-3, the representative pixel value calculation unit 131a calculates a representative pixel value that is a pixel value that represents each divided cluster. The representative pixel value calculation unit 131a calculates the cluster center pixel, which is the center of the cluster grouped by the process of step S102a-2, as a representative pixel value. Further, the representative pixel value calculation unit 131a calculates the number of pixels belonging to each grouped cluster. This is achieved by holding information on which cluster each pixel (pixel value) belongs to during the clustering process using the K-means method in step S102a-2. The correspondence between pixels (pixel values) and clusters is called labeling information. The representative pixel value calculation unit 131a calculates the number of pixels belonging to each cluster based on this labeling information.

In step S102a-4, the storage/readout unit 19 associates the number of pixels and the representative pixel value for each cluster and stores them in the pixel value management DB 1001. As shown in FIG. 9, the pixel value management DB 1001 stores and manages the number of pixels included in a cluster and the cluster center pixel, which is a representative pixel value, in association with each other in a representative color management table for each cluster number.

Thereby, the image processing device 10a selects, for each captured image 800 related to the captured image data 200, a representative pixel that is a characteristic related to a color representing a cluster formed by clustering processing of the pixel values of each pixel constituting the captured image 800. The value can be calculated for each cluster included in the captured image 800.

Returning to FIG. 13, the explanation of the correction parameter calculation process will be continued. In step S103a, the representative color determination unit 132a of the image processing device 10a selects the representative pixel value of the cluster with the largest number of pixels from among the representative pixel values calculated by the representative pixel value calculation unit 131a as the representative color of the captured image data 200. Decide as a color. The representative color is a pixel value that indicates a characteristic related to a color that represents the photographed image data 200 based on the pixel value of each pixel constituting the photographed image 800 related to the photographed image data 200. For example, in the case of the processing results shown in the representative color management table shown in FIG. 9, the representative color of the photographed image data 200 (for example, the photographed image 800a shown in FIG. The cluster center pixel (representative pixel value) of “1” is (R, G, B)=(109,107,100). Further, the representative color of the photographed image data 200 (for example, the photographed image 800b shown in FIG. 15) identified by the image ID "P02" is the cluster center pixel (representative pixel value) of the cluster No. "1" (R, G, B)=(118,108,96). Further, the representative color of the photographed image data 200 (for example, the photographed image 800c shown in FIG. 15) identified by the image ID "P03" is the cluster center pixel (representative pixel value) of cluster No. "3" (R, G, B)=(127,115,86). Further, the representative color of the photographed image data 200 (for example, the photographed image 800d shown in FIG. 15) identified by the image ID "P04" is the cluster center pixel (representative pixel value) of cluster No. "2" (R, G, B)=(115,110,102). Note that the method for calculating the representative color is not limited to this, and the representative color of the captured image data 200 is determined by weighted averaging of the cluster center pixel (representative pixel value) of each cluster depending on the number of pixels included in each cluster. The configuration may be such that For example, if there are multiple clusters with the largest number of pixels, the representative color determining unit 132a may determine the average value or median value of the cluster center pixels (representative pixel values) of the relevant clusters as the representative color. good.

In step S104a, the storage/reading unit 19 stores the representative color determined by the representative color determining unit 132a in the correction parameter management DB 1003. As shown in FIG. 11, the correction parameter management DB 1003 stores and manages pixel values of representative colors corresponding to the photographed image data 200 in a correction parameter management table for each image ID for identifying the photographed image data 200. do.

In step S105a, if the image processing device 10a has read all of the captured image data 200 received by the network communication unit 11, the image processing device 10a moves the process to step S106a. On the other hand, if there is unread photographed image data 200 among the photographed image data 200 received by the network communication unit 11, the image processing device 10a repeats the process from step S101a.

In step S106a, the calculation unit 13a of the image processing device 10a calculates the overall representative color using the representative color determined by the process in step S103a. The comprehensive representative color is a pixel value that indicates a characteristic related to a color that represents the plurality of photographed image data 200 based on the representative color of the plurality of photographed image data 200.

Here, details of the overall representative color calculation process will be explained using FIG. 17. FIG. 17 is a flowchart illustrating an example of comprehensive representative color calculation processing in the image processing apparatus according to the first embodiment.

In step S106a-1, the comprehensive representative color calculation unit 133a classifies the pixel values indicated by the representative colors determined by the representative color determination unit 132a into a plurality of clusters by clustering processing. The clustering process is performed using the K-means method, similar to the clustering process by the representative pixel value calculation unit 131a (see step S102a-2). The input data in this case are pixel values of a plurality of representative colors determined by the representative color determination unit 132a, and are grouped into clusters using the spatial distances of the pixel values (RGB values).

In step S106a-2, the comprehensive representative color calculation unit 133a calculates a representative pixel value that is a pixel value that represents each classified cluster. The overall representative color calculation unit 133a calculates the cluster center pixel, which is the center of the cluster grouped by the process of step S106a-1, as a representative pixel value.

In step S106a-3, the storage/reading unit 19 associates the image ID of the representative color to which each cluster belongs with the representative pixel value for each cluster and stores them in the pixel value management DB 1001. As shown in FIG. 10, the pixel value management DB 1001 stores and manages cluster center pixels and image IDs in association with each other in a general representative color management table for each cluster.

In step S106a-4, the storage/reading unit 19 stores the representative pixel value associated with the cluster to which the largest number of representative colors belongs in the correction parameter management DB 1003 as the overall representative color. As shown in FIG. 11, the correction parameter management DB 1003 stores and manages comprehensive representative colors that indicate characteristics related to colors representative of a plurality of captured image data 200 in a correction parameter management table. For example, in the case of the processing results shown in the comprehensive representative color management table shown in FIG. becomes.

As a result, the overall representative color calculation unit 133a uses the representative colors of all the captured image data 200 acquired by the image processing unit 12 to calculate an overall representative color that indicates characteristics related to colors that represent the plurality of captured image data. can do. Note that the method for calculating the overall representative color is not limited to the method shown in FIG. 17, and, for example, the overall representative color may be calculated by a simple average of a plurality of representative colors determined by the representative color determination unit 132a. Further, the overall representative color calculation unit 133a uses only representative colors whose brightness, hue, or saturation falls within a certain standard range from among the plurality of representative colors determined by the representative color determination unit 132a. , a comprehensive representative color calculation process may be performed.

Returning to FIG. 13, the explanation of the correction parameter calculation process will be continued. In step S107a, the RGB gain calculation unit 134a calculates an RGB gain value, which is a correction parameter, using the representative color determined by the representative color determination unit 132a and the comprehensive representative color calculated by the comprehensive representative color calculation unit 133a. do. Specifically, the RGB gain calculation unit 134a calculates the correction parameter by dividing the overall representative color by the representative color. Here, the correction parameter is an RGB gain value that is multiplied by the pixel value of each pixel constituting the photographed image 800 related to the photographed image data 200, and is calculated for each photographed image data 200. Further, since the overall representative color is one representative color calculated based on the representative color representing each photographed image data 200, it can be said to be a pixel value that represents a data set of a plurality of photographed image data 200. In other words, the image processing device 10a regards the comprehensive representative color calculated by the comprehensive representative color calculation unit 133a as the target value for brightness/tint correction, and corrects the pixel value of each captured image 800, thereby allowing multiple captured images to be processed. Brightness and color correction processing can be performed based on the image data 200.

As described above, in the process of step S15 in FIG. 12, the correction unit 14 performs color correction on each of the plurality of captured image data 200. The correction unit 14 multiplies the pixel value of each pixel constituting the photographed image 800 related to the photographed image data 200 by an RGB gain value, which is a correction parameter, to obtain corrected image data whose brightness and color tone have been corrected. generate. Since the RGB value of each pixel has information on both brightness and tint, not only the tint but also the brightness changes by multiplying by the RGB gain value, which is a correction parameter. Furthermore, if the multiplication result exceeds the maximum value of 255 for an 8-bit image, the correction unit 14 may perform a process of clipping it to 255.

Here, the corrected image displayed on the display terminal 90 will be explained. FIG. 18 is a diagram illustrating an example of a corrected image according to the first embodiment. Corrected images 850a to 850d shown in FIGS. 18A to 18D are images obtained by performing color correction on each of the photographed images 800a to 800d (see FIG. 15). As shown in FIG. 18, the corrected image 850 can be displayed without any discomfort among the plurality of images by matching the brightness and color tone.

〇Preview image display processing Here, using FIGS. 19 and 20, the preview image is provided to the user (administrator X shown in FIG. 1) using the communication terminal 50 (the viewer Y shown in FIG. 1). Processing for checking images in advance will be explained. FIG. 19 is a sequence diagram illustrating an example of preview image display processing in the communication terminal according to the first embodiment. The process shown in FIG. 19 is executed after the process in step S14 shown in FIG. 12 is performed. Note that the process shown in FIG. 19 does not have to be performed.

In step S151, the display screen generation unit 16 of the image processing device 10a generates preview image data. Specifically, first, when the correction parameter is calculated by the calculation unit 13a, the correction unit 14 of the image processing device 10a uses the predetermined resized (reduced) image data in step S101a of FIG. Performs color correction of image data. Then, the display screen generation unit 16 generates preview image data including image data subjected to color correction using the correction parameters.

In step S152, the network communication unit 11 of the image processing device 10a transmits the preview image data to the communication terminal 50 via the communication network 5. Thereby, the network communication unit 51 of the communication terminal 50 receives the preview screen data.

In step S153, the display control unit 54 of the communication terminal 50 causes the display 908 to display a preview screen 600a related to the preview image data received by the network communication unit 51.

Here, the preview screen 600a will be explained using FIG. 20. FIG. 20 is a diagram showing an example of a preview screen according to the first embodiment. A preview screen 600a shown in FIG. 20 includes a display image selection area 620a in which an image to be provided to the user (viewer Y) can be selected, and an OK button 611 to be pressed when performing color correction corresponding to the selected image. , and a cancel button 615 that is pressed to cancel the selection process. In the display image selection area 620a, preview image data before and after color correction are selectively displayed. In the example of FIG. A photographed image 800) and a corrected image after color correction (for example, corrected image 850 in FIG. 18) are displayed. As a result, the user using the communication terminal 50 (for example, the administrator (Viewer Y shown in FIG. 1) can select which image to provide. Although FIG. 20 shows a pre-correction image and a corrected image related to one captured image 800, by displaying the pre-correction image and corrected image related to a plurality of captured images 800 side by side, a plurality of captured images 800 can be displayed. 800 may be displayed so that the difference in brightness and color tone can be easily understood.

In step S154, the receiving unit 53 of the communication terminal 50 receives the selection of the display image selection area 620a of the preview screen 600a and the OK button 611. In the example of FIG. 20, the receiving unit 53 receives, for example, selection of a corrected image.

In step S155, the network communication unit 51 of the communication terminal 50 transmits information on the selection result by the reception unit 53 to the image processing device 10a via the communication network 5. The network communication unit 11 of the image processing device 10a receives information on the selection result. Thereby, the image processing device 10a performs color correction on the photographed image data 200 using the correction parameters corresponding to the image related to the received selection result, as shown in step S15 of FIG.

Note that the image processing device 10a may calculate different correction parameters for the same captured image data 200, and display a plurality of corrected images on which color correction has been performed using the different correction parameters on the preview screen 600a. In this case, the user of the communication terminal 50 can select an image that has been optimally color-corrected among a plurality of color-corrected patterns of images. Further, the different correction parameters may be adjustable (set) according to environmental conditions such as brightness of the base P, photographing conditions of the photographing device 30, and the like. Further, in the process shown in FIG. 19, an example in which the preview screen 600a is displayed on the communication terminal 50 has been described, but the image processing system 1a may be configured to display the preview screen 600a on the display terminal 90. Further, in the process of FIG. 19, an example has been described in which preview image data is generated using resized (reduced) image data, but the image processing device 10a uses real image data (captured image data 200) to generate preview image data. It may also be configured to generate image data.

●Effects of the first embodiment Therefore, the image processing system 1a according to the first embodiment calculates the representative color for each photographed image 800 from the photographed images 800 related to a plurality of photographed image data 200 having different brightness and color tone. Then, one comprehensive representative color representing each of the determined representative colors is calculated. Then, the image processing system 1a calculates a correction parameter to be used for color correction for each photographed image data 200, using the representative color in each photographed image 800 and the overall representative color common to the plurality of photographed images 800, Color correction of the captured image data 200 is performed using the calculated correction parameters. Therefore, the image processing system 1a can generate corrected image data 250 with uniform brightness and color tone for a plurality of captured image data 200 with different brightness and color tone.

●Second embodiment●
Next, an image processing system according to a second embodiment will be described. Note that the same configurations and functions as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted. The image processing system 1b according to the second embodiment corrects the brightness value of the photographed image 800 according to the photographed image data 200 instead of the color correction using the correction parameters calculated based on the overall representative color in the image processing system 1a. This is a system that performs brightness correction using

●Outline FIG. 21 is a diagram illustrating an example of an outline of processing of the image processing system according to the second embodiment. Note that FIG. 21 briefly explains the outline of the image processing system according to the second embodiment, and details such as functions realized by the image processing system 1b will be explained using drawings etc. described later. .

First, the image processing device 10b acquires a plurality of photographed image data 200 (200a, 200b, 200c, 200d) in which different subjects are photographed by the photographing device 30. The image processing device 10b calculates, for each cluster, a representative pixel value indicating a characteristic related to a color representing the cluster obtained by clustering processing of the pixel values of each pixel constituting the photographed image 800 related to the acquired photographed image data 200. do.

Next, the image processing device 10b uses the representative pixel values calculated by the representative pixel value calculation process to calculate, for each piece of photographed image data 200, a representative brightness value that indicates a characteristic related to brightness that is representative of the photographed image data 200. . Then, the image processing device 10b calculates a brightness correction parameter for performing brightness correction of the photographed image data 200 for each photographed image data 200 using the representative brightness value calculated by the representative brightness value calculation process. .

Next, the image processing device 10b performs a brightness correction process for each photographed image data 200 using the brightness correction parameter calculated by the brightness correction parameter calculation process. The image processing device 10b then transmits the corrected image data 250 (250a, 250b, 250c, 250d) on which the brightness correction process has been performed to the display terminal 90. Thereby, the display terminal 90 can display the plurality of received corrected image data 250 in which different subjects are photographed, thereby allowing the user to confirm the image in which the base P where the photographing device 30 is located is photographed.

Therefore, the image processing system 1b calculates a representative brightness value for the photographed image 800 related to the photographed image data 200, and uses the calculated representative brightness value to calculate a brightness correction parameter used for brightness correction. The image processing system 1a then corrects the brightness of the captured image 800 using the calculated brightness correction parameter. As a result, the image processing system 1b can generate a corrected image 850 in which only the brightness of the photographed image 800 is corrected, thereby increasing the convenience of the user who views the image by adjusting the brightness of each photographed image 800. Image correction can be performed in accordance with the needs of service providers and users who wish to improve the image. Note that, as shown in FIG. 21, in the following description, a configuration for correcting the brightness of a photographed image 800 related to a plurality of photographed image data 200 acquired by the image processing device 10b will be described; Since the brightness correction in is performed for each captured image 800, the image processing device 10b acquires only one captured image data 200 and performs the brightness correction of the captured image 800 related to the acquired captured image data 200. It may be a configuration.

●Functional configuration Here, the functional configuration of the image processing system according to the second embodiment will be described. Note that the functions other than the calculation unit 13b, which are functions executed by the image processing device 10b, are the same as the functions of the image processing system 1a (see FIG. 7), so the description thereof will be omitted.

FIG. 22 is a diagram illustrating an example of a detailed functional configuration of the calculation unit according to the second embodiment. The calculation unit 13b shown in FIG. 22 includes a representative pixel value calculation unit 131b, a brightness value calculation unit 132b, a representative brightness value calculation unit 133b, and a brightness correction parameter calculation unit 134b. Note that the representative pixel value calculation unit 131b is the same as the representative pixel value calculation unit 131a, so the description thereof will be omitted.

The brightness value calculation unit 132b has a function of calculating a brightness value based on the representative pixel value calculated by the representative pixel value calculation unit 131b. The brightness value calculation unit 132b converts each of the representative pixel values calculated by the representative pixel value calculation unit 131 into a brightness value.

The representative brightness value calculation unit 133b has a function of calculating a representative brightness value for each photographed image data 200 using the brightness value calculated by the brightness value calculation unit 132b. The representative brightness value is a brightness value that indicates a characteristic related to brightness that is representative of the captured image data 200. The representative brightness value calculated by the representative brightness value calculation unit 133b is an example of the first representative brightness value. The representative brightness value calculation unit 133b is an example of representative brightness value determining means.

The brightness correction parameter calculation unit 134b has a function of calculating a brightness correction parameter for correcting the brightness of the captured image 800 using the representative brightness value calculated by the representative brightness value calculation unit 133b. The brightness correction parameter calculating unit 134b calculates brightness correction parameters using the representative brightness value and an LUT (Lookup table) (see FIG. 25) stored in the storage unit 1000. The brightness correction parameter calculation unit 134b is an example of correction parameter calculation means.

〇Representative Brightness Value Management Table FIG. 23 is a conceptual diagram showing an example of the representative brightness value management table according to the second embodiment. In the storage unit 1000, a pixel value management DB 1001 is constructed, which is constituted by a representative brightness value management table as shown in FIG. In the representative brightness value management table shown in FIG. It is managed for each image ID. In the representative brightness value management table, the number of pixels included in the cluster, the cluster center pixel, and the brightness value calculated using the cluster center pixel are stored in association with each other for each cluster number for identifying divided clusters. It is well managed. Further, in the representative brightness value management table, representative brightness values indicating characteristics related to brightness that represent the photographed image data 200 are stored and managed for each photographed image data 200. Specific processing for obtaining the processing results shown in FIG. 23 will be described later.

●Processing or operation of second embodiment Next, processing or operation of the image processing system 1b according to the second embodiment will be described using FIGS. 24 and 26. Note that the processing other than the correction parameter calculation processing shown in step S14 in FIG. 12 is the same as the processing or operation of the image processing system 1a, so the explanation will be omitted.

FIG. 24 is a flowchart illustrating an example of correction parameter calculation processing in the image processing apparatus according to the second embodiment. Note that the processing in step S101b and the processing in step S102b are the same as step S101a and step S102a shown in FIG. 13, so their explanation will be omitted.

In step S103b, the brightness value calculation unit 132b of the calculation unit 13b calculates the brightness value in each cluster using the cluster center pixel calculated by the representative pixel value calculation unit 131b. Specifically, the brightness value calculation unit 132b calculates the brightness value from the RGB pixel values using a known conversion formula. As an example, the brightness value calculation unit 132b uses the ITU-R BT.601 conversion formula shown in Equation 1 below.

More appropriately, since the conversion formula changes depending on the color space in which the RGB pixel value is calculated, the brightness value calculation unit 132b selects an appropriate conversion formula to calculate the brightness value. It is preferable to do so.

In step S104b, the storage/readout unit 19 stores the number of pixels included in the cluster and the cluster center pixel calculated by the representative pixel value calculation unit 131b, and the brightness value calculated by the brightness value calculation unit 132b in the pixel value management DB 1001. Remember. As shown in FIG. 23, the pixel value management DB 1001 stores and manages the number of pixels, cluster center pixel, and brightness value for each cluster in a representative brightness value management table in association with the image ID.

In step S105b, the representative brightness value calculation unit 133b calculates a representative brightness value indicating characteristics related to brightness representative of the captured image data 200 using the brightness value calculated by the brightness value calculation unit 132b. Specifically, the representative brightness value calculation unit 133b calculates the representative brightness value of the captured image data 200 using the k brightness values calculated by the brightness value calculation unit 132b and the number of pixels belonging to the corresponding cluster. calculate. In this case, the representative brightness value calculation unit 133b calculates the representative brightness value by weighted averaging using the k brightness values and the number of pixels belonging to the corresponding cluster. Note that here we will explain an example of calculating the representative brightness value using a weighted average, but in addition to weighting based on the pure number of pixels, it is also possible to increase the weighting of clusters with a large number of pixels, or to increase the weighting of clusters with a small number of pixels. The configuration may be such that the representative brightness value is calculated by reducing the weight.

In step S106b, the storage/reading unit 19 stores the representative brightness value calculated by the representative brightness value calculation unit 133b in the pixel value management DB 1001. As shown in FIG. 23, the pixel value management DB 1001 stores and manages representative brightness values corresponding to the photographed image data 200 in a representative brightness value management table for each image ID for identifying the photographed image data 200. .

In step S107b, the brightness correction parameter calculation unit 134b calculates a brightness correction parameter for correcting the brightness of the captured image data 200 using the representative brightness value calculated by the representative brightness value calculation unit 133b. Here, the brightness correction parameter calculation process will be described using FIG. 25. FIG. 25 is a diagram illustrating an example of brightness correction parameters according to the second embodiment. FIG. 25 shows an example of calculating an LUT for changing the brightness of a photographed image according to the representative brightness value calculated by the representative brightness value calculation unit 133b as a brightness correction parameter. FIG. 25 shows three LUTs as examples of brightness correction parameters. The LUT is used to convert input data into a form suitable for processing, and the horizontal axis represents input values and the vertical axis represents output values. The input data in this case is the brightness (luminance value) of each photographed image data 200, and FIG. 25 shows the relationship between the possible range of pixel values and the output, standardized to a value of 0-1. has been done. Example 1 is a LUT that greatly increases the brightness from low to high brightness, example 2 is a LUT that has about half the conversion of example 1, and example 3 is a linear LUT where input and output do not change.

The brightness correction parameter calculation unit 134b determines which LUT to use according to the representative brightness value calculated by the representative brightness value calculation unit 133b. For example, if the representative brightness value is smaller than 100, it is determined that the image is dark, and the LUT of example1 is calculated (adopted). Furthermore, if the representative brightness value is greater than 150, it is determined that the image is bright and example 3 is calculated (adopted). Furthermore, if the representative brightness value is 100 or more and 150 or less, an LUT whose slope is changed according to the value (representative brightness value) is calculated (adopted). For example, the LUT of example2 shows an example of the LUT when the representative brightness value is 125.

Here, the brightness correction parameter calculation unit 134b uses a plurality of LUTs stored in advance in the storage unit 1000 to select an appropriate LUT according to the representative brightness value calculated by the representative brightness value calculation unit 133b. Select as correction parameter. Note that the brightness correction parameter calculation unit 134b has an S-shaped input/output characteristic suitable for brightness correction of the image according to the calculated representative brightness value by using, for example, a sigmoid curve (sigmoid function). The configuration may be such that the LUT is dynamically calculated. Although FIG. 25 shows an example of three LUTs (example 1 to example 3), the brightness correction parameter calculation unit 134b has a configuration that calculates brightness correction parameters more precisely using four or more LUTs. It may be.

In step S108b, if the image processing unit 12 has not read all of the captured image data 200 received by the network communication unit 11, the image processing device 10b repeats the processing from step S101b. On the other hand, when the image processing unit 12 reads all of the captured image data 200 received by the network communication unit 11, the image processing device 10b ends the process.

Thereby, the correction unit 14 of the image processing device 10b performs brightness correction on the captured image data 200 using the brightness correction parameter calculated by the calculation unit 13b, as shown in step S15 of FIG. When the captured image data 200 is RGB data, brightness correction using a brightness correction parameter (LUT) is performed by converting the RGB image format to a YCbCr image format representing brightness and color difference, and applying the brightness correction parameter. By applying (LUT) only to the brightness, it is possible to change the brightness while maintaining the color tone. Note that when a brightness correction parameter (LUT) is applied to each of the RGB pixels, although some color changes occur, it is possible to generate an image whose brightness has generally changed as expected.

Here, the preview screen 600b will be explained using FIG. 26. FIG. 26 is a diagram showing an example of a preview screen according to the second embodiment. A display image selection area 620b included in the preview screen 600b shown in FIG. 26 differs from the display image selection area 620a shown in FIG. 20 in that preview image data related to a plurality of captured image data 200 is displayed. In addition, on the preview screen 600b, for each piece of photographed image data 200, a pre-correction image before brightness correction or a corrected image after brightness correction is selectably displayed. As a result, the user using the communication terminal 50 (for example, the administrator It is possible to select which image to provide to the user. In addition, the user who uses the communication terminal 50 can simultaneously check the images before and after correction related to a plurality of photographed image data 200, thereby making the user feel uncomfortable when, for example, the user is allowed to view a plurality of photographed images 800. Preview image data related to each captured image 800 can be selected so as to prevent this from occurring. Note that the image processing device 10b may be configured to display only images before and after correction related to one captured image data 200, as in the preview screen 600a shown in FIG. 20. Similarly, the image processing device 10a according to the first embodiment may be configured to display images before and after correction regarding a plurality of captured image data 200, as in the preview screen 600b shown in FIG.

●Effects of the second embodiment Therefore, the image processing system 1b according to the second embodiment calculates the representative brightness value for the photographed image 800 according to the photographed image data 200, and uses the calculated representative brightness value to calculate the brightness. A brightness correction parameter used for the brightness correction is calculated, and the brightness of the captured image 800 is corrected using the calculated brightness correction parameter. Therefore, the image processing system 1b can generate corrected image data 250 in which only the brightness is corrected for the captured image data 200.

●Variation example 1 of the second embodiment●
Next, an image processing system according to Modification 1 of the second embodiment will be described. The image processing system 1bb according to the first modification of the second embodiment is a system that corrects the brightness of a plurality of photographed image data using a comprehensive representative brightness value that indicates a characteristic related to brightness that is representative of the plurality of photographic image data. It is. The image processing system 1bb performs brightness correction based on brightness-related characteristics of a plurality of captured images 800, compared to performing brightness correction for each captured image 800 using representative brightness values of individual captured images 800. Therefore, when the user is allowed to view the plurality of captured images 800, the brightness between the plurality of captured images 800 can be automatically adjusted. Note that in the image processing system 1b described above, it has been explained that the number of captured image data 200 acquired by the image processing device 10b may be plural or one; It is assumed that a plurality of photographed image data 200 are acquired by the image processing apparatus 10bb according to the first modification of the second embodiment in order to make the brightness uniform among the images.

●Functional Configuration Here, the functional configuration of the image processing system according to Modification 1 of the second embodiment will be described. Note that the functions other than the calculation unit 13bb, which are functions executed by the image processing device 10bb, are the same as the functions of the image processing system 1a (see FIG. 7), so the description thereof will be omitted.

FIG. 27 is a diagram illustrating an example of a detailed functional configuration of a calculation unit according to Modification 1 of the second embodiment. In addition to the configuration of the calculation unit 13b shown in FIG. 22, the calculation unit 13bb shown in FIG. 27 includes a comprehensive representative brightness value calculation unit 135b.

The overall representative brightness value calculation unit 135b has a function of calculating an overall representative brightness value that indicates characteristics related to brightness that represent the plurality of captured image data 200, using the representative brightness value calculated by the representative brightness value calculation unit 133b. . The overall representative brightness value calculated by the overall representative brightness value calculation unit 135b is an example of the second representative brightness value. The comprehensive representative brightness value calculation unit 135b is an example of representative brightness value calculation means.

Further, the brightness correction parameter calculation unit 134b in the calculation unit 13bb calculates, based on the representative brightness value calculated by the representative brightness value calculation unit 133b and the total representative brightness value calculated by the total representative brightness value calculation unit 135b, A brightness correction parameter for each photographed image 800 is calculated.

Comprehensive Representative Brightness Value Management Table FIG. 28 is a conceptual diagram showing an example of a comprehensive representative brightness value management table according to Modification 1 of the second embodiment. In the storage unit 1000, a pixel value management DB 1001 is constructed, which is composed of a comprehensive representative brightness value management table as shown in FIG. The comprehensive representative brightness value management table shown in FIG. 28 shows the results of clustering processing of a plurality of representative brightness values calculated by the representative brightness value calculation unit 133b. The comprehensive representative brightness value management table includes photographed image data 200 corresponding to the cluster center brightness, which is the center (center of gravity) of the cluster, and the representative brightness value belonging to the cluster, for each cluster number for identifying divided clusters. An image ID for identification is stored and managed in association with the image ID. In the example of FIG. 28, the cluster with cluster number "1" has a cluster center luminance of "117.31771" and a representative luminance value corresponding to one photographed image data 200 identified by image ID "P01". belong to In addition, the cluster No. 2 has a cluster center luminance of 122.58848, and each of the three photographed image data 200 identified by the image IDs "P02", "P03", and "P04" has a cluster center brightness of "122.58848". The corresponding representative luminance value belongs.

- Correction Parameter Management Table FIG. 29 is a conceptual diagram showing an example of a correction parameter management table according to Modification 1 of the second embodiment. In the storage unit 1000, a correction parameter management DB 1003 is constructed, which is composed of a correction parameter management table as shown in FIG. In the correction parameter management table shown in FIG. 29, representative brightness values calculated by the representative brightness value calculation unit 133b are stored and managed for each image ID for identifying the captured image data 200. Further, in the correction parameter management table shown in FIG. 29, the overall representative brightness value calculated by the overall representative brightness value calculation unit 135b is commonly stored and managed for a plurality of image IDs.

Processing or Operation of Modification 1 of Second Embodiment Next, processing or operation of the image processing system 1bb according to Modification 1 of the second embodiment will be described using FIGS. 30 and 33. FIG. 30 is a flowchart illustrating an example of correction parameter calculation processing in the image processing apparatus according to Modification 1 of the second embodiment. Note that the processing from step S101bb to step S106bb in FIG. 30 is the same as the processing from step S101b to step S106b shown in FIG. 24, so the description thereof will be omitted.

In step S107bb, if the image processing unit 12 has not read all of the plurality of captured image data 200 received by the network communication unit 11, the image processing device 10bb repeats the processing from step S101bb. On the other hand, when the image processing unit 12 reads all of the plurality of captured image data 200 received by the network communication unit 11, the image processing device 10bb shifts the process to step S108bb.

In step S108bb, the calculation unit 13bb of the image processing device 10bb calculates a total representative brightness value using the representative brightness value calculated in the process of step S105bb. Here, details of the calculation process of the overall representative brightness value will be explained using FIG. 31. FIG. 31 is a flowchart illustrating an example of overall representative brightness value calculation processing in the image processing apparatus according to Modification 1 of the second embodiment.

In step S108bb-1, the overall representative brightness value calculation unit 135b classifies the representative brightness values calculated by the representative brightness value calculation unit 133b into a plurality of clusters by clustering processing. The clustering process by the overall representative brightness value calculation unit 135b is performed using the K-means method, similar to the clustering process by the representative pixel value calculation unit 131b. The input data in this case is a plurality of representative brightness values calculated by the representative brightness value calculation unit 133b, and is grouped into clusters using the spatial distances of the brightness values.

In step S108bb-2, the overall representative brightness value calculation unit 135b calculates cluster center brightness, which is a brightness value representative of each classified cluster. The overall representative brightness value calculation unit 135b calculates the brightness value of the center of the cluster grouped by the process of step S108bb-1 as the cluster center brightness.

In step S108bb-3, the storage/reading unit 19 stores the image ID related to the representative brightness value belonging to each cluster and the cluster center brightness in the pixel value management DB 1001 in association with each other for each cluster. As shown in FIG. 28, the pixel value management DB 1001 stores and manages cluster center brightness and image ID in association with each other in a comprehensive representative brightness value management table for each cluster.

In step S108bb-4, the storage/reading unit 19 stores the cluster center brightness associated with the cluster having the largest number of representative brightness values in the correction parameter management DB 1003 as a comprehensive representative brightness value. As shown in FIG. 29, the correction parameter management DB 1003 stores and manages comprehensive representative brightness values indicating characteristics related to brightness representative of a plurality of captured image data 200 in a correction parameter management table. For example, in the case of the processing results shown in the overall representative brightness value management table shown in FIG. 28, "122.58848", which is the cluster center brightness of cluster No. "2", becomes the comprehensive representative brightness value.

Thereby, the overall representative brightness value calculation unit 135b uses the representative brightness values of all the captured image data 200 acquired by the image processing unit 12 to calculate the overall representative brightness value that indicates the characteristics related to the brightness that is representative of the plurality of captured image data. The value can be calculated. Note that the method for calculating the overall representative brightness value is not limited to the method shown in FIG. 31; for example, the overall representative brightness value may be calculated by a simple average of a plurality of representative brightness values calculated by the representative brightness value calculation unit 133b. Good too.

Returning to FIG. 30, the explanation of the correction parameter calculation process will be continued. In step S109bb, the brightness correction parameter calculation unit 134b uses the representative brightness value calculated by the representative brightness value calculation unit 133b and the total representative brightness value calculated by the total representative brightness value calculation unit 135b to calculate the captured image data. A brightness correction parameter for correcting the brightness of 200 is calculated. Here, the brightness correction parameter calculation process will be described using FIGS. 32 and 33. FIG. 32 is a diagram illustrating an example of brightness correction parameters according to Modification 1 of the second embodiment. FIG. 33 is a diagram illustrating an example of brightness correction gain according to Modification 1 of the second embodiment. 32 and 33 show that the brightness of a photographed image is adjusted as a brightness correction parameter according to the representative brightness value calculated by the representative brightness value calculation unit 133b and the total representative brightness value calculated by the total representative brightness value calculation unit 135b. An example of calculating an LUT for changing the brightness will be shown below. FIG. 32 shows four LUTs as examples of brightness correction parameters. Note that Figure 32 is the same as the example shown in Figure 25, except that the LUT of example 4 is added.

The brightness correction parameter calculation unit 134b uses the representative brightness value calculated by the representative brightness value calculation unit 133b and the overall representative brightness value calculated by the overall representative brightness value calculation unit 135b to determine which LUT to use. do. For example, if the representative brightness value is smaller than 100, it is determined that the image is dark, and the LUT of example1 is calculated (adopted). Furthermore, if the representative brightness value is greater than 150, it is determined that the image is bright and example 3 is calculated (adopted). Further, when the representative brightness value is between 100 and 150, an LUT whose slope is changed according to that value (representative brightness value) and the overall representative brightness value is calculated (adopted).

FIGS. 33(a) and 33(b) show the brightness correction gain, which is a parameter used to determine the brightness correction parameter. The brightness correction gain is a parameter for calculating the slope of the LUT, which is a brightness correction parameter, and the slope of the LUT is determined by the first brightness correction gain shown in FIG. It is calculated based on the second brightness correction gain shown in 33(b). In FIG. 33(a), the relationship between the possible range of pixel values and the brightness correction gain value (first brightness correction gain value) is shown normalized to 0-1, and the overall representative brightness value is The relationship between the value divided by the representative brightness value and the brightness correction gain value (second brightness correction gain value) is shown.

Here, as an example, a method of calculating the brightness correction parameter when the representative brightness value is 125 and the overall representative brightness value is 100 will be described. When the photographed image 800 whose brightness is to be corrected is an 8-bit image, first, the brightness correction parameter calculation unit 134b calculates that the brightness normalized to 0-1 is "125/255=0.5". Therefore, using FIG. 33(a), the first brightness correction gain value is calculated as 3.0. On the other hand, since the overall representative brightness value is 100, the brightness correction parameter calculation unit 134b uses FIG. The second brightness correction gain value corresponding to " is calculated as 0.6. Then, the brightness correction parameter calculation unit 134b calculates the value obtained by multiplying the first brightness correction gain value and the second brightness correction gain value (“3.0×0.8=2.4”) into the LUT. Calculate as a slope.

In this way, the brightness correction parameter calculation unit 134b uses not only the representative brightness value calculated by the representative brightness value calculation unit 133b but also the overall representative brightness value calculated by the overall representative brightness value calculation unit 135b to adjust the brightness. Calculate the correction parameters. In other words, as shown in the second embodiment, when only the representative brightness value is used, the brightness correction parameter calculation unit 134b calculates the LUT of example2 as the brightness correction parameter, but the brightness correction parameter calculation unit 134b calculates the LUT of example 4 as a brightness correction parameter by using the overall representative brightness value.

Note that the brightness correction parameter calculation unit 134b may select an LUT according to the brightness correction gain value as the brightness correction parameter using a plurality of LUTs stored in advance in the storage unit 1000 as described above. However, by using a sigmoid curve (sigmoid function), a configuration may be adopted in which an LUT having an S-shaped input/output characteristic suitable for correcting the brightness of the image is dynamically calculated. Further, when performing brightness correction using only the representative brightness value as shown in the second embodiment, the brightness correction parameter calculation unit 134b calculates the first brightness correction gain shown in FIG. 33(a). A configuration may be adopted in which an LUT having a corresponding slope (first brightness correction gain value) is calculated as a brightness parameter using only the values.

Thereby, the correction unit 14 of the image processing device 10bb corrects the brightness of the plurality of captured image data 200 using the brightness correction parameter calculated by the calculation unit 13bb, as shown in step S15 of FIG. Therefore, each corrected image data 250 can be generated so that the brightness of the plurality of captured images 800 approaches the reference brightness (total representative brightness value) calculated from the plurality of captured image data 200. It is possible to perform image correction according to the needs of service providers and users who wish to improve the convenience of users who view images by adjusting the images.

●Modification 2 of the second embodiment●
Next, an image processing system according to a second modification of the second embodiment will be described. The image processing system 1bbb according to the second modification of the second embodiment is a system that selects representative brightness values used in the overall representative brightness value calculation process (see FIG. 31) in the first modification. For example, in a case where images used for a real estate preview service include images of a night view from a balcony, there may be an extreme difference in brightness between the plurality of captured images 800. If such an image is subjected to brightness correction processing using the comprehensive representative brightness value as in Modified Example 1 above, the brightness between the captured images 800 will be equalized, but the brightness of the image whose brightness is not appropriate will be As a result, the brightness of other images may deteriorate. Therefore, the image processing system 1bbb according to the second modification of the second embodiment sets a predetermined condition for the representative brightness value used in the calculation process of the overall representative brightness value, and performs the process using only the brightness value that satisfies the predetermined condition. I do.

Specifically, the overall representative brightness value calculation unit 135b of the calculation unit 13bbb according to the second modification of the second embodiment performs the calculation by the representative brightness value calculation unit 133b as pre-processing of step S108bb-1 shown in FIG. It is determined whether each of the calculated representative brightness values is within a predetermined brightness range. Then, the overall representative brightness value calculation unit 135b performs the processes from step S108bb-1 onwards using the representative brightness value determined to be within the predetermined brightness range. Note that for the photographed image 800 corresponding to the representative brightness value that was determined to be outside the predetermined brightness range and was not used in the processing by the comprehensive representative brightness value calculation unit 135b, the brightness No correction shall be made. As a result, the image processing device 10bbb according to the second modification of the second embodiment can integrate the luminance values of images whose brightness is not appropriate even when performing brightness correction using a plurality of captured images 800. For example, only the brightness value of the image to be corrected can be used for calculating the overall representative brightness value without using it for calculating the representative brightness value. Therefore, even when adjusting the brightness between a plurality of captured images 800, the image processing device 10bbb can perform appropriate brightness correction that meets the needs of the service provider and the user.

●Third embodiment●
Next, an image processing system according to a third embodiment will be described. Note that the same configurations and functions as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted. The image processing system 1c according to the third embodiment allows the image processing device 10c to calculate both the correction parameters as shown in the first embodiment and the brightness correction parameters as shown in the second embodiment. It is a system with many functions.

●Functional Configuration Here, the functional configuration of the image processing system according to the third embodiment will be described. Note that the functions executed by the image processing device 10c, other than the calculation unit 13c, are the same as the functions of the image processing system 1a (see FIG. 7), and therefore the description thereof will be omitted.

FIG. 34 is a diagram illustrating an example of a detailed functional configuration of the calculation unit according to the third embodiment. The calculation unit 13c shown in FIG. 34 includes a representative pixel value calculation unit 131c, a representative color determination unit 132c, a comprehensive representative color calculation unit 133c, an RGB gain calculation unit 134c, a brightness value calculation unit 135c, a representative brightness value calculation unit 136c, and a brightness value calculation unit 135c. It includes a correction parameter calculation unit 137c.

The representative pixel value calculation section 131c, the representative color determination section 132c, the comprehensive representative color calculation section 133c, and the RGB gain calculation section 134c are the representative pixel value calculation section 131a, the representative color determination section 132a, the comprehensive representative color calculation section 133a, and the RGB gain calculation section 134c. This function is similar to that of the section 134a. Further, the brightness value calculation unit 135c, the representative brightness value calculation unit 136c, and the brightness correction parameter calculation unit 137c have the same functions as the brightness value calculation unit 132b, the representative brightness value calculation unit 133b, and the brightness correction parameter calculation unit 134b. . The calculation unit 13c calculates either the RGB gain value by the RGB gain calculation unit 134c or the brightness correction parameter by the brightness correction parameter calculation unit 137c, based on the user's designation and pre-stored settings, or You can do both of these. The image processing device 10c uses either or both of the correction parameters calculated by the RGB gain calculation unit 134c and the brightness correction parameters calculated by the brightness correction parameter calculation unit 137c to process the captured image data 200. Image correction of the captured image 800 can be performed.

●Effects of the third embodiment Therefore, the image processing system 1c according to the third embodiment calculates both the correction parameter, which is the RGB gain value, and the brightness correction parameter by the calculation unit 13c of the image processing device 10c. I can do it. By calculating these two correction parameters, the image processing device 10c can mainly correct the hue using the RGB gain value and correct the brightness of the captured image 800 using the brightness correction parameter. When displaying a tour in a real estate viewing service, it is preferable that the displayed images have the same color tone and have appropriate brightness as described above, so the image processing device 10c is capable of adjusting both color and brightness. By correcting (adjusting), a more preferable image can be displayed on the display terminal 90. Furthermore, the image processing device 10c has RGB gain values whose main purpose is color correction, and an LUT whose main purpose is brightness correction. correction processing can be performed. In other words, the image processing device 10c can perform different correction processes such as correcting color and brightness, correcting only color, and correcting only brightness. Furthermore, the image processing device 10c uses the result of multiplying the RGB gain value calculated by the RGB gain calculation unit 134c to the brightness value calculated by the brightness value calculation unit 135c, and the brightness value calculated by the representative brightness value calculation unit 136c. By calculating the average value with the representative brightness value, the correction result by multiplication of the RGB gain value can also be taken into consideration in calculating the representative brightness value.

●Variation 1 of the third embodiment●
Next, an image processing system according to modification 1 of the third embodiment will be described. In the image processing system 1cc according to the first modification of the third embodiment, the image processing device 10cc uses the correction parameters as shown in the first embodiment and the modification 1 or 2 of the second embodiment. This system has the function of being able to calculate both of the brightness correction parameters as shown.

●Functional Configuration Here, the functional configuration of the image processing system according to Modification 1 of the third embodiment will be described. Note that the functions other than the calculation unit 13cc, which are functions executed by the image processing device 10cc, are the same as the functions of the image processing system 1a (see FIG. 7), so a description thereof will be omitted.

FIG. 35 is a diagram illustrating an example of a detailed functional configuration of a calculation unit according to Modification 1 of the third embodiment. In addition to the configuration of the calculation unit 13c shown in FIG. 34, the calculation unit 13cc shown in FIG. 35 includes a comprehensive representative brightness value calculation unit 138c. The overall representative brightness value calculation unit 138c has the same function as the overall representative brightness value calculation unit 135b. Similar to the calculation unit 13c shown in the third embodiment, the calculation unit 13cc calculates the RGB gain value and the brightness correction parameter by the RGB gain calculation unit 134c based on the user's designation and pre-stored settings. Either or both of the brightness correction parameter calculations by the calculation unit 137c can be performed. In addition, the image processing device 10cc uses either or both of the correction parameters calculated by the RGB gain calculation unit 134c and the brightness correction parameters calculated by the brightness correction parameter calculation unit 137c. Image correction of the captured image 800 related to the data 200 can be performed.

●Other embodiments●
Next, the configuration of an image processing system according to another embodiment will be described. Note that the same configurations and the same functions as in each of the above embodiments are denoted by the same reference numerals, and the description thereof will be omitted.

First, as shown in FIGS. 36 and 37, the image processing system 1d is a system in which an image processing device 40a that performs color correction or brightness correction of photographed image data 200 is present within a base P. FIG. 36 is a diagram illustrating an example of a system configuration of an image processing system according to another embodiment. The image processing system 1d shown in FIG. 36 differs from the above-described embodiment (for example, see FIG. 1) in that it includes an image processing device 40a and a web server 70.

The image processing device 40a, like the image processing devices 10a, 10b, 10bb, 10bbb, 10c, and 10cc described above, calculates the captured image data 200 to be corrected based on the plurality of captured image data 200 transmitted from the communication terminal 50. Performs correction processing. The web server 70 is a web server that provides a predetermined web application via the communication network 5. The web server 70 provides a web application for displaying, on the display terminal 90, the image that has been corrected by the image processing device 40a. Note that the image processing device 40a and the Web server 70 will be described as having the same hardware configuration as the image processing device 10a shown in FIG. 3, for example.

FIG. 37 is a diagram illustrating an example of a functional configuration of an image processing system according to another embodiment. The functions realized by the image processing device 40a include a network communication section 41, a transmission/reception section 42, an image processing section 43, a calculation section 44, a correction section 45, a judgment section 46, a storage/reading section 49, and a storage section 4000. Note that the network communication section 41, image processing section 43, correction section 45, judgment section 46, storage/reading section 49, and storage section 4000 are the same as the network communication section 11, image processing section 12, correction section 14, and judgment section 15 described above. , the configuration is similar to that of the storage/reading section 19 and the storage section 1000, so the explanation will be omitted. Further, the calculation unit 44 may include any of the functions of the calculation unit 13a, calculation unit 13b, calculation unit 13bb, calculation unit 13bbb, calculation unit 13c, and calculation unit 13cc described above.

The transmitting/receiving unit 42 has a function of communicating with the communication terminal 50 using short-range wireless communication technology such as Wi-Fi. The transmitting/receiving unit 42 is mainly realized by the processing of the network I/F 109 and the CPU 101 shown in FIG.

The functions realized by the web server 70 include a network communication section 71 and a display screen generation section 72. The network communication unit 71 has a function of transmitting and receiving various data (or information) to and from the image processing device 40a, the communication terminal 50, or the display terminal 90 via the communication network 5. The network communication unit 71 receives, for example, corrected image data 250 transmitted from the image processing device 40a and corrected by the image processing device 40a. Further, the network communication unit 71 transmits the corrected image data 250 transmitted by the image processing device 40a to the display terminal 90 via the communication network 5. The network communication unit 71 is mainly realized by the network I/F 109 shown in FIG. 3 and the processing of the CPU 101.

The display screen generation unit 72 has a function of generating various display screens to be displayed on the communication terminal 50 or the display terminal 90 using a web application. The display screen generation unit 72 generates a preview screen for display on the communication terminal 50, for example. The display screen generation unit 72 generates, for example, a display screen for displaying the corrected image data on the display terminal 90. The display screen generation unit 72 is mainly realized by the processing of the CPU 101 shown in FIG.

Thereby, the image processing system 1d can perform the above-described correction processing according to the present invention using the image processing device 40a located at the base P, and display the corrected image on the display terminal 90 via the Web server 70. can be done.

Next, another example of an image processing system according to another embodiment will be described using FIGS. 38 and 39. FIG. 38 is a diagram illustrating an example of a system configuration of an image processing system according to another embodiment. In the image processing system 1e shown in FIG. 38, an image processing device 40b that performs color correction or brightness correction of photographed image data 200 exists within a base P, and a communication terminal 50 generates a corrected image corrected by the image processing device 40b. This is a system for displaying.

FIG. 39 is a diagram illustrating an example of a functional configuration of an image processing system according to another embodiment. The function realized by the image processing device 40b is different from the image processing device 40a (see FIG. 37) in that it further includes a display screen generation section 47. The display screen generation unit 47 has a function of generating various display screens to be displayed on the communication terminal 50. The display screen generation unit 47 generates a preview screen for display on the communication terminal 50, for example. The display screen generation unit 72 generates, for example, a display screen for displaying the corrected image data on the communication terminal 50. The display screen generation unit 47 is mainly realized by the processing of the CPU 101 shown in FIG.

Thereby, the image processing system 1e can use the services provided by the image processing system 1e without going through the communication network 5.

In each of the embodiments described above, the image processing apparatuses (10a, 10b, 10bb, 10bbb, 10c, 10cc, 40a, 40b) may be configured to use correction parameters or Although an example has been described in which image correction is performed on the acquired photographic image data 200 using the brightness parameter, the image processing device stores the calculated parameters in the storage unit 1000 or the storage unit 4000, and performs image correction on the newly acquired photographic image data 200. The configuration may be such that image correction is performed on the image data 200 using stored parameters. In this case, the image processing apparatus does not need to calculate correction parameters and the like each time, for example, when performing image correction on the captured images 800 that share a common shooting scene, so that the processing load can be reduced.

●Summary●
As described above, an image processing apparatus according to an embodiment of the present invention is an image processing apparatus that performs color correction of photographed image data 200 to be corrected based on a plurality of photographed image data 200 (an example of image data). 10a, 10c, 10cc, 40a, 40b, based on the pixel value of each pixel constituting the captured image 800 related to the captured image data 200, a representative color ( An example of a first representative color) is determined for each photographed image data 200, and based on the determined representative colors, a comprehensive representative color (an example of a second representative color) is determined for each photographed image data 200. An example of a representative color) is calculated. Then, the image processing devices 10a, 10c, 10cc, 40a, and 40b use the representative color corresponding to the photographed image data 200 to be corrected and the overall representative color for color correction of the photographed image data 200 to be corrected. A correction parameter is calculated, and the color correction of the photographed image data 200 to be corrected is performed using the calculated correction parameter. Thereby, the image processing apparatuses 10a, 10c, 10cc, 40a, and 40b can perform correction to make the brightness and color tone uniform for a plurality of photographed image data 200 in which different subjects are photographed. Furthermore, although a spherical (360°) panoramic image has been shown as an example of a photographed image, the image processing apparatus of the present invention can also take a general photographic image that can be taken with a normal digital camera or smartphone. is possible.

Further, the image processing device according to an embodiment of the present invention calculates, for each cluster, a representative pixel value indicating a characteristic related to a color representing the cluster obtained by clustering processing of pixel values, and A representative pixel value corresponding to a specific cluster with a large number of clusters is determined as a representative color (an example of a first representative color) that indicates a characteristic related to a color that represents the captured image data 200 (an example of image data). Thereby, the image processing devices 10a, 10c, 10cc, 40a, and 40b represent clusters formed based on the pixel values of each pixel constituting the captured image 800 for each captured image 800 related to the captured image data 200. By calculating representative pixel values, which are characteristics related to color, for each cluster included in the photographed image 800, correction parameters used for color correction of the photographed image data 200 can be calculated.

Furthermore, the image processing device according to an embodiment of the present invention performs clustering processing on a representative color (an example of a first representative color) showing characteristics related to a color representative of a plurality of captured image data 200 (an example of image data). Among the obtained clusters, the representative pixel value of the color representing the specific cluster that contains the most representative colors in the cluster is determined by the general representative color (second An example of a representative color). As a result, the image processing devices 10a, 10c, 10cc, 40a, and 40b regard the overall representative color commonly used in a plurality of photographed image data 200 as the target value for brightness and color correction, and apply the same to each photographed image data 200. By correcting the pixel values of the captured image 800, brightness and color correction processing can be performed based on the plurality of captured image data 200.

Further, the image processing device according to an embodiment of the present invention can process a plurality of comprehensive representative colors (an example of a second representative color) that indicate characteristics related to colors representative of the plurality of captured image data 200 (an example of image data). A correction parameter, which is an RGB gain value, is calculated by dividing the captured image data 200 of 200 by a representative color (an example of a first representative color) that indicates a characteristic related to a representative color. As a result, the image processing devices 10a, 10c, 10cc, 40a, and 40b use the representative color calculated for each photographed image data 200 and the overall representative color commonly used for a plurality of photographed image data 200 to Correction can be performed on a plurality of photographed image data 200 in which a subject is photographed so that the brightness and color tone are uniform.

Furthermore, the image processing apparatus according to an embodiment of the present invention processes the photographed image data 200 to be corrected based on the pixel value of each pixel constituting the image related to the photographed image data 200 to be corrected (an example of image data). A representative brightness value indicating characteristics related to brightness that is representative of the brightness is determined, and using the determined representative brightness value, a brightness correction parameter for performing brightness correction of the photographed image data 200 to be corrected is calculated. The color correction and brightness correction of the photographed image data 200 to be corrected are performed using the correction parameters and brightness correction parameters used for color correction of the photographed image data 200 of. Thereby, the image processing devices 10c, 40a, and 40b can perform brightness correction on a plurality of photographed image data 200 in which different subjects are photographed.

Further, the image processing systems 1a, 1c, and 1d according to an embodiment of the present invention can communicate with the image processing devices 10a, 10c, and 40a via the communication network 5. A photographing system 7 including a photographing device 30 connected to the camera and photographing a subject is provided. The image processing devices 10a, 10c, 10cc, and 40a reduce the image size of photographed image data 200 (an example of image data), and generate a plurality of preview image data in which color correction has been performed on the reduced image data. , is transmitted to the photographing system 7. The photographing system 7 also accepts selection of specific preview image data from among the plurality of preview image data transmitted from the image processing devices 10a, 10c, 10cc, and 40a, and converts the received specific preview image data into an image. It is transmitted to the processing devices 10a, 10c, 10cc, and 40a. Then, the image processing devices 10a, 10c, 10cc, and 40a perform color correction on the photographed image data 200 using the correction parameters corresponding to the specific preview image data transmitted from the photographing system 7. As a result, the image processing systems 1a, 1c, and 1d allow the user using the imaging system 7 (for example, the administrator Among them, the user (viewer Y shown in FIG. 1) can be allowed to select which image to provide.

Further, the image processing systems 1a, 1c, and 1d according to an embodiment of the present invention include a display terminal 90 that can communicate with the image processing devices 10a, 10c, 10cc, and 40a via the communication network 5. , 10c, 10cc, and 40a transmit a plurality of corrected image data 250 on which color correction has been performed to the display terminal 90. Then, the display terminal 90 displays the plurality of corrected image data 250 transmitted from the image processing devices 10a, 10c, 10cc, and 40a on the display 908 (an example of a display unit). Thereby, the image processing systems 1a, 1c, and 1d can display on the display terminal 90 a plurality of corrected images that have been corrected by the image processing devices 10a, 10c, 10cc, and 40a and have the same brightness and color tone. .

Further, the image processing method according to an embodiment of the present invention includes image processing apparatuses 10a, 10c, and 10cc, 40a, 40b executes an image processing method, which is a representative method that indicates characteristics related to colors representative of the photographed image data 200 based on the pixel values of each pixel constituting the photographed image 800 related to the photographed image data 200. A representative color determining step of determining a color (an example of a first representative color) for each photographed image data 200, and a characteristic related to the color representing the plurality of photographed image data 200 based on the plurality of determined representative colors. A representative color calculation step for calculating a comprehensive representative color (an example of a second representative color) shown in FIG. a correction parameter calculation step of calculating a correction parameter to be used for color correction of the photographed image data 200 to be corrected; a correction step of performing color correction of the photographed image data 200 to be corrected using the calculated correction parameters; Execute. As a result, the image processing method according to the embodiment of the present invention can perform correction to make the brightness and color tone uniform for a plurality of photographed image data 200 in which different subjects are photographed.

Further, the image processing apparatus according to an embodiment of the present invention is the image processing apparatuses 10b, 10c, 40a, and 40b that perform brightness correction of photographed image data 200 (an example of image data). A representative pixel value indicating a characteristic related to a color representative of the cluster obtained by clustering processing of the pixel values of each pixel constituting the photographed image 800 related to the photographed image data 200 is calculated for each cluster, and based on the calculated representative pixel value. Then, a representative brightness value (an example of a first representative brightness value) indicating a characteristic related to brightness representative of the captured image data 200 is determined. Then, the image processing devices 10b, 10c, 40a, and 40b calculate brightness correction parameters for performing brightness correction of the captured image data 200 using the determined representative brightness values, and correct the calculated brightness. The brightness of the captured image data 200 is corrected using the parameters. Thereby, the image processing devices 10b, 10c, 40a, and 40b can perform brightness correction on the captured image data 200.

●Supplement●
Note that the functions of each embodiment can be realized by a computer-executable program written in a legacy programming language such as assembler, C, C++, C#, Java (registered trademark), or an object-oriented programming language. A program for performing the functions of can be distributed over telecommunications lines.

In addition, programs for executing the functions of each embodiment include ROM, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), flash memory, flexible disk, CD (Compact Disc). Distribute by storing in device-readable recording media such as ROM, CD-RW (Re-Writable), DVD-ROM, DVD-RAM, DVD-RW, Blu-ray disc, SD card, MO (Magneto-Optical disc), etc. You can also do it.

Furthermore, some or all of the functions of each embodiment can be implemented on a programmable device (PD) such as an FPGA (Field Programmable Gate Array), or as an ASIC, and each embodiment circuit configuration data (bitstream data) to be downloaded to the PD in order to implement the functions on the PD, HDL (Hardware Description Language) for generating circuit configuration data, VHDL (Very High Speed Integrated Circuits Hardware Description Language), It can be distributed on a recording medium as data written in Verilog-HDL or the like.

Although an image processing device, an image processing system, an image processing method, and a program according to an embodiment of the present invention have been described so far, the present invention is not limited to the embodiment described above, and can be applied to other embodiments. Additions, changes, deletions, and other changes can be made within the scope of those skilled in the art, and any aspect is included within the scope of the present invention as long as the functions and effects of the present invention are achieved.

1a, 1b, 1bb, 1bbb, 1c, 1cc, 1d, 1e Image processing system 5 Communication network 7 Photographing system 10a, 10b, 10bb, 10bbb, 10c, 10cc Image processing device 11 Network communication unit (an example of a first transmission means) )
12 Image processing unit (an example of image processing means)
13a, 13b, 13bb, 13bbb, 13c, 13cc Calculation unit 14 Correction unit (an example of correction means)
30 Photographing devices 40a, 40b Image processing device 43 Image processing unit (an example of image processing means)
44 Calculation unit 45 Correction unit (an example of correction means)
50 Communication terminal 51 Network communication unit (an example of second transmission means)
53 Reception Department (an example of reception means)
70 Web server 90 Display terminal 93 Display control unit (an example of display control means)
131a, 131b, 131c Representative pixel value calculation unit (an example of representative pixel value calculation means)
132a, 132c Representative color determining unit (an example of representative color determining means)
133a, 133c Comprehensive representative color calculation unit (an example of representative color calculation means)
134a, 134c RGB gain calculation unit (an example of correction parameter calculation means)
132b, 135c Brightness value calculation unit 133b, 136c Representative brightness value calculation unit (an example of representative brightness value determining means)
134b, 137c Brightness correction parameter calculation unit (an example of correction parameter calculation means)
135b, 138c Comprehensive representative brightness value calculation unit (an example of representative brightness value calculation means)
908 Display (example of display part)

Japanese Patent Application Publication No. 2010-268430

Claims (15)

An image processing system comprising: an image processing device that performs color correction of image data to be corrected based on a plurality of image data; and a photographing system that is communicatively connected via a communication network and includes a photographing device that photographs a subject. And,
The image processing device includes:
representative color determining means for determining, for each image data, a first representative color indicating a characteristic related to a color representative of the image data, based on the pixel value of each pixel constituting the image related to the image data;
representative color calculation means for calculating a second representative color indicating a characteristic related to a color representing a plurality of image data based on the plurality of determined first representative colors;
a correction parameter calculation means for calculating a correction parameter to be used for the color correction of the image data to be corrected, using the first representative color and the second representative color corresponding to the image data to be corrected; ,
a correction unit that performs the color correction of the image data to be corrected using the calculated correction parameter;
image processing means for reducing the image size of the image data;
a first transmitting means for transmitting a plurality of preview image data in which the color correction has been performed on the reduced image data to the photographing system;
The photographing system includes:
A receiving unit that receives a selection of specific preview image data from among the plurality of preview image data transmitted from the image processing device;
a second transmitting means for transmitting the accepted specific preview image data to the image processing device;
The correction means is an image processing system that performs the color correction using the correction parameter corresponding to the specific preview image data transmitted from the photographing system. The image processing system according to claim 1, further comprising:
comprising a representative pixel value calculation means for calculating, for each cluster, a representative pixel value indicating a characteristic related to a color representative of the cluster obtained by the clustering process of the pixel values;
In the image processing system, the representative color determining means determines, as the first representative color, the representative pixel value corresponding to a specific cluster having the largest number of pixels included in the cluster. The representative color calculation means calculates a representative pixel value regarding a color representing a specific cluster in which the first representative color included in the cluster is the largest among clusters obtained by clustering processing of the plurality of first representative colors. The image processing system according to claim 1 or 2, wherein: is calculated as the second representative color. The image processing system according to any one of claims 1 to 3, wherein the correction parameter calculation means calculates the correction parameter by dividing the second representative color by the first representative color. The image processing system according to any one of claims 1 to 4, wherein the correction parameter is an RGB gain value. The image processing system according to any one of claims 1 to 5, wherein the image processing device further comprises:
representative brightness value determining means for determining a first representative brightness value indicating a characteristic regarding brightness representative of the image data to be corrected, based on the pixel value of each pixel constituting the image related to the image data to be corrected; Prepare,
The correction parameter calculating means calculates a brightness correction parameter for performing brightness correction of the image data to be corrected, using the determined first representative brightness value,
The correction means is an image processing system that performs the color correction and the brightness correction of the image data to be corrected using the correction parameter and the brightness correction parameter. The image processing system according to claim 1, further comprising:
comprising a display terminal capable of communicating with the image processing device via the communication network,
The first transmitting means of the image processing device includes:
transmitting a plurality of corrected image data subjected to the color correction by the correction means to the display terminal;
The display terminal is
An image processing system comprising a display control means for displaying the plurality of corrected image data transmitted from the image processing device on a display unit. An image processing system comprising: an image processing device that performs color correction of image data to be corrected based on a plurality of image data; and a photographing system that is communicatively connected via a communication network and includes a photographing device that photographs a subject. An image processing method performed by
The image processing device includes:
a representative color determining step of determining, for each of the image data, a first representative color indicating a characteristic related to a color representative of the image data, based on the pixel value of each pixel constituting the image related to the image data;
a representative color calculating step of calculating a second representative color indicating a characteristic regarding a color representing a plurality of image data based on the plurality of determined first representative colors;
Correction used for the color correction of the image data to be corrected using the first representative color corresponding to the image data to be corrected and the second representative color calculated by the representative color calculation step. a correction parameter calculation step for calculating parameters;
a correction step of performing the color correction of the image data to be corrected using the calculated correction parameter;
an image processing step of reducing the image size of the image data;
a first transmission step of transmitting a plurality of preview image data in which the color correction has been performed on the reduced image data to the photographing system;
The photographing system includes:
a reception step of accepting a selection of specific preview image data from among the plurality of preview image data transmitted from the image processing device;
a second sending step of sending the accepted specific preview image data to the image processing device;
The correction step executed by the image processing device performs the color correction using the correction parameter corresponding to the specific preview image data transmitted from the imaging system.
Image processing method. An image processing system comprising: an image processing device that performs brightness correction of image data; and a photographing system that is communicably connected via a communication network and includes a photographing device that photographs a subject.
The image processing device includes:
representative pixel value calculation means for calculating, for each cluster, a representative pixel value indicating a characteristic related to a color representative of a cluster obtained by clustering processing of pixel values of each pixel constituting an image related to the image data;
representative brightness value determining means for determining a first representative brightness value indicating a characteristic regarding brightness representative of the image data based on the representative pixel value;
correction parameter calculation means for calculating a brightness correction parameter for performing the brightness correction of the image data using the determined first representative brightness value;
a correction unit that performs the brightness correction of the image data using the calculated brightness correction parameter;
image processing means for reducing the image size of the image data;
a first transmitting means for transmitting a plurality of preview image data in which the brightness correction has been performed on the reduced image data to the photographing system;
The photographing system includes:
A receiving unit that receives a selection of specific preview image data from among the plurality of preview image data transmitted from the image processing device;
a second transmitting means for transmitting the accepted specific preview image data to the image processing device;
The correction means performs the brightness correction using the brightness correction parameter corresponding to the specific preview image data transmitted from the photographing system.
Image processing system. The image processing system according to claim 9, wherein the correction parameter calculation means calculates the brightness correction parameter using an LUT (Lookup table) according to the determined first representative brightness value. The image processing system according to claim 9,
The representative brightness value determining means determines the first representative brightness value for each of the image data,
Furthermore,
comprising representative brightness value calculation means for calculating a second representative brightness value indicating a characteristic related to brightness representing a plurality of image data based on the plurality of determined first representative brightness values;
The correction parameter calculation means uses the first representative brightness value corresponding to the image data to be corrected and the second representative brightness value calculated by the representative brightness value calculation means to calculate the correction parameter of the image data to be corrected. Calculating the brightness correction parameter used for the brightness correction of image data,
The correction means is an image processing system that performs the brightness correction of the image data to be corrected using the calculated brightness correction parameter. The representative brightness value calculating means calculates the first representative brightness value by using a first representative brightness value that is within a predetermined brightness range among the plurality of first representative brightness values determined by the representative brightness value determining means. The image processing system according to claim 11 , wherein the image processing system calculates the representative brightness value of 2. The representative brightness value calculation means calculates a brightness value representing a specific cluster in which the first representative brightness values included in the cluster are the largest among clusters obtained by clustering processing of the plurality of first representative brightness values. The image processing system according to claim 11 , wherein: is calculated as the second representative brightness value. When the first representative brightness value corresponding to the image data to be corrected is within a predetermined brightness range, the correction parameter calculation means calculates the first representative brightness value and the second representative brightness value. calculate the brightness correction parameter using
When the first representative brightness value corresponding to the image data to be corrected is outside a predetermined brightness range, the correction parameter calculation means calculates the brightness correction parameter using the first representative brightness value. The image processing system according to claim 11 , wherein the image processing system calculates. The image processing system according to claim 9, wherein the image processing device further comprises:
representative color determining means for determining, for each image data, a first representative color indicating a characteristic related to a color representative of the image data, based on the pixel value of each pixel constituting the image related to the image data;
representative color calculation means for calculating a second representative color indicating a characteristic related to a color representing a plurality of image data based on the plurality of determined first representative colors;
The correction parameter calculating means calculates a correction parameter to be used for color correction of the image data to be corrected, using the first representative color and the second representative color corresponding to the image data to be corrected. ,
The correction means is an image processing system that uses the brightness correction parameter and the correction parameter to perform the brightness correction and the color correction of the image data to be corrected.