JP2023150525A - Image processing system and imaging device - Google Patents

Abstract

To provide an image processing system and an imaging device that can easily order prints of images from the imaging device.SOLUTION: In an image processing system according to one aspect of the present invention, an imaging device captures a first image for user identification and a second image for printing, analyzes the captured first image to acquire user identification information, and transmits the acquired user identification information and second image, and print instruction information about the second image to a server device. The server device stores the second image transmitted from the imaging device in association with the user identification information, and transfers the print instruction information to a printing device. The printing device prints the second image when receiving the print instruction information from the server device.SELECTED DRAWING: Figure 14

Description

The present invention relates to an image processing system and an imaging device that can order prints of images from the imaging device.

Regarding the technology for ordering prints of images from an imaging device, for example, Patent Document 1 discloses that inputs such as a user ID for identifying the user and a shop ID of the print request destination are prepared in advance using the photographing function of a digital camera. It is described that this is done by photographing the two-dimensional pattern information written on the card etc. Further, Patent Document 2 describes an image printing system that conducts a photo rally in which users are automatically photographed at a plurality of locations at a tourist spot, etc., and prints out the photographed images after the rally ends.

Japanese Patent Application Publication No. 2005-110125 Japanese Patent Application Publication No. 2018-074281

When printing an image captured with an imaging device such as a digital camera, it is common to transfer the captured image via a medium or via short-range wireless communication to a terminal device such as a smartphone or PC (personal computer) and then instruct it to print. images, making selecting and printing images a labor-intensive task. Therefore, depending on the configuration and functions of the imaging device, it has not been possible to easily print images. However, conventional techniques such as those disclosed in Patent Documents 1 and 2 described above do not sufficiently take such circumstances into consideration.

One embodiment of the technology of the present disclosure provides an image processing system and an imaging device that can easily order prints of images from the imaging device.

An image processing system according to a first aspect of the present invention is an image processing system including an imaging device, a server device, and a printing device, wherein the imaging device includes an imaging section that captures an image, and an imaging section that captures an image. an identification information acquisition unit that analyzes the captured first image to obtain user identification information; a transmission unit that transmits the user identification information and the second image captured by the imaging unit to the server device; a print instruction unit that transmits print instruction information to the server device; the server device includes an image storage unit that stores the second image transmitted from the transmission unit in association with user identification information; and a print instruction unit a print instruction transfer unit that transfers print instruction information transmitted from the server device to the printing device, and the printing device includes an image print unit that prints the second image when receiving the print instruction information transferred from the server device. Equipped with

In the image processing system according to the second aspect, in the first aspect, the imaging device includes an initial authentication information storage unit that stores the user identification information and the initial authentication information in association with each other, and acquires the first authentication information of the user. The apparatus further includes a first authentication information acquisition section and an authentication section that compares the first authentication information and initial authentication information to authenticate the user. In the second aspect, it is preferable that the transmitter of the imaging device transmits the user identification information and the second image to the server device after the user is authenticated.

In the image processing system according to the third aspect, in the second aspect, the initial authentication information storage section acquires the initial authentication information by analyzing the third image captured by the imaging section, and the first authentication information acquisition section: The fourth image captured by the imaging unit is analyzed to obtain first authentication information.

In the image processing system according to a fourth aspect, in any one of the first to third aspects, the imaging device includes an operation history information acquisition unit that acquires operation history information regarding image processing of the second image; The server device further includes an operation history information transmitting unit that associates the information and the second image and transmits the information to the server device, and the server device includes an operation history information storage unit that associates and stores the operation history information and the second image. , and an operation history information transmitter that transmits operation history information to the printing device.

In the image processing system according to the fifth aspect, in the fourth aspect, the operation history information acquisition unit of the imaging device starts from a first time before the start of imaging of the second image, and from a second time before the end of imaging of the first image. Obtain operation history information about operations performed up to a certain time.

In the image processing system according to the sixth aspect, in the fourth or fifth aspect, the operation history information acquisition unit of the imaging device selects one or more of the density, contrast, saturation, white balance, and edge enhancement of the second image. Obtain information about operations as operation history information.

In the image processing system according to the seventh aspect, in any one of the fourth to sixth aspects, the operation history information acquisition unit of the imaging device analyzes the second image to determine the main subject in the second image. information, information on the shadow area in the second image, or information on the highlight area in the second image, and transmits the acquired information on the main subject, information on the shadow area, or information on the highlight area to the user of the imaging device. information about the user's operation of the imaging device in response to the presentation is acquired as operation history information. A "shadow area" is an area with black crushing or an area close to black crushing, and is an area that is overall dark and has low contrast between light and dark. A “highlight portion” is an area with highlight clipping or an area close to highlight clipping, and is an area that is bright overall and has low contrast between brightness and darkness.

In the image processing system according to an eighth aspect, in any one of the fourth to seventh aspects, the printing device includes an operation history information receiving unit that receives operation history information from the server device, and an operation history information receiving unit that receives operation history information from the server device; a print information transmission section that transmits print information, the image print section prints the second image based on the received operation history information, and the print information transmission section prints the second image based on the print instruction information. When printing, print information is transmitted that includes information regarding at least one of the corrections made to the second image, the second image after the corrections, and settings related to printing the second image.

In the image processing system according to the ninth aspect, in the eighth aspect, the print information transmitting unit transmits the print information to the server device, and the server device stores the print information and the second image in association with each other. The image forming apparatus further includes a print information storage section.

In the image processing system according to a tenth aspect, in the ninth aspect, the server device further includes a print information transmitting section that transmits print information to the imaging device.

An imaging device according to an eleventh aspect of the present invention includes: an imaging unit that captures an image; an information acquisition unit that analyzes a first image captured by the imaging unit to obtain user identification information; and a transmitting unit configured to transmit the second image captured by the unit to a server device of the image processing system.

The imaging device according to the twelfth aspect in the eleventh aspect further includes a print instruction section that instructs the printing device of the image processing system to print the second image via the server device.

An imaging device according to a thirteenth aspect in the eleventh or twelfth aspect includes an initial authentication information acquisition unit that stores user identification information and initial authentication information of the imaging device in association with each other, and an initial authentication information acquisition unit that acquires the first authentication information. The apparatus further includes a first authentication information acquisition section, and an authentication section that compares the first authentication information and the initial authentication information to authenticate the user.

In any one of the eleventh to thirteenth aspects, an imaging device according to a fourteenth aspect includes an operation history information acquisition unit that acquires operation history information regarding image processing of the second image, and an operation history information acquisition unit that acquires operation history information regarding image processing of the second image. and an operation history information transmitter that associates the information with the information and transmits the information to the server device.

FIG. 1 is a diagram showing the overall configuration of an image processing system according to the first embodiment. FIG. 2 is an external perspective view of the imaging device. FIG. 3 is a rear view of the imaging device. FIG. 4 is a block diagram showing a schematic configuration of the imaging device. FIG. 5 is a diagram showing the functional configuration of the processor of the imaging device. FIG. 6 is a diagram showing a schematic configuration of the server device. FIG. 7 is a diagram showing a schematic configuration of the printing device. FIG. 8 is a flowchart showing how user identification information is acquired through image analysis. FIG. 9 is a flowchart showing how a user is authenticated by image analysis. FIG. 10 is a flowchart showing processing in the printing device. FIG. 11 is a diagram showing the functional configuration of the processor of the imaging device in the second embodiment. FIG. 12 is a diagram showing a schematic configuration of a server device in the second embodiment. FIG. 13 is a diagram showing a schematic configuration of a printing device in the second embodiment. FIG. 14 is a diagram showing an outline of processing in the second embodiment. FIG. 15 is a flowchart showing processing in the printing device. FIG. 16 is a diagram illustrating how the imaging device issues adjustment instructions for each area. FIG. 17 is a flowchart regarding adjustment processing for each area. FIG. 18 is another diagram showing how the imaging device issues adjustment instructions for each area. FIG. 19 is a flowchart showing main subject identification and correction processing. FIG. 20 is a diagram showing how the finish is evaluated.

One embodiment of the image processing system and imaging device according to the present invention is as follows. In the description, reference is made to the accompanying drawings as necessary.

[First embodiment]
[Image processing system configuration]
FIG. 1 is a diagram showing the configuration of an image processing system 10 (image processing system) according to the first embodiment. The image processing system 10 includes an imaging device 100 (imaging device), a server device 200 (server device), and a printing device 300 (printing device), which are connected via a network NW and communicate with each other. is possible. In the image processing system 10 shown in FIG. 1, it is possible to order a print of an image from the imaging device 100, as described below.

Note that in the image processing system 10, the number of imaging devices connected to the server device 200 and the printing device 300 may be one or more. Furthermore, the number of server devices 200 and printing devices 300 may be one or more. When the image processing system 10 includes a plurality of printing devices 300, the server device 200 is configured to print to a specific printing device 300 (for example, a printing device located at a business office nearest to the user's address or residence, or a printing device located at a location specified by the user). device) and print with that printing device 300.

[Configuration of imaging device]
[overall structure]
FIG. 2 is an external perspective view of the imaging device 100. As shown in the figure, the imaging device 100 includes a camera body 110 and a lens device 103 (having an optical axis L1) that is detachable from the camera body 110. , dial 106 are provided. The shutter button 104 is a two-stage stroke button that can be pressed in two steps; when pressed halfway through the stroke, focus and exposure settings are locked, and when pressed to the end of the stroke, photography is performed. The dial 106 is a dial for setting shutter speed and exposure. Note that the lens device 103 is a component of the imaging section 102 described later, and the shutter button 104 and dial 106 are components of the operation section 150.

FIG. 3 is a rear view of the imaging device 100. As shown in the figure, the back of the imaging device 100 is provided with a finder eyepiece 105, a cross button 107, a MENU/OK button 108, and a display 140 (display device). The user can visually check images including captured images and shooting information through the finder eyepiece 105, and use the cross button 107 and MENU/OK button 108 to set shooting conditions, select images, print instructions, etc. operations can be performed. Note that the imaging device 100 includes a finder section (not shown) including a finder eyepiece section 105. The finder section may be electronic, optical, or a hybrid of electronic and optical.

The display 140 can display various information such as preview images and photographed images. Further, the display 140 is a so-called vari-angle display whose direction can be changed by a mechanism (not shown), and the user of the imaging device 100 can look at the display 140 while looking at the display 140. (take a so-called "selfie").

The cross button 107 and the MENU/OK button 108 are components of an operation unit 150, which will be described later. Further, the display 140 may be configured with a touch panel type display and may be used as the operation unit 150. In this case, the display 140 preferably accepts operations for specifying movement, rotation, enlargement, reduction, etc. of the displayed image, operations for specifying the main subject, shadow areas, highlights, etc., operations for specifying the focus area, etc. .

[Functional configuration]
FIG. 4 is a diagram showing the functional configuration of the imaging device 100. As shown in the figure, the imaging device 100 includes an imaging unit 102 (imaging unit), a processor 130 (processor), a display 140, an operation unit 150, a flash ROM 160 (ROM: Read Only Memory), and a RAM 170 (RAM). :Random Access Memory) and a memory card 180, and can capture still images and moving images of a subject in response to user instructions via the operation unit 150.

The imaging unit 102 includes an image sensor, an analog signal processing circuit, a digital signal processing circuit, etc. (not shown) in addition to the lens device 103 shown in FIG. 2, and outputs image data of a subject. Note that various types of image pickup devices such as a CMOS (complementary metal-oxide semiconductor) type and a CCD (charge-coupled device) type can be used as the image pickup device. The processor 130 is a processor that centrally controls processing (shooting control, image processing, print instructions, etc.) in the imaging apparatus 100, and its detailed functions will be described later. The memory card 180 is a non-temporary and tangible recording medium that can be attached to and removed from the imaging device 100, and records information such as captured images.

[Processor functional configuration]
FIG. 5 is a diagram showing the functional configuration of the processor 130. As shown in the figure, the processor 130 includes an image processing section 130A, an identification information acquisition section 130B, an information transmission/reception section 130C, a print instruction section 130D, an initial authentication information storage section 130E, and a first authentication information acquisition section. 130F, an authentication section 130G, and a display control section 130H. Details of processing such as print instructions using these functions will be described later. Note that the processor 130 may include a memory (non-temporary and tangible recording medium) for storing initial authentication information, first authentication information, user identification information, and the like.

The processor 130 includes various processors and electric circuits, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), and a PLD (Programmable Logic Device). When these processors and electrical circuits execute software (programs), they flash code that can be read by the computer (for example, the various processors and electrical circuits that make up the processor, and/or a combination thereof). The software is stored in a non-temporary and tangible recording medium such as the ROM 160, and the computer refers to the software. Software stored in a non-temporary and tangible recording medium is a program for executing imaging, user authentication, various image processing, print instructions, etc. (imaging program, authentication program, image processing program, print instruction program) , and data used during execution. Instead of the flash ROM 160, the code may be recorded in a non-temporary and tangible recording medium such as a ROM or an EEPROM (Electronically Erasable and Programmable Read Only Memory). Note that this "non-temporary and tangible recording medium" does not include a non-tangible recording medium such as a carrier wave signal or a propagation signal itself. During processing using software, the RAM 170 is used as a temporary storage area or work area, and it is also possible to refer to data stored in a memory (not shown), for example.

[Server device configuration]
FIG. 6 is a diagram showing the configuration of the server device 200. As shown in the figure, the server device 200 includes a processor 210, a ROM 230, a RAM 240, and a database 250.

The processor 210 includes a user information management section 211, an image storage section 212 (image storage section), a print instruction transfer section 214 (print instruction transfer section), a print information transmission section 220 (print information transmission section), and a print information transfer section 212 (image storage section). A storage section 222 (print information storage section). Details of processing using these units will be described later. Note that, like the processor 130 described above, the processor 210 can be configured of various processors and electric circuits such as a CPU, GPU, FPGA, and PLD. When these processors and electric circuits execute software (programs), they refer to the programs (programs that save images, print instructions, send print information, etc.) and data stored in the ROM 230, and temporarily store them. RAM 240 is used as an area or work area. The processor 210 may perform processing by referring to a memory (non-temporary and tangible recording medium) other than the ROM 230 and RAM 240 (not shown).

The database 250 (image storage unit) is composed of a recording medium such as a hard disk, a magneto-optical recording medium, a semiconductor memory, and a control device thereof, and stores the captured image (second image), user identification information, and print information received from the imaging device 100. etc. are linked and saved.

Note that the server device 200 may further include a display (display device; not shown) that displays the operating status, contents of the database 250, etc., an operation unit (not shown) that receives user operations, and the like. Moreover, the "server" in this embodiment includes a device that exists on the cloud and has a function similar to that of a server.

[Print device configuration]
FIG. 7 is a diagram showing the configuration of the printing device 300 (printing device). As shown in the figure, the printing device 300 includes a processor 310, a ROM 330, a RAM 340, a database 350, a printing mechanism 360 (image printing section), a display 370 (display device), and an operation section 380. Be prepared. The printing device 300 can be installed in a print laboratory, and automatically performs processing such as printing of photographed images according to the operation of the user or the print laboratory staff, or without any operation of the user or the print laboratory staff. .

The processor 310 includes an information receiving section 312 (print instruction receiving section), a print control section 314 (image printing section), and a print information transmitting section 320 (print information transmitting section). Details of processing using these units will be described later. Note that, like the processors 130 and 210 described above, the processor 310 can be configured of various processors and electric circuits such as a CPU, GPU, FPGA, and PLD. When these processors and electric circuits execute software (programs), they refer to the programs (programs that print images, send print information, etc.) and data stored in the ROM 330, and use temporary storage areas or work RAM 340 is used as the area. The processor 310 may perform processing by referring to a memory (non-temporary and tangible recording medium) other than the ROM 330 and the RAM 340 (not shown).

The database 350 is made up of recording media such as hard disks, magneto-optical recording media, and semiconductor memories, and their control devices, and includes photographed images (second images) received from the server device 200, print instruction information, and corrections (image processing) for the photographed images. ) contents etc. are saved.

The print mechanism 360 includes a transport mechanism (not shown) for print paper (recording medium), an exposure head (not shown), and the like, and constitutes an image print section together with the print control section 314 described above. While looking at the printing image (second image) etc. displayed on the display 370, the print lab staff enters detailed print conditions (for example, image density, contrast, saturation, white balance, outline, etc.) via the operation unit 380. The image processing system 10 according to the first embodiment can thereby obtain prints that match the user's preferences.

[Processing in image processing system]
Processing in the image processing system 10 having the above-described configuration will be described below.

[Obtaining user identification information through image analysis]
FIG. 8 is a flowchart showing how user identification information is acquired through image analysis. Prior to obtaining user identification information, the user must first provide user registration information such as name, nickname, address, telephone number, email address, various SNS (social networking service) or social media accounts, and billing method. is transmitted to the server device 200. This transmission may be performed via a smartphone, tablet terminal, personal computer, etc. connected to the server device 200 via a network.

The user information management unit 211 of the server device 200 receives the information, creates an account for the user, and further generates an image for acquiring identification information (first image) and provides it to the user. The identification information acquisition image may include information such as the URL of the access destination and the user ID. Furthermore, the identification information acquisition image may be, for example, a two-dimensional pattern image such as a two-dimensional barcode, but is not limited to this example. The user information management unit 211 may send the identification information acquisition image to the smartphone, tablet terminal, personal computer, etc. that sent the above-mentioned user registration information, or may send it to the user in a printed state. . The user displays the identification information acquisition image on a display of a smartphone or the like or opens the printed matter to make the identification information acquisition image visible.

The imaging unit 102 of the imaging device 100 captures this identification information acquisition image (first image) in response to the user's operation via the operation unit 150 (step S100 in FIG. 8: identification information acquisition step). The identification information acquisition unit 130B (identification information acquisition unit) analyzes the photographed identification information acquisition image and acquires user identification information (step S102: identification information acquisition step).

Note that the information transmitting/receiving unit 130C (transmitting unit) transmits the acquired user identification information and the photographed image (image for printing; second image) to the server device 200 (step S126 in FIG. 9), as described later. : transmission process).

[User authentication]
FIG. 9 is a flowchart showing the process of authenticating the user of the imaging device 100. The imaging unit 102 captures an image of a subject for initial authentication in response to a user's operation to obtain an initial authentication image (third image), and the initial authentication information storage unit 130E (initial authentication information acquisition unit) (Step S112: Initial authentication information acquisition step) and acquires initial authentication information (Step S114: Initial authentication information acquisition step). The user of the imaging device 100 can select an object unique to the user, such as his or her own face, iris, fingerprint, vein pattern, etc., as the object for initial authentication. When capturing an image of one's own face or the like, the user may have another person image the user. Furthermore, a sensor for iris imaging may be provided inside the finder eyepiece section 105. The initial authentication information may be the subject image itself. However, in the unlikely event that the initial authentication information is stolen, the initial authentication information storage unit 130E analyzes the subject image while retaining the information necessary for authentication, converts it into feature quantities, and initializes the image. Preferably, authentication information is obtained. The initial authentication information storage unit 130E stores the above-described user identification information and initial authentication information in association with each other (step S116: information storage step). The storage destination may be a memory (not shown) included in the initial authentication information storage section 130E itself, or may be the flash ROM 160.

For example, when the imaging device 100 is started for the first time on that day, the imaging unit 102 images a subject for first authentication in response to a user's operation and obtains a first authentication image (fourth image). The first authentication information acquisition unit 130F (first authentication information acquisition unit) analyzes the first authentication image (step S120: first authentication information acquisition step) and acquires the first authentication information (step S122: first authentication information acquisition process). The user of the imaging device 100 can select an object such as his or her face, iris, fingerprint, vein pattern, etc. as the object for first authentication, but the object for first authentication may be the same as the object for initial authentication. shall be. The authentication unit 130G (authentication unit) compares the first authentication information and the initial authentication information to authenticate the user (step S124: authentication step). The authentication unit 130G may authenticate the user by extracting the feature amounts of the initial authentication image and the first authentication image and comparing them, or may be constructed using a machine learning algorithm such as deep learning. User authentication may be performed using a comparator (trained model) that has been trained. The comparator may be constructed by a neural network such as a convolutional neural network (CNN).

According to the above-described method, even a digital camera that does not have an authentication function using a SIM card (Subscriber Identity Module Card) in a smartphone can perform user authentication. Note that it is preferable that the above-mentioned initial authentication information (third image) and first authentication information (fourth image) can be deleted according to a predetermined procedure from the viewpoint of preventing leakage of personal information. It is also preferable that the server device 200 be able to erase user identification information and suspend or delete user accounts.

When the user is authenticated, the information transmitting/receiving unit 130C (transmitting unit) transmits user identification information and a print image (second image) to the server device 200 in accordance with the user's operation (step S126: print instruction step). , the print instruction unit 130D (print instruction unit) transmits print instruction information regarding the print image to the server device 200 in response to the user's print instruction (step S126: print instruction step). The user identification information, the print image, and the print instruction information may or may not be transmitted simultaneously. Further, the image for printing to be transmitted is not limited to a still image, but may be a moving image. When transmitting a moving image, the print instruction information described later may include instructions regarding extracting frames from the moving image (for example, extracting and printing frames at fixed time intervals) and instructions regarding the main subject to be printed ( For example, it may include printing a frame in which the main subject is clearly visible.

Note that instead of sending a print image each time you print, the print image is sent and stored in advance to the server device 200 without print instruction information, and a print instruction is sent to the image selected from the stored images. You may also do this.

[Print instruction information]
The above-mentioned print instruction information may include information such as the image to be printed, the number of sheets, whether or not image processing is to be performed, and the contents, in addition to the information itself indicating the execution of printing. It may also include instructions to process or combine multiple images and print them. The print instruction information can be recorded, for example, in a DPOF (Digital Print Order Format) format or an extended format thereof, but other formats may also be used. Furthermore, the print instruction information may be recorded as part of the image data, or may be recorded as data independent of the image data.

[Processing on server device]
The image storage unit 212 (image storage unit) of the server device 200 stores the print image (second image) sent from the information transmission/reception unit 130C (transmission unit) of the imaging device 100 in association with user identification information. (image storage process). The storage destination may be a memory (not shown) included in the image storage section 212 or the database 250. The print instruction transfer unit 214 (print instruction transfer unit) transfers the print instruction information and print image transmitted from the print instruction unit 130D (print instruction unit) to the printing device 300 (information transfer step). The print instruction information and the print image do not have to be transferred at the same time.

Note that the user information management unit 211 (server device 200) may perform billing processing based on the above-described processing, and provide service usage points (usage benefits) and decoration images (user information management step).

[Processing by printing device]
FIG. 10 is a diagram showing processing in the printing device 300. The information receiving unit 312 receives the print instruction and the print image (second image) transmitted by the server device 200 (step S300: information receiving step). As described above, these print instructions and the print image need not be received at the same time. When the print control unit 314 (image print unit) and the print mechanism 360 (image print unit) receive the print instruction information transferred from the server device 200, the print control unit 314 (image print unit) and print mechanism 360 (image print unit) print an image for printing (second image) based on the print instruction information. (Step S302: image printing process).

The print control unit 314 and the print mechanism 360 may automatically print based on print instruction information. Note that when printing is performed automatically in this way, the user may not necessarily be satisfied with the finished product. While the color, brightness, and contrast of a print (an image printed on a medium such as paper; the same applies hereinafter) change depending on the setting values of the printing device 300, the user, for example, This is because the user expects the print image tone, brightness, contrast, etc. displayed on the display 140. Therefore, in the image processing system 10 according to the first embodiment, print lab staff or the like may manually correct or adjust the image for printing, and then print the image. Such manual correction and printing can be performed in response to the user's operation based on the information included in the print instruction information, thereby making it possible to provide a print that meets the user's preferences. Note that whether or not to perform such manual correction depends on the user's instructions included in the print instruction information, the price setting, the user's rank (for example, membership registration, membership fee amount, number of times and frequency of use of the print service, etc.) ), etc.

[Transmission of print information]
The print information transmitting unit 320 (print information transmitting unit) can transmit print information regarding printing after printing of the print image (second image) is completed (step S304: print information transmitting step). The print information transmission section 320 may display the print information on the display 370. The print information may include information regarding at least one of corrections made to the print image, print images after the correction, and settings related to printing of the print images. Further, the print information may include information indicating that printing has been completed. By including such information, the user of the printing device 300 and the user of the imaging device 100 can understand that printing has been completed and the contents of corrections and settings for the image to be printed. The print information sending unit 320 may send the print information to the imaging device 100 via the server device 200, or may print the print information on a medium such as plain paper and send it to the user along with the print.

[Effects of the first embodiment]
According to the first embodiment described above, it is possible to easily order a print using the imaging device 100 (imaging device) without going through a smartphone, personal computer, or the like. Furthermore, in the imaging device 100, the user can be easily authenticated by image analysis.

[Second embodiment]
A second embodiment of an image processing system and an imaging device according to the present invention will be described. The second embodiment differs from the above-described first embodiment in that the imaging device 100 acquires operation history information regarding a photographed image (print image, second image), and prints images based on this operation history information. They differ in that they perform the following. In the second embodiment, this operation history information can be used to provide prints that match the user's preferences.

[Image processing system configuration]
The image processing system and imaging device in the second embodiment will be described below, focusing on the differences from the first embodiment. Similar configurations will be given the same reference numerals and detailed descriptions will be omitted. The overall configuration of the image processing system in the second embodiment is the same as that in the first embodiment (see FIG. 1), so it will be described as an image processing system 10. Further, the imaging device will be described as the imaging device 100, the server device as the server device 202, and the printing device as the printing device 302.

[Processor of imaging device]
FIG. 11 is a diagram showing the configuration of the processor 132 (processor) of the imaging device 100 in the second embodiment. Compared to the processor 130 in the first embodiment (see FIG. 5), the processor 132 includes an operation history information acquisition unit 130I that acquires operation history information regarding image processing of images for printing (photographed images, second images). It differs in that it includes an operation history information acquisition unit. The operation history information acquisition unit 130I can acquire, as operation history information, information on operations related to one or more of the density, contrast, saturation, white balance, and outline enhancement of the print image.

The operation history information acquisition unit 130I may acquire operation history information about still images, or may acquire operation history information about moving images. The operation history information acquisition unit 130I also stores information regarding operations (for example, shooting mode settings, exposure conditions settings, white balance settings, etc.) that are performed in advance before shooting images (print images, second images). Operation history information may be acquired, or operation history information regarding operations performed after photographing may be acquired. The acquisition period of such operation history information is, for example, "from the first time before the start of capturing the captured image (second image) to the second time after the end of capturing the captured image (second image)". It can be stipulated.

A specific example of acquiring operation history information will be described later (see FIGS. 15 to 19, etc.).

The other configuration of the imaging device 100 is the same as the first embodiment (see FIG. 5). In the second embodiment, the information transmitting/receiving unit 130C (operation history information transmitting unit) transmits operation history information in addition to user identification information and the like to the server device 200 (step S126 in FIGS. 8 and 9).

[Processor of server device]
FIG. 12 is a diagram showing the configuration of the server device 202 in the second embodiment. In the server device 202 in the second embodiment, compared to the server device 200 in the first embodiment (see FIG. 6), the processor 210A has an operation history information storage unit 216 (operation history information storage unit) and an operation history information storage unit 216 (operation history information storage unit). The difference is that it includes an information transmitter 218 (operation history information transmitter). The operation history information storage unit 216 stores the operation history information and the print image (second image) in association with each other. The storage destination may be a memory (not shown) included in the operation history information storage section 216 or the database 250. Further, the operation history information transmitting unit 218 transmits operation history information to the printing device 300.

[Printing device processor]
FIG. 13 is a diagram showing the configuration of a printing device 300 in the second embodiment. In the embodiment shown in the figure, compared to the printing apparatus 300 in the first embodiment (see FIG. 7), the processor 310A has an operation history information receiving section 316 (operation history information receiving section) and an operation history information receiving section 316 (operation history information receiving section). It is different in that it includes a history information storage section 318 (operation history information storage section). The operation history information receiving unit 316 receives operation history information from the server device 200, and the operation history information storage unit 318 stores the received operation history information in association with the print image (second image). The storage location may be a memory (not shown) included in the operation history information storage section 318 or the database 350.

The print control unit 314 (image printing unit) and the printing mechanism 360 (image printing unit) print a print image (second image) based on the received operation history information.

[Processing in the second embodiment]
FIG. 14 is a schematic diagram showing an outline of processing in the second embodiment. As shown in the figure, in the second embodiment, acquisition and transmission/reception of operation history information, accumulation and analysis of print evaluation information, etc. are performed (described later). Further, the print information transmitting unit 320 transmits print information in the same manner as in the first embodiment, but in the second embodiment, the print information includes information indicating that corrections and adjustments have been made based on operation history information, and the contents thereof. It may be included.

[Processing in the imaging device]
[Processing related to operation history information]
FIG. 15 is a diagram illustrating an example of acquisition of operation history information in the imaging device 100. In the example shown in the figure, the user's preferences regarding print finish are accepted. The user visually confirms the information displayed on the display 140 and operates the cross button 107 and MENU/OK button 108 to set white balance correction, contrast correction, saturation correction, sharpness correction (outline emphasis), etc. It can be carried out. The settings may include, for example, selecting "standard", "none", "strong", "weak", etc. Further, density correction may be performed in addition to the items shown in FIG. 15. In response to such user operations, the operation history information acquisition unit 130I acquires operation history information about one or more of the density, contrast, saturation, white balance, and edge enhancement of the print image (second image). It can be acquired as information (operation history information acquisition step). The operation history information acquisition unit 130I may acquire information on operations related to moving, rotating, enlarging, or reducing an image as operation history information. Note that the user may make these settings before taking a picture, or may make these settings when ordering a print after taking a picture. Further, the user may perform these settings as the basic settings of the imaging device 100, for a group of multiple images to be printed, or for each image to be printed. .

[Adjustment instructions for each area]
FIGS. 16 and 17 are diagrams illustrating how adjustment instructions are given for each area of a photographed image (print image, second image). In the examples shown in these figures, the image processing unit 130A and the operation history information acquisition unit 130I acquire information about the main subject in the captured image by analyzing the captured image or by acquiring the enlargement operation on the display 140 during shooting or playback. , or obtain information about shadow areas in a captured image. The operation history information acquisition unit 130I presents the acquired information on the main subject or information on the shadow area to the user of the imaging device 100. Specifically, in the example shown in part (a) of FIG. 16, the image processing unit 130A and the operation history information acquisition unit 130I add a frame F1 around the mountain that is estimated to be the main subject, and "Are you here?" is added and displayed on the display 140 by the display control unit 130H. In addition, in the example shown in part (b) of FIG. 16, the image processing unit 130A and the operation history information acquisition unit 130I add a frame F2 around the foot portion, which is the shadow portion, in the photographed image, and Would you like to lower the contrast?'' is added and displayed on the display 140 by the display control unit 130H.

The image processing unit 130A can perform the display shown in part (b) of FIG. 16 based on the area of the shadow portion (or it may be the ratio of the entire photographed image; the same applies hereinafter). Specifically, as shown in FIG. 17, the image processing unit 130A determines whether the area of the shadow portion exceeds a predetermined threshold (step S400), and if it exceeds the area (b) in FIG. As shown in (step S402), a frame F2 is displayed to confirm with the user whether or not correction (image processing) is necessary. The image processing unit 130A does not perform any special processing on the shadow portion if the area of the shadow portion does not exceed the threshold (step S404) or if the user performs an operation indicating that no correction is required even if the area exceeds the threshold. .

Note that as the correction (image processing) for lowering the contrast of the shadow portion, for example, a “dodging” method can be used. "Dodging" refers to the process of covering a specific area of the photographic paper in a silver halide photographic print, reducing the exposure to that area, brightening that area of the print, and increasing the exposure to other areas. This process darkens that area of the print. If you want to lower the contrast in shadow areas, you can dodge the main subject area. Note that a method called "burn-in" is also known, and this refers to processing that darkens a specific area by increasing the amount of exposure to that area. In order to lower the contrast of the shadow area by "burning", it is sufficient to burn the shadow area. In the printing apparatus 300, these exposure amount adjustments can be performed digitally. Note that in the imaging device 100, digital image processing can be realized by the image processing unit 130A.

Note that similar operations can be performed for correction of highlight portions.

The image processing unit 130A and the operation history information acquisition unit 130I acquire information on the main subject in the captured image or highlight portions in the captured image by analyzing the captured image or by acquiring an enlargement operation on the display 140 during shooting or playback. Get information about. The operation history information acquisition unit 130I presents the acquired main subject information or highlighted portion information to the user of the imaging device 100. Specifically, in the example shown in part (a) of FIG. 18, the image processing unit 130A and the operation history information acquisition unit 130I add a frame F3 around the sea and sky that are estimated to be the main subjects, and Is this the main subject?'' is added and displayed on the display 140 by the display control unit 130H. In the example shown in part (b) of FIG. 18, the image processing unit 130A and the operation history information acquisition unit 130I add a frame F4 around the sandy beach that is the highlight part in the captured image, and Would you like to lower the contrast?'' is added and displayed on the display 140 by the display control unit 130H.

Note that the image processing unit 130A performs different judgments and processes depending on the area of the highlighted portion, as in FIGS. 16 and 17.

In addition, as for correction (image processing) to lower the contrast of the highlight area, it is sufficient to perform either dodge on the highlight area or burn-in on the main subject area. This is similar to FIG. 17.

The user of the imaging device 100 can visually confirm such a display and use the cross button 107 and the MENU/OK button 108 to instruct whether or not correction is necessary, and the operation history information acquisition unit 130I converts this instruction into operation history information. can be obtained as.

In the embodiment described above, the contrast of the main subject is increased, and the contrast of the shadow or highlight portion is decreased.

However, exposure conditions may be set for the main subject so that the contrast of the main subject is high, and the exposure conditions may be applied to the entire image. For example, if the main subject is a bride wearing white at a wedding, the exposure conditions may be set to increase the contrast of the bride in the entire image. Furthermore, if the main subject is a dark object such as a bronze statue, the exposure conditions may be set such that the contrast of the bronze statue is high in the entire image.

In image processing, there are times when it is better to adjust the brightness of the entire image, and there are times when it is better to deliberately lower the contrast in the shadows or highlights to bring out the main subject. It is preferable to consider the judgment of the user of the imaging device 100 regarding how to utilize the range for printing. Therefore, by presenting information on the main subject or information on shadow areas and highlight areas to the user and accepting the user's judgment (operation) as in the examples of FIGS. 16 and 18, prints that match the user's tastes can be created. Obtainable. Note that both the main subject information and the shadow part information may be presented to the user.

[Identification and correction of main subject]
FIG. 19 is a flowchart showing main subject identification and correction processing. The image processing unit 130A analyzes the photographed image to obtain information on the main subject as described above, and determines whether position information on the main subject exists in the photographed image (step S410). The image processing unit 130A can determine whether the subject is the main subject based on whether the subject is panned or tilted or the area in which the subject is present is enlarged during photographing. Further, when the photographed image is being played back on the display 140 after photographing, the subject appearing in the enlarged area of the photographed image can be determined to be the main subject by an enlargement operation. In addition, in the case of a moving image, the image processing unit 130A selects one of the subjects shown in the preview image or live view image (the moving image displayed on the display 140 before starting recording to the memory card 180 or the like). When AF (auto focus) areas and AE (auto exposure) areas are set for more than one, the subject for which these AF areas and AE areas are set can be recognized as the main subject. Note that the number of main subjects is not particularly limited, and may be one or more.

The image processing unit 130A and the display control unit 130H display the position information and size information of the main subject, or information for performing correction, on an identification display as shown in part (a) of FIG. 16 described above, or on a display. When the user of the imaging device 100 performs a correction approval operation or selection operation in response to the presentation, the operation history information acquisition unit 130I displays information regarding the approval operation or selection operation. information as operation history information.

In addition, with devices equipped with touch-panel displays such as smartphones and tablet devices, you can specify the main subject, adjust focus or exposure, and give shooting instructions by tapping the subject shown on the display when shooting. I can do it. Similar operations can be performed via the display 140 in the imaging device 100, but when shooting using the viewfinder eyepiece 105 (finder) and shutter button 104, the user cannot perform such operations when shooting. Have difficulty. However, in the image processing system 10 including the imaging device 100 according to the present embodiment, the user's preferences regarding the main subject, shadow areas, etc. can be reflected in the print even after shooting by performing correction based on operation history information.

Note that the content of corrections and corrections to be presented is not limited to brightness and contrast, and the image processing unit 130A performs similar processing on one or more of brightness, contrast, saturation, density, white balance, and edge enhancement. It's fine. In addition, the image processing unit 130A presents the user with a correction for rotating the captured image when the main subject is photographed at an angle greater than or equal to a predetermined angle, and acquires information on the operation according to the presentation as operation history information. You may.

[Processing in the server device]
In the server device 200, the operation history information storage unit 216 receives the operation history information from the imaging device 100, stores it in association with the print image (in the memory or database 250, not shown), and transmits the operation history information to the operation history information transmission unit 218. sends operation history information to the printing device 300. The server device 200 may perform some or all of the correction based on the operation history information and transmit the corrected image to the printing device 300.

If there is no positional information of the main subject in the photographed image, information for normal correction (correction that does not take into account the positional information of the main subject) is calculated for the photographed image (step S412). If there is position information, the image processing unit 130A determines whether or not there is size information of the main subject (step S414), and if there is size information, respects the brightness (or contrast) in the area where the main subject exists. Information for executing the given (prioritized) correction is calculated (step S416). "Correction that respects (prioritizes) the brightness (or contrast) in the area where the main subject exists" refers to correction that appropriately maintains the brightness (or contrast) in the area where the main subject exists (correction in other areas) means to the extent possible). On the other hand, if there is no size information, the image processing unit 130A divides the photographed image into multiple regions (for example, four quadrants) and creates an area in which the brightness (or contrast) is weighted and averaged based on the position information of the main subject. information for executing correction that respects the brightness (or contrast) in the area where the main subject is present (increasing the weight of the weighted average) (step S416).

[Processing in printing device]
In the printing apparatus 300, the print control unit 314 performs automatic correction based on the received information (step S310 in FIG. 14). Alternatively, instead of this automatic correction, primary correction (automatic correction) may be performed (step S320), and the user (print lab staff, etc.) may check the result and perform an operation to adjust the correction content ( S330). The reason why such adjustments are made is that while the color, brightness, and contrast of printed matter change depending on the setting values of the printing device 300, the user adjusts the tone and brightness of the print image displayed on the display 140 of the imaging device 100, for example. This is because we expect contrast, etc. Adjustment of such correction results can be performed in accordance with the user's instructions included in the print instruction information, thereby making it possible to provide printed materials that meet the user's preferences. Note that whether or not to perform such adjustment can be determined depending on the user's instructions included in the print instruction information, the price setting, the user's rank, etc., as described above in the first embodiment. . Note that the primary correction in step S320 can reflect an analysis result of preference information based on print evaluation information (described later).

The printing device 300 performs the above-described automatic correction or primary correction and adjustment on the print image, executes printing (step S340 in FIG. 14), and transmits print information as in the first embodiment. (Step S304). Note that the correction based on the acquired operation history information may be performed by the server device 200 or the printing device 300. By making corrections based on such operation history information, the user's preferences regarding the main subject and shadow areas can be reflected in the print.

[Analysis of print evaluation information and reflection on prints]
FIG. 20 is a diagram showing an example of inputting print evaluation information. In the example of FIG. 20, the user of the imaging apparatus 100 inputs an evaluation of the print result in terms of the number of stars (four in the example of the figure) via the display 140, cross button 107, MENU/OK button 108, etc. be able to. Evaluation of print results may be broken down into specific items. For example, it may be divided into hue, brightness, sharpness, contrast, etc. Alternatively, the evaluation may be input in text. The information transmitting/receiving unit 130C transmits information indicating the input evaluation to the server device 200 as print evaluation information.

The server device 200 can store the received print evaluation information in the database 250 as user preference information in association with user identification information and print images. By inquiring the server device 200, the printing device 300 can call up the preference information stored in the database 250 of the server device 200 and store it in the database 350. The printing device 300 can analyze the preference information accumulated in the database 350 using, for example, an analyzer 322 constructed by machine learning (see FIG. 13), and can reflect the analysis results in the correction in step S320. . By reflecting such print evaluation information in analysis and correction, it becomes possible to provide prints that match the user's preferences as the user uses the printer, thereby improving user satisfaction.

Note that in the first and second embodiments described above, a case is explained in which a print is ordered from the imaging device 100. However, images for printing, operation history information, etc. can be sent to smartphones, personal computers, or in-store terminals (devices that can display images, input print instructions, print, etc.) via media or short-range wireless communication. and order prints from those devices.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible.

10 Image processing system 100 Imaging device 102 Imaging section 103 Lens device 104 Shutter button 105 Finder eyepiece section 106 Dial 107 Cross button 108 MENU/OK button 110 Camera body 130 Processor 130A Image processing section 130B Identification information acquisition section 130C Information transmission/reception section 130D Print instruction section 130E Initial authentication information storage section 130F First authentication information acquisition section 130G Authentication section 130H Display control section 130I Operation history information acquisition section 132 Processor 140 Display 150 Operation section 180 Memory card 200 Server device 202 Server device 210 Processor 210A Processor 211 User information management unit 212 Image storage unit 214 Print instruction transfer unit 216 Operation history information storage unit 218 Operation history information transmission unit 220 Print information transmission unit 222 Print information storage unit 250 Database 300 Printing device 302 Printing device 310 Processor 310A Processor 312 Information reception Unit 314 Print control unit 316 Operation history information reception unit 318 Operation history information storage unit 320 Print information transmission unit 322 Analyzer 350 Database 360 Print mechanism 370 Display 380 Operation unit F1 Frame F2 Frame F3 Frame F4 Frame S100 to S102 User identification information acquisition Steps S110-S126 of user authentication Steps S300-S304 of processing in the printing device Steps S310-S350 of processing in the printing device

Claims (14)

An image processing system comprising an imaging device, a server device, and a printing device,
The imaging device includes:
an imaging unit that captures an image;
an identification information acquisition unit that acquires user identification information by analyzing a first image captured by the imaging unit;
a transmitting unit that transmits the user identification information and a second image captured by the imaging unit to the server device;
a print instruction unit that transmits print instruction information regarding the second image to the server device;
Equipped with
The server device includes:
an image storage unit that stores the second image transmitted from the transmission unit in association with the user identification information;
a print instruction transfer unit that transfers the print instruction information transmitted from the print instruction unit to the printing device;
Equipped with
The printing device includes:
comprising an image printing unit that prints the second image when receiving the print instruction information transferred from the server device;
Image processing system. The imaging device includes:
an initial authentication information storage unit that stores the user identification information and initial authentication information in a linked manner;
a first authentication information acquisition unit that acquires first authentication information;
an authentication unit that authenticates a user by comparing the first authentication information and the initial authentication information;
The image processing system according to claim 1, further comprising: The initial authentication information storage unit acquires the initial authentication information by analyzing a third image captured by the imaging unit,
The image processing system according to claim 2, wherein the first authentication information acquisition unit acquires the first authentication information by analyzing a fourth image captured by the imaging unit. The imaging device includes:
an operation history information acquisition unit that acquires operation history information regarding image processing of the second image;
an operation history information transmitter that links the operation history information and the second image and transmits the result to the server device;
Furthermore,
The server device includes:
an operation history information storage unit that associates and stores the operation history information and the second image;
an operation history information transmitter that transmits the operation history information to the printing device;
further comprising,
The image processing system according to any one of claims 1 to 3. The operation history information acquisition unit of the imaging device acquires the operation history of operations performed from a first time before the start of capturing the second image to a second time after the end of capturing the second image. The image processing system according to claim 4, which acquires information. 5. The operation history information acquisition unit of the imaging device acquires information on operations related to one or more of density, contrast, saturation, white balance, and edge enhancement of the second image as the operation history information. 5. The image processing system according to 5. The operation history information acquisition unit of the imaging device includes:
By analyzing the second image, information on a main subject in the second image, information on a shadow part in the second image, or information on a highlight part in the second image is obtained,
Presenting the acquired information of the main subject, the information of the shadow part, or the information of the highlight part to a user of the imaging device,
acquiring information about a user's operation of the imaging device in response to the presentation as the operation history information;
The image processing system according to any one of claims 4 to 6. The printing device includes:
an operation history information receiving unit that receives the operation history information from the server device;
a print information transmitter that transmits print information regarding printing of the second image;
Equipped with
The image printing unit prints the second image based on the received operation history information,
When printing the second image based on the print instruction information, the print information transmitting unit:
transmitting print information including information regarding at least one of a correction applied to the second image, the second image after the correction, and settings regarding printing of the second image;
The image processing system according to any one of claims 4 to 7. The print information transmitting unit transmits the print information to the server device,
The server device further includes a print information storage unit that stores the print information and the second image in a linked manner.
The image processing system according to claim 8. The image processing system according to claim 9, wherein the server device further includes a print information transmitter that transmits the print information to the imaging device. an imaging unit that captures an image;
an identification information acquisition unit that acquires user identification information by analyzing a first image captured by the imaging unit;
a transmitting unit that transmits the user identification information and a second image captured by the imaging unit to a server device of an image processing system;
An imaging device comprising: The imaging device according to claim 11, further comprising a print instruction section that instructs a printing device of the image processing system to print the second image via the server device. an initial authentication information storage unit that stores the user identification information and the initial authentication information of the imaging device in a linked manner;
a first authentication information acquisition unit that acquires first authentication information;
an authentication unit that authenticates a user by comparing the first authentication information and the initial authentication information;
The imaging device according to claim 11 or 12, further comprising: an operation history information acquisition unit that acquires operation history information regarding image processing of the second image;
an operation history information transmitter that links the operation history information and the second image and transmits the result to the server device;
The imaging device according to any one of claims 11 to 13, further comprising:

Images