JP2022181272A - Imaging apparatus - Google Patents

Abstract

To provide an imaging apparatus capable of obtaining a satisfactory captured image in a case of imaging under stationary light.SOLUTION: An imaging apparatus 100 comprises: a plurality of illuminating parts 103; an imaging part 110 for imaging; illumination state detection means 101a which detects an illumination state by analyzing the captured image obtained by the imaging part 110; light volume adjustment means 101b which adjusts the light volumes of the illuminating parts 103 based on the illumination state; and an image output part 112 which outputs the captured image obtained by the imaging part 110 after the adjustment by the light volume adjustment means 101b.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to imaging devices.

2. Description of the Related Art Conventionally, in stores, public facilities, etc., a photographing box for photographing a user is installed in a housing for photographing, and is used for photographing an ID photograph. Further, in order to irradiate a subject with light suitable for chromakey processing, an apparatus has also been developed that uses ambient light for photographing (see Patent Document 1).

Japanese Patent No. 6801246

2. Description of the Related Art When taking an ID photograph or the like, a strobe is used to emit a flash of light in order to take a clear image of a subject. On the other hand, it is known that, in order to photograph a natural expression, it is preferable to photograph under constant light such as room light rather than using strobe light emission. However, when shooting under constant light, it is easily affected by outside light, and exposure and color balance may not be stable. It is also affected by how the light hits it and how shadows are formed.

Accordingly, an object of the present disclosure is to provide an imaging apparatus capable of obtaining a good captured image when performing imaging under constant light.

In order to solve the above problems, in the present disclosure,
a plurality of lighting units;
a shooting unit that shoots,
lighting state detection means for analyzing a photographed image obtained by the photographing unit and detecting a lighting state;
light amount adjusting means for adjusting the light amount of the illumination unit based on the lighting state;
an image output unit for outputting a photographed image obtained by the photographing unit after adjustment by the light amount adjusting means;
To provide an imaging device having

In addition, in the imaging device of the present disclosure,
The illumination state detection means is
A learning model based on machine learning using teacher data may be used to estimate the lighting state.

Further, the imaging device of the present disclosure is
further having a plurality of light intensity sensors,
The illumination state detection means is
The illumination state obtained by analyzing the photographed image may be corrected based on the output of the light amount sensor.

Further, the imaging device of the present disclosure is
further having a strobe light emitting part,
When the illumination state satisfies a predetermined condition, the strobe light emitting unit emits light;
The image output section may output a photographed image after light emission by the strobe light emission section.

Further, the imaging device of the present disclosure is
storing a reference value of the illumination state for each of a plurality of service modes;
The light quantity adjusting means may adjust the light quantity of the illumination unit so as to approach the reference value corresponding to the selected service mode.

According to the present disclosure, it is possible to obtain a good photographed image when photographing under constant light.

1 is a schematic side view of an imaging device according to an embodiment of the present disclosure; FIG. 2 is a diagram showing the hardware configuration of the imaging device 100; FIG. FIG. 3 is a diagram showing an installation location of a light amount sensor 108; 2 is a functional block diagram of the imaging device 100; FIG. It is a flow chart which shows the processing operation of an imaging device.

Preferred embodiments of the present disclosure will be described in detail below with reference to the drawings.
<1. Device configuration>
1 is a schematic side view of an imaging device according to an embodiment of the present disclosure; FIG. The overall structure of the photographing apparatus 100 according to the present embodiment has a space SP within the frame, as in a conventional photographing box, and includes constituent elements such as the photographing unit 110 within a housing 10 provided in the frame. It has become. A chair CH is installed in the space SP, and a person can enter the space SP and be photographed by the photographing unit 110 while sitting on the chair CH. The imaging device 100 according to this embodiment has an illumination unit 103 that irradiates stationary light as room light in the space SP. The example in FIG. 1 has five lighting units 103a to 103e.

The illumination unit 103a is above the front of the user, the illumination unit 103b is below the front of the user, the illumination unit 103c is in front of the upper surface of the user, the illumination unit 103d is behind the upper surface of the user, and the illumination unit 103e is behind the user. Each is placed in an illuminated position. Although not shown, a curtain is installed to separate the space SP from the outside. When the curtain is closed after entering the room, the five lighting units 103 keep the space SP at a constant brightness. By closing the curtain, the space SP is illuminated with stationary light. Ambient light is light that changes little over time and has little unevenness. Therefore, external light affected by sunlight and flashing light from a strobe are excluded.

A photographing apparatus 100 is a photographing apparatus installed in a store, a public facility, or the like. FIG. 2 is a hardware configuration diagram of the imaging device 100. As shown in FIG. As shown in the schematic side view of FIG. 1, the photographing apparatus 100 has a frame provided with a housing 10, a space SP for a person to enter is provided inside, and a display unit 105 such as a liquid crystal display and a photographing unit such as a camera are provided. A unit 110 is installed in the housing 10 in a state recognizable by the user.

As shown in FIG. 2, the imaging apparatus 100 includes a control unit 101 that controls the entire apparatus and performs various types of arithmetic processing, and a nonvolatile storage device 102 (for example, a , hard disk, flash memory, etc.), an illumination unit 103 that illuminates the space SP, an instruction input unit 104 such as a keyboard and a touch panel, a display unit 105 such as a liquid crystal display and an organic EL display, and data such as photographed images are output to a printer. a printer 106 for printing, a communication unit 107 for communicating with other terminals and server computers via a network, a light intensity sensor 108 for measuring the amount of light, a human sensor 109 for detecting a person, and a user as a subject. A photographing unit 110 (such as a CCD camera) for photographing and a strobe light emitting unit 111 for emitting flash light are provided, and are electrically connected to each other via a bus.

The control unit 101 can be realized by a computer. For example, the control unit 101 includes a CPU (Central Processing Unit) and a RAM (Random Access Memory) which is a main memory. Realize calculation and control.

The lighting unit 103 is implemented by a lighting device. Various lighting devices such as LEDs, fluorescent lamps, light bulbs, etc. can be used as long as they can adjust the amount of light. As a lighting device that realizes the lighting unit 103, it is preferable to use an LED from the viewpoint of handling, cost, and the like. The lighting unit 103 has a light control function with variable illuminance and color temperature. The illumination unit 103 can be realized by, for example, combining individual LEDs of R, G, and B into a unit. Alternatively, a unit may be used in which individual R, G, and B LEDs or color-variable LED lights are added to the main white LED for increasing the amount of light and for adjusting the color.

The instruction input unit 104 and the display unit 105 can be realized in various forms, but in this embodiment, the instruction input unit 104 and the display unit 105 are realized as a touch panel display. Therefore, the touch panel display serves both as the instruction input section 104 and the display section 105 . A communication unit 107 performs communication via a computer network such as the Internet. The communication unit 107 can transmit data such as captured images to a smartphone or the like held by the user via the Internet. Furthermore, a short-range wireless communication unit may be provided for performing direct wireless communication with other terminals without going through a network. The communication method used by the short-range wireless communication unit is not particularly limited as long as it is a format for wireless communication at a relatively short distance, and various known methods such as Bluetooth and wireless LAN can be used. The printer 106 functions as an image output unit 112 that prints out the captured image, the communication unit 107, and the short-range wireless communication unit as the image output unit 112 that outputs data of the captured image. Each component shown in FIG. 2 is housed in a housing having an external appearance as shown in FIG.

The light amount sensor 108 is a sensor that measures the amount of light, and is realized by a known illuminance meter or the like using a photodiode, a phototransistor, or the like. The light amount sensor 108 detects the amount of light such as illuminance, but may also be capable of detecting color. The light amount sensor 108 is installed at a position suitable for measuring the amount of light in the range reflected in the captured image. The number and positions of the light amount sensors 108 to be installed can be determined as appropriate. FIG. 3 is a diagram showing the installation location of the light intensity sensor 108. As shown in FIG. FIG. 3 is a view seen from the AA line side in FIG. This embodiment has five light amount sensors 108a to 108e. As shown in FIG. 3, the five light amount sensors are installed on the wall surface of the user's back side. The light sensor 108a is above the user, the light sensor 108b is on the right side of the user (left side in FIG. 3), the light sensor 108c is on the right side of the user (left side in FIG. 3), and the light sensor 108d is on the left side of the user (in FIG. 3). The light intensity sensor 108e is installed on the user's left side (right side in FIG. 3) on the bottom side.

FIG. 4 is a functional block diagram of the imaging device 100. As shown in FIG. In the hardware configuration shown in FIG. 2, the CPU reads the program stored in the storage device 102 into the RAM and executes it, so that the control unit 101 implements the illumination state detection means 101a and the light amount adjustment means 101b.

The lighting state detection means 101a analyzes the photographed image obtained by the photographing unit 110 and obtains the lighting state including the influence of external light. The light quantity adjusting means 101b adjusts the light quantity of the lighting section 103 based on the lighting condition calculated by the lighting condition detecting means 101a. The image output unit 112 outputs a photographed image obtained after adjusting the amount of light. The image output unit 112 is implemented by the printer 106 that prints out the captured image and the communication unit 107 that outputs data of the captured image. A short-range wireless communication unit may be used as the image output unit 112 .

The storage device 102 stores programs for executing various functions realized by the control unit 101 in addition to the programs for realizing the illumination state detection means 101a and the light amount adjustment means 101b. The storage device 102 also stores various data necessary for processing of the imaging device 100 . The storage device 102 also realizes a function as a reference value storage unit that stores reference values of lighting conditions.

The reference value storage means stores the reference value of the illumination state in association with each service mode. The reference value of the illumination state is a value indicating the illumination state of only the light (steady light) from the illumination unit 103 without including the influence of external light. A service mode is a mode that determines the mode of a captured image. Service modes include, for example, an ID photo mode and a fan photo mode. The ID photo mode is for various ID photos, and the fan photo mode is for viewing and enjoying photos. In each of these modes, the mode of illumination when capturing a captured image is different. Therefore, the reference value of the illumination state also differs depending on the service mode.

<2. Processing operation>
Next, the processing operation of the photographing apparatus according to this embodiment shown in FIG. 1 will be described. FIG. 5 is a flow chart showing processing operations by the imaging device. In the photographing device 100, the human sensor 109 is operating. When a person enters the room, the human sensor 109 detects the person (step S1). When detecting a person, the human sensor 109 transmits a signal indicating that the person has been detected to the control unit 101 . When a signal indicating that a person has been detected is received from the human sensor 109, the control unit 101 of the photographing apparatus 100 turns on the lighting unit 103 (step S2). Specifically, the control unit 101 controls the lighting units 103a to 103e so that they are lit with predetermined illuminance and color.

Subsequently, the control unit 101 displays a screen for selecting a service mode on the display unit 105. FIG. Specifically, the display unit 105 displays a plurality of selectable service modes on the screen (step S3). In this embodiment, the display unit 105 and the instruction input unit 104 are implemented by a touch panel. Therefore, it is possible to display the name of the service in a predetermined area of the display screen, and determine that the service mode has been selected when the area is touched.

More specifically, when displaying the service mode, the amount corresponding to the service is also displayed. When a service mode is selected by the user, a billing process corresponding to the selected service mode is performed by a billing processing unit (not shown).

When the service mode is selected, the control unit 101 refers to the storage device 102 and acquires the lighting state reference value corresponding to the selected service mode (step S4). Furthermore, in parallel with the process of step S4, photographing and light amount measurement are started (step S5). In step S5, the photographing unit 110 starts photographing. Specifically, the photographing unit 110 performs photographing and acquires a photographed image, which is image data. The photographing unit 110 performs photographing at a predetermined photographing rate and sends the obtained photographed image to the control unit 101 . Then, the control unit 101 sequentially sends the acquired captured images to the display unit 105 . The display unit 105 sequentially displays the captured images acquired from the control unit 101 . That is, the photographing unit 110 performs continuous photographing, and the photographed images obtained by photographing are consecutively photographed as a moving image.

The shooting rate (fps: frames per second) can be set as appropriate. For example, when the shooting rate is 30 fps, 30 shot images are obtained per second, and the display unit 105 sequentially displays the 30 shot images per second. Therefore, the user can recognize the captured image displayed on the display unit 105 as a moving image. Therefore, so-called live view display is performed on the display unit 105 .

On the other hand, in step S5, measurement of the amount of light is started. Specifically, the control unit 101 transmits a signal to the light amount sensor 108 to cause the light amount to be measured. The light intensity sensor 108 acquires the light intensity by measurement. The amount of light can be acquired, for example, as brightness (illuminance: unit lux). Each of the light amount sensors 108a to 108e transmits the acquired light amount to the control section 101. FIG.

Further, the control unit 101 extracts one photographed image at an arbitrary timing from among the sequentially obtained photographed images. Then, the lighting condition detection means 101a analyzes the extracted photographed image and detects the lighting condition including the influence of outside light (step S6). In this embodiment, in step S6, the illumination state is specified not only by analyzing the captured image but also by using the light amount values obtained from the light amount sensors 108a to 108e. Any index may be used as the illumination state as long as it reflects the illumination state. In this embodiment, illuminance (unit lux) corresponding to the amount of light is used. The illumination state detected in step S6 includes the influence of outside light. Details of the detection of the illumination state in step S6 will be described later.

After the lighting condition is detected, it is determined whether or not the detected lighting condition is within a correctable range (step S7). Whether or not the illumination state is within a correctable range can be determined by whether or not the illumination state satisfies a predetermined condition. The predetermined condition can be appropriately set according to the target illumination state. For example, the predetermined condition may be that the detected lighting condition is less than a predetermined ratio to the set value. Specifically, the control unit 101 refers to the storage device 102, which is the reference value storage means, in the selected service mode, and acquires the reference value of the illumination state corresponding to the service mode. Then, the control unit 101 compares the lighting state detected in step S6 with the acquired reference value. If the calculated illumination state is equal to or greater than the predetermined ratio, it is determined that the correction is not possible. Here, the predetermined ratio can be set as appropriate. If the calculated illumination state is equal to or greater than a predetermined ratio with respect to the reference value, it means that the influence of outside light is too great. In this case, since it is difficult to appropriately illuminate the user with the constant light from the lighting unit 103, it is determined that the correction by the lighting unit 103 is impossible.

If it is determined that the correction is possible (step S7: YES), the illuminance of the lighting unit is adjusted based on the detected lighting state (step S8). Specifically, the light amount adjusting means 101b adjusts the light amount of the lighting unit 103 so as to approach the reference value corresponding to the selected service mode. The relationship between the illumination state and the light amount of the illumination unit 103 can be set in various ways. For example, in a photographed image, an area (pixel block, set of pixels) affected by each illumination unit is set as an area of influence, and according to the lighting state of each area of influence obtained by analyzing the photographed image, The light amount of the corresponding lighting unit 103 may be adjusted. Furthermore, the exposure in the photographing unit 110 may be adjusted. In this case, the control unit 101 performs aperture control of a diaphragm installed parallel to the lens surface of the imaging unit 110 to adjust exposure.

On the other hand, if it is determined in step S7 that it is out of the correctable range (step S7: NO), the photographing is changed to strobe photography (step S9). Specifically, the strobe light emitting unit 111 is controlled to prepare for strobe light emission.

After completing the process of step S8 or step S9, the captured image is output (step S10). Specifically, the control unit 101 acquires a captured image captured after the light amount of the illumination unit 103 is adjusted by the light amount adjusting unit 101 b or after strobe light emission, and passes the captured image to the image output unit 112 . Then, the image output unit 112 outputs the captured image acquired from the control unit 101 . At this time, the control unit 101 captures a still image to be used for outputting the captured image. At this time, the mode may be switched to a high quality mode or to a still image shooting mode in order to make the captured image have a high resolution. When the photographed image is to be printed out, it is printed out from the printer 106 which is the image output unit 112. When the photographed image is to be data-outputted, it is sent from the communication unit 107 which is the image output unit 112 via the Internet to the user's smartphone. etc., and output the captured image. That is, when the illumination state satisfies a predetermined condition, the strobe light emitting unit 111 emits light, and the image output unit 112 outputs the photographed image after the strobe light emitting unit 111 emits light.

<Detection of lighting conditions>
The detection of the illumination state in step S6 will be described. The detection of the illumination state is performed by image analysis of the photographed image extracted by the control unit 101 by the illumination state detection means 101a. Image analysis by the illumination state detection means 101a may use a learning model that has already been learned by machine learning, or may use an image analysis algorithm.

When using a learning model, a learning model based on deep learning using AI (artificial intelligence) represented by a neural network is used. In this case, the lighting state detection means 101a estimates the lighting state using a learning model based on machine learning using teacher data. A learning model provided in the illumination state detection means 101a has an input layer G1A, a plurality of intermediate layers G1B, and an output layer G1C. The input layer G1A has the same number of nodes as the number of pixels of the captured image to be input. For example, when the number of pixels of an input photographed image is horizontal X×vertical Y, there are X×Y input nodes. The intermediate layer G1B is a layer positioned between the input layer G1A and the output layer G1C and has multiple layers. In each layer, parameters such as weights and biases between nodes are set, and by updating these parameters, the values of the nodes on the output side are made different. A pixel value of each shooting pixel is input to the input layer G1A, and an illumination state corresponding to a predetermined area is output to each output node of the output layer G1C.

The predetermined area is an area made up of one or more pixels in the captured image. If one pixel constitutes one region, the same number of output nodes as the number of pixels are required. As each area is set in smaller units, it becomes possible to calculate an illumination state with a more detailed position specified. Any index may be used as the illumination state as long as it reflects the illumination state. In this embodiment, illuminance (unit lux) corresponding to the amount of light is used.

In the learning stage of the learning model, first, teacher data is prepared. As training data, a set of a photographed image and a lighting condition actually measured for each region when the photographed image was photographed is used. As the illumination state at this time, the difference between the theoretical value of the amount of light in a state in which external light is completely blocked and the actual value of the amount of light measured by a light amount sensor in the same environment as during actual operation is used. In the same environment as during actual operation, the amount of light affected by outside light is obtained.

Then, the output value of the learning model is evaluated using the illumination state, which is teacher data. Specifically, the output result is evaluated using an error function (also referred to as an objective function, a loss function, or a cost function) based on the lighting state, which is the output result from the output layer G1C, and the lighting state included in the teacher data. do.

Next, based on the obtained evaluation result, it is determined whether or not to update parameters such as weights and biases between nodes used in the intermediate layer G1B. For example, in the process of optimizing (minimizing or maximizing) the error function by a gradient descent method such as the steepest descent method, if the error function is below (or above) a threshold, it is determined not to update the parameters. For example, when the error function exceeds the threshold (or becomes less than the threshold), it is determined to update the parameters. This threshold can be set arbitrarily. If it is determined to update the parameters, the parameters used in the intermediate layer G1B are updated. A trained learning model is obtained by performing such processing on a large number of teacher data.

The lighting condition detection means 101a equipped with the learning model trained as described above receives the captured image, inputs the value of each pixel to each input node of the input layer, and outputs the lighting condition of each area. do. A program and data for realizing a learned learning model can be stored in a storage device and read by the control unit 101 at the time of processing to be executed.

When an image analysis algorithm is used without using a learning model, for example, the illumination state detection means 101a is implemented by the CPU executing a program using the image analysis algorithm. As an image analysis algorithm, for example, pixel values are used for each predetermined area to detect brightness (brightness), and the illuminance in each area is estimated from the relationship with surrounding pixels.

<Reflection of measured value by light intensity sensor>
In step S6, the illumination state can also be detected only by analyzing the photographed image. However, in this embodiment, in order to further improve the detection accuracy, the illumination state is specified using the light amount values obtained from the light amount sensors 108a to 108e. That is, the illumination state detection means 101a corrects the illumination state obtained by analyzing the photographed image based on the outputs of the light amount sensors 108a to 108e. Specifically, the illuminance values obtained using the learning model or the image analysis algorithm are corrected using the illuminance values obtained from the light amount sensors 108a to 108e.

It is possible to set in advance which areas in the captured image are affected by the light amounts measured by the light amount sensors 108a to 108e. Therefore, the illuminance of each region in the captured image can be corrected based on the measured values of the respective light amount sensors 108a to 108e. For example, assume that the measurement value of the light amount sensor 108a is set to affect the illuminance in the A area by 30% and the illuminance in the B area by 20%. In this case, the illuminance of the A region is corrected by a value corresponding to 30% of the difference from the illuminance of the light quantity sensor 108a, and the illuminance of the B region is corrected by a value of 20% of the difference from the illuminance of the light quantity sensor 108a. . Similarly, the illuminance value of each area in the captured image is corrected according to the value of the light amount sensor, and the illumination state of each area is obtained.

Although the preferred embodiments of the present disclosure have been described above, the present disclosure is not limited to the above embodiments, and various modifications are possible. For example, in the imaging apparatus according to the above embodiment, the display unit and the instruction input unit are configured by a touch panel, but the instruction input unit may be physical buttons electrically connected to the control unit 101 .

DESCRIPTION OF SYMBOLS 100... Imaging device 101a... Illumination state detection means 101b... Light quantity adjustment means 101... Control part 102... Storage device 103... Illumination part 104... Instruction input part 105... Display unit 106... Printer 107... Communication unit 108... Light sensor 109... Human sensor 110... Photographing unit 111... Strobe light emitting unit 112... Image output unit

Claims (5)

a plurality of lighting units;
a shooting unit that shoots,
lighting state detection means for analyzing a photographed image obtained by the photographing unit and detecting a lighting state;
light amount adjusting means for adjusting the light amount of the illumination unit based on the lighting state;
an image output unit for outputting a photographed image obtained by the photographing unit after adjustment by the light amount adjusting means;
A photographing device having a The illumination state detection means is
2. The photographing device according to claim 1, wherein the lighting state is estimated using a learning model based on machine learning using teacher data. further having a plurality of light intensity sensors,
The illumination state detection means is
3. The photographing apparatus according to claim 1, wherein the illumination state obtained by analyzing the photographed image is corrected based on the output of the light amount sensor. further having a strobe light emitting part,
When the illumination state satisfies a predetermined condition, the strobe light emitting unit emits light;
4. The photographing apparatus according to claim 1, wherein said image output section outputs a photographed image after light emission by said strobe light emitting section. storing a reference value of the illumination state for each of a plurality of service modes;
5. The photographing apparatus according to claim 1, wherein said light amount adjusting means adjusts the light amount of said lighting section so as to approach said reference value corresponding to said selected service mode.

Images