JP2019179952A - Image photographing apparatus, image transmission method, and image analysis system - Google Patents

Abstract

To solve the problem in which, when image processing such as trimming is performed on an input image, there is a problem that it is difficult to grasp a situation where a subject is placed because a background of the subject becomes narrow and on the other hand, there is also a desire to perform personal recognition and authentication while leaving a background area as much as possible so that a user can easily determine a surrounding situation of the subject from the image.SOLUTION: The image photographing apparatus includes: means for acquiring image analysis performance information of a server; means for determining whether analysis is possible on the server by comparing the analysis performance information and a subject detection area information of a live view image; means for performing a predetermined zoom process if it is determined that the analysis performance is not satisfied; and means for imaging a subject and transmitting it to the server.SELECTED DRAWING: Figure 7

Description

  The present invention relates to an image processing system that controls a camera and processes captured images.

  2. Description of the Related Art Conventionally, in an image recognition / authentication system including a camera and a server, an image obtained by photographing with a camera is transmitted to a server, and the server recognizes or authenticates a subject based on the image (hereinafter referred to as “recognition / authentication process”). An image recognition / authentication system is known. For example, there are a system for searching for lost children and a system for watching suspicious person information for children attending and leaving school.

  In the system described in Patent Document 1, if the performance required for recognition is obtained from the server and the recognition target in the image does not satisfy the performance, the server is necessary after performing image processing such as trimming or resolution change. By transferring an image that satisfies the recognition performance, the recognition accuracy is improved.

JP 2014-22970 A

  However, when image processing such as trimming is performed on the input image, there is a problem that it is difficult to grasp the situation where the subject is placed because the background of the subject becomes narrow.

  On the other hand, there is also a demand for recognition / authentication of a person while leaving a background area as much as possible so that the user can easily determine the situation around the subject from the image.

In order to solve the above-described problem, a photographing apparatus according to the present invention includes:
Means for acquiring image analysis performance information of the server, analysis performance information setting means for setting image analysis performance information, means for detecting a subject from a live view image, means for acquiring detection region information of the subject, When the analysis performance information and the detection area information are compared to determine whether analysis is possible or not at the server, and when the analysis availability determination means determines that the analysis performance is not satisfied, a predetermined zoom process is performed. Zoom means for performing, means for capturing an image of a subject and storing a recorded image, and means for transmitting the recorded image to a server.

  According to the photographing apparatus according to the present invention, by acquiring the analysis performance information from the server, an image satisfying the analysis performance of the server at the time of photographing can be photographed and sent to the server. In addition, it is possible to shoot an image that can determine the surrounding situation of the subject by shooting with a setting that satisfies the analysis performance of the server as a minimum.

1 is a schematic diagram illustrating a configuration of an imaging transmission analysis processing apparatus according to a first embodiment. FIG. 1 is a block diagram illustrating a configuration of a camera and a server according to Embodiment 1. 1 is a block diagram illustrating a configuration of a camera according to Embodiment 1. FIG. Captured image diagram illustrating an image acquired from an imaging unit according to the first embodiment and target detection information Captured image diagram illustrating an enlargement range for enlarging an object by controlling the imaging unit according to the first embodiment Captured image diagram illustrating an acquired image after target enlargement and target detection information by control of the imaging unit according to the first embodiment Sequence diagram of photographing and transmission processing according to embodiment 1 Sequence diagram of photographing and transmission processing according to embodiment 2 Schematic diagram of a camera according to Example 3 Sequence diagram of shooting and transmission processing according to embodiment 3

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. However, the constituent elements described in this embodiment are merely examples, and are not intended to limit the scope of the present invention only to them.

<Example 1>
FIG. 1 is a schematic diagram showing a configuration of an image processing system according to the present embodiment.

  In FIG. 1, the image processing system includes a camera 102 that captures an image, a server apparatus 101 that transmits and receives an image via a network, and a terminal 103 that can refer to the image stored in the server.

  The camera 102 may be a fixed camera or a wearable camera. There may be a system configuration in which a plurality of cameras exist in a large-scale system, and each camera captures a plurality of subjects at different positions. At this time, the servers to which each camera is connected may be the same, or may be servers with different performance.

  In this system, an image whose face is detected by a camera is taken and face analysis / authentication is performed by a server. An image taken by the camera is used as data for searching for a specific person. When a search target person is found as a result of recognition and authentication processing in the server 101 in a state where the search target is pre-registered in the terminal 103, the terminal 103 is notified. As an example of the terminal 103, there is a smart phone of a guardian who wants to know information of a child who is attending school.

  FIG. 2 is a block diagram showing the configuration of the server apparatus 101.

  2, the server device 101 includes a monitor 201, a VRAM 202, a BMU 203, an input device 204, a CPU 205, a ROM 206, a RAM 207, an external storage device 208, and a network I / F 209.

  For example, an image received from the camera 102 is displayed on the monitor 201.

  An image to be displayed on the monitor 201 is drawn on the VRAM 202. The image data generated in the VRAM 202 is transferred to the monitor 201 according to a predetermined rule, whereby an image is displayed on the monitor 201.

  The BMU (bit move unit) 203 is, for example, data transfer between memories (for example, between the VRAM 202 and another memory) or data between the memory and each I / O device (for example, the network I / F 209). Control the transfer.

  The input device 204 has various keys for inputting characters and the like.

  The CPU 205 controls each device based on a control program stored in the ROM 206 and the external storage device 208.

  The ROM 206 stores various control programs and data.

  The RAM 207 has a work area for the CPU 205, a data save area during error processing, a control program load area, and the like.

  The external storage device 208 stores each control program and data executed in the information processing apparatus.

  A network I / F 209 is an interface for connecting to a network line such as the Internet.

  The CPU bus 210 includes an address bus, a data bus, and a control bus. The provision of the control program to the CPU 205 can be performed from the ROM 206 and the external storage device 208, or can be performed from another information processing apparatus or the like via the network via the network I / F 209. The block diagram of the terminal 103 is the same as that in FIG.

  FIG. 3 is a block diagram illustrating the configuration of the camera 102.

  In FIG. 3, the imaging unit 301 includes a zoom lens, a diaphragm, an imaging device, and a signal processing circuit.

  The imaging unit 301 captures a subject to be recognized and generates an image.

  The imaging control unit 302 controls the imaging unit.

  A system control unit 303 is a control unit that controls the overall operation of the camera, and includes a microcomputer.

  The subject detection unit 304 analyzes an image input from the imaging unit and detects a predetermined recognition and authentication target. The target is, for example, a person or a person's face. Here, it is assumed that a known face detection technique or object detection technique is used.

  The image data recording unit 305 creates an image data file (image file) and stores it in the storage medium 306.

  The storage medium 306 is a randomly accessible storage medium.

  The image processing unit 307 performs image processing on the image data recorded on the storage medium.

  The ROM unit 308 stores the control program, the imaging setting value file 101, and various data.

  The RAM 309 includes a control program load area, an analysis performance threshold value of the server 101, and the like.

  The transmission unit 310 transmits information to the server 101. The receiving unit 311 receives information from the server 101.

  FIG. 4 is a photographed image diagram illustrating the image acquired by the imaging unit 301 and the target detection information detected by the subject detection unit 304.

  In FIG. 4, image information 401 indicates an image main body, and a face detection unit 403 of a person 402 that is a subject to be detected is included as meta information of the image. The meta information of the face detection unit includes face frame coordinates and size.

  FIG. 5 is a photographed image diagram illustrating an enlarged range for controlling the imaging unit 301 according to the analysis performance of the server 101 and zooming up the target.

  In FIG. 5, an image portion 501 is an enlargement target image, a subject 502 is a zoom-up photographing target, a frame 504 is a frame-out range due to enlargement of the imaging portion, and an image portion 504 is a portion that is a photographing target after enlargement.

  FIG. 6 is a photographed image diagram illustrating an acquired image after target enlargement under the control of the imaging unit 301 and target detection information.

  In FIG. 6, an image 601 shows an image main body obtained by zooming up and acquired from the imaging unit 301, and includes a face detection unit 603 of a person 602 that is a subject to be detected as meta information of the image.

FIG. 7 is a sequence diagram of the imaging transmission analysis process. Hereinafter, the imaging transmission analysis process will be described with reference to FIGS. 4, 5, 6, and 7. This process includes the following (a) to (c).
Get analysis performance threshold from server.
The imaging unit is controlled according to the face detection information and the analysis performance value of the live view image.
Transfer the shot image to the server and analyze the image.

  First, (a) will be described.

  The camera 102 transmits an analysis performance threshold acquisition request from the transmission unit 310 to the connection server 101 (S701). The server 101 transmits the analysis performance threshold value as a response to the receiving unit 311 of the camera 102 (S702). The analysis performance threshold received by the camera 102 is stored in the RAM unit 309. The analysis performance threshold is, for example, a ratio of the area of a face or an object to a pixel or an image.

  Next, (b) will be described.

  In the camera 102, live view images are repeatedly acquired from the imaging unit 301 (S703). Human face detection is performed on the acquired live view image (S704). When the person 402 is detected, the detected face frame information is acquired (S705), and the size of the face frame 403 is compared with the face frame size of the analysis performance threshold set in (a). If the detected face frame size satisfies the analysis performance threshold, shooting is performed (S707). At this time, the zoom-out may be performed to a minimum value that satisfies the analysis performance threshold (S706). On the other hand, if the size of the detected face 403 does not satisfy the analysis performance threshold, the imaging control unit 302 zooms up (S711). The zoom magnification at this time is obtained by the ratio between the detected face frame size and the analysis performance threshold. 5 and 6 show a state in which the person 402 detected in FIG. 4 is zoomed up. A person 602 detected in the live view image obtained again after zooming up is a person zoomed up the person 402. It is determined at the next live view image acquisition (S703) whether or not the size of the detected face frame 603 of the person 602 satisfies the analysis performance threshold.

  If the person 402 cannot be detected in the acquired live view image, the imaging control unit 302 zooms out to the initial value (S712).

  Next, (c) will be described.

  The captured image is transferred to the server 101 (S708). The server 101 performs image analysis (S709) on the transferred image, and stores the analysis result in the external storage device 208 (S710).

  Through the above processing, the camera can capture an appropriate target image in accordance with the analysis performance of the server to be connected, and can transfer only an image effective for the face recognition processing to the server. As a result, face recognition accuracy can be improved, transfer data can be reduced, and the load on the server can be reduced.

  The image data photographed and analyzed by this image processing system is used for the purpose of searching for a specific person. When a person to be searched is included in the image data, a photograph showing the target person is displayed on the screen of the terminal 103. In the present embodiment, it is also possible to perform shooting control so that the face of the search target person displayed is an image having a size that satisfies the server analysis performance threshold at a minimum. Therefore, it is easy to determine the situation where the search target person is placed from the background of the image.

<Example 2>
Hereinafter, another embodiment of the present invention will be described in detail according to the attached images.

  The zoom-up (S709) process in Embodiment 1 can only be performed by a camera equipped with a zoom mechanism. In this embodiment, an image processing system that captures an image in accordance with an analysis performance threshold acquired from a server even if the camera does not have a zoom mechanism will be described.

  The system configuration according to the present embodiment, and the block diagram of the server 101, the camera 102, and the terminal 103 are the same as those in the first embodiment, and thus the description thereof is omitted.

  FIG. 8 is a sequence diagram showing shooting and image editing processing by the camera 102 and transfer processing of the stored image to the server 101. Hereinafter, the captured image editing process of the camera 102 according to the present embodiment will be described with reference to FIG. In FIG. 8, the sequence other than the sequence in the case where the size of the detected face frame does not satisfy the server analysis performance threshold is the same as that in the first embodiment, and the description thereof is omitted.

  If the size of the detected face frame does not satisfy the analysis performance threshold, the imaging control unit 302 changes the resolution (number of pixels) of the recorded image greatly (S801), and then performs imaging (S802). The resolution changing process executed at this time is performed with the minimum resolution (number of pixels) for the detected face frame that satisfies the server analysis performance threshold. The captured image is transferred to the server 101 (S803). The server 101 performs image analysis (S804) on the transferred image and stores the analysis result in the external storage device 208 (S805). After the transfer to the server is completed, the resolution of the recorded image of the camera 102 is returned to before the change (S806).

  With the above processing, even a camera that cannot be optically zoomed can perform photographing with the number of photographing pixels appropriate for face recognition based on the analysis performance threshold of the server and the detected size of the face frame. Normally, the resolution of the recorded image is set to the minimum in consideration of the load on the network.

<Example 3>
Hereinafter, the third embodiment will be described in detail according to the attached image.

  In the zoom-up (S709) process in the first embodiment, depending on the target position, the frame may be out during zoom-up. In this embodiment, an image that captures image data necessary for effective face analysis / recognition processing even when the camera is equipped with a pan / tilt mechanism and the object is not in the center of the frame and when the camera or the object moves out of the frame. The processing system will be described.

  The system configuration according to the present embodiment, and the block diagram of the server 101, the camera 102, and the terminal 103 are the same as those in the first embodiment, and thus the description thereof is omitted.

  FIG. 9 includes a pan head 901 and a photographing system 902. The camera platform 901 includes a drive system that changes the direction of the imaging system 902 in the horizontal direction (hereinafter referred to as “pan”) and the vertical direction (hereinafter referred to as “tilt”). As a result, the shooting direction of the shooting system 902 can be changed from −30.0 degrees to +210.0 degrees with pan being 360 ° endless turning and tilt being 0 degrees in the horizontal direction.

  FIG. 10 is a sequence diagram illustrating a shooting process by the camera 102 and a transfer process of the stored image to the server 101. Hereinafter, the shooting process of the camera 102 according to the present embodiment will be described with reference to FIGS. 9 and 10. 10 is the same as FIG. 7 of the first embodiment except for the pan / tilt control (S1001, S1002) of the camera 102, and a description thereof will be omitted. In FIG. 10, when the face frame size does not satisfy the analysis performance threshold, the direction of the optical system is changed in the pan / tilt direction so that the face frame is captured at the center of the imaging frame from the coordinate information of the face frame detected in the live view (S1001) Zoom up (S711) is performed. If no face frame has been detected, the zoom magnification and pan / tilt value are reset to the initial values (S712, S1002).

  With the above processing, it is possible to perform shooting with the size of the detection frame corresponding to the analysis performance threshold of the server, even with respect to the object to be framed out or moved when zooming in depending on the position.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

101 servers, 102 cameras, 103 terminals

Claims (9)

Means for acquiring image analysis performance information of the server;
Analysis performance information setting means for setting the image analysis performance information;
Means for detecting the subject from the live view image;
Means for acquiring detection area information of the subject;
An analysis availability determination unit that compares the analysis performance information and the detection area information and determines whether analysis is possible at the server;
If it is determined that the analysis availability determination means does not satisfy the analysis performance, zoom means for performing a predetermined zoom process;
Means for imaging a subject and storing a recorded image;
Means for transmitting the recorded image to a server;
A photographing apparatus comprising: The imaging apparatus according to claim 1, wherein the image analysis performance information is a number of pixels or a ratio of an area of a face or an object to an image. The photographing apparatus according to claim 1, wherein the predetermined zoom process is a zoom-up process that increases a zoom magnification until the image analysis performance is satisfied. The photographing apparatus according to claim 1, wherein the predetermined zoom process is a zoom-out process that lowers a zoom magnification to a value that satisfies the image analysis performance as a minimum. 2. The panning / tilting mechanism according to claim 1, wherein the photographing apparatus includes a pan / tilt mechanism and pans / tilts the subject or the subject that is out of frame with the movement of the photographing device so that the subject is placed in the frame. Image processing system. 4. The image processing system according to claim 3, wherein the photographing apparatus includes a pan / tilt mechanism, and includes means for performing pan / tilt so that the subject that falls out of the frame when zooming up is included in the frame. Means for acquiring image analysis performance information of the server;
Analysis performance information setting means for setting the image analysis performance information;
Means for detecting the subject from the live view image;
Means for acquiring detection area information of the subject;
An analysis availability determination unit that compares the analysis performance information and the detection area information and determines whether analysis is possible at the server;
If it is determined that the analysis availability determination means does not satisfy the analysis performance, resolution changing means for increasing the resolution of the captured image;
Means for imaging a subject and storing a recorded image;
Means for transmitting the recorded image to a server;
A photographing apparatus comprising: The image capturing apparatus according to claim 7, wherein the image analysis performance information is a number of pixels or a ratio of an area of a face or an object to an image. A program for realizing the apparatus according to claim 1, a computer-readable storage medium describing the program, and a method.