JP2015126475A - Imaging apparatus, image distribution system, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus and a method capable of executing trimming processing in an imaging apparatus such as monitor camera based on instructions given by a client.SOLUTION: An imaging apparatus 100 includes: an imaging unit 101; a storage 108 that stores images taken by the imaging unit 101; an image processing section 106 that carries out an image processing on an image stored in the storage 108; and a communication section 111 that transmits a processed image generated by the image processing section 106 to the client. The image processing section 106 executes the trimming processing on the taken image. The image processing section 106 determines a cut-out area in the image according to a piece of trimming area instruction information received from the client 120. The images stored in the storage 108 are images before trimming, and the client 120 is capable of designating a desired area as a trimming area.

Description

  The present disclosure relates to an imaging apparatus, an image distribution system, an image processing method, and a program. Specifically, for example, the present invention relates to an imaging apparatus, an image distribution system, an image processing method, and a program that transmit a captured image of a monitoring camera or the like to a client such as a remote PC via a network.

  Recently, surveillance cameras are installed in various places as one configuration of the security system. Many surveillance cameras have a wide-angle lens such as a fisheye lens, and have a configuration for capturing a wide range of images around the camera. The captured image of the monitoring camera is transmitted to a client such as a PC via a network, displayed on a display unit on the client side, and stored in a storage unit such as a hard disk.

  However, many surveillance cameras include a wide-angle lens such as a fisheye lens as described above, and can capture a wide range of images around the camera. For this reason, distortion occurs in the captured image, and when image display is performed, it is necessary to perform image correction such as distortion correction as a pre-process. In addition, a large number of videos continuously captured by a large number of surveillance cameras attached to various locations are transmitted to a client such as a PC that performs image confirmation via a network. When the number of cameras is large, the amount of transmission data increases, which may cause network congestion and delay image distribution.

As a configuration for solving such a problem, a method of cutting out a predetermined part of an image from a captured image in a data processing unit on the monitoring camera side and transmitting only the part of the image to a client is used in many systems. ing.
In other words, image clipping, that is, trimming processing is executed on the monitoring camera side, and only a part of the trimmed image is transmitted to the client. By performing this trimming process, transmission data is reduced, and it is possible to smoothly transmit the captured image data of a large number of surveillance cameras.

However, when trimming is performed on the monitoring camera side and transmitted to the client in this way, the client can confirm only an image of a partial area in the captured image, that is, an image cut out by trimming.
That is, there arises a problem that the image area cut out by the trimming cannot be confirmed thereafter.
For example, Japanese Patent Application Laid-Open No. 2006-148767 is known as a prior art that discloses a system that distributes a camera image to a client.

JP 2006-148767 A

  The present disclosure has been made in view of the above-described problems, for example, an imaging apparatus, an image distribution system, and an image capable of confirming, on the client side, all images captured by a camera including a wide-angle lens, for example. It is an object to provide a processing method and a program.

The first aspect of the present disclosure is:
An imaging unit;
An image processing unit that performs image processing on a captured image of the imaging unit;
A communication unit that transmits the processed image generated by the image processing unit to the client;
The image processing unit
In the imaging apparatus, a trimming process for cutting out a partial area from the photographed image is executed, and an image cut-out area in the trimming process is determined according to trimming area designation information received from the client.

  Furthermore, in an embodiment of the imaging device of the present disclosure, the imaging device has a storage unit that stores a captured image of the imaging unit, and the storage unit stores image data of the entire region of the captured image before trimming processing. The image processing unit stores the image and performs a trimming process on the image stored in the storage unit in accordance with a request from the client.

  Furthermore, in an embodiment of the imaging device according to the present disclosure, the communication unit receives a processing request command from the client, and the image processing unit determines a trimming region according to trimming region designation information included in the processing request command. To do.

  Furthermore, in an embodiment of the imaging device of the present disclosure, the processing request command is a reproduction start command that is transmitted to the imaging device when the client reproduces a captured image.

  Furthermore, in an embodiment of the imaging apparatus of the present disclosure, the processing request command is a trimming area designation command that is transmitted to the imaging apparatus when the client designates a trimming area.

  Furthermore, in an embodiment of the imaging apparatus according to the present disclosure, the image processing unit includes a distortion correction unit that performs distortion correction of the captured image, and a trimming process that extracts a partial region from the distortion correction image generated by the distortion correction unit. A trimming unit for executing

  Furthermore, in an embodiment of the imaging device of the present disclosure, the imaging unit includes a wide-angle lens, and is configured to capture a wide-area image around the imaging unit via the wide-angle lens, and the distortion correction unit includes the wide-angle lens. Correction for removing distortion included in the image taken through the camera is performed.

  Furthermore, in an embodiment of the imaging apparatus according to the present disclosure, the imaging apparatus includes an encoding unit that encodes the processed image generated by the image processing unit, and the communication unit is generated by the encoding unit. The encoded image is transmitted to the client.

  Furthermore, in an embodiment of the imaging apparatus according to the present disclosure, the imaging apparatus executes an encoding process of a captured image of the imaging unit to generate an encoded image, and the encoding unit generates A storage unit that stores an encoded image; and a decoding unit that generates a decoded image by decoding the encoded image stored in the storage unit, and the image processing unit is configured to process the decoded image generated by the decoding unit. Perform image processing.

  Furthermore, in an embodiment of the imaging apparatus according to the present disclosure, the decoding process in the decoding unit and the image processing in the image processing unit are executed according to a reproduction process start command received from the client.

  Furthermore, in an embodiment of the imaging apparatus according to the present disclosure, the encoding process of the captured image in the encoding unit and the storage process in the storage unit are executed according to a recording start command received from the client.

  Furthermore, in one embodiment of the imaging device of the present disclosure, the imaging device is a surveillance camera.

  Furthermore, in an embodiment of the imaging device of the present disclosure, the imaging device is a camera used for a security system.

Furthermore, the second aspect of the present disclosure is:
An image distribution system having an imaging device and a client,
The client
Trimming area designation information that is designation information of an image cut-out area from a captured image of the imaging apparatus is transmitted to the imaging apparatus,
The imaging device
An imaging unit;
An image processing unit that performs image processing on a captured image of the imaging unit;
A communication unit that transmits the processed image generated by the image processing unit to the client;
The image processing unit
In the image distribution system, an image cutout area from the photographed image is determined according to trimming area designation information received from the client, and a trimming process is executed.

Furthermore, the third aspect of the present disclosure is:
An image processing method executed in the imaging apparatus,
The image processing unit executes an image processing step of performing image processing on the captured image of the imaging unit,
In the image processing step,
Perform a trimming process to cut out a partial area from the captured image,
In the trimming process, the image processing method determines an image cutout area according to trimming area designation information received from the client.

Furthermore, the fourth aspect of the present disclosure is:
A program for executing image processing in an imaging apparatus;
Let the image processing unit execute image processing on the captured image of the imaging unit,
In the image processing,
A trimming process for cutting out a partial area from the captured image is executed,
In the trimming process, a program for determining an image cutout area according to trimming area designation information received from the client is provided.

  Note that the program of the present disclosure is a program that can be provided by, for example, a storage medium or a communication medium provided in a computer-readable format to an information processing apparatus or a computer system that can execute various program codes. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the information processing apparatus or the computer system.

  Other objects, features, and advantages of the present disclosure will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

According to the configuration of an embodiment of the present disclosure, an apparatus and a method for executing trimming processing in an imaging device such as a monitoring camera based on an instruction from a client are realized.
Specifically, the imaging unit, a storage unit that stores a captured image of the imaging unit, an image processing unit that performs image processing on the stored image of the storage unit, and a processed image generated by the image processing unit are transmitted to the client. The imaging apparatus includes a communication unit, and the image processing unit executes a trimming process of a captured image. The image processing unit determines an image clipping region according to trimming region designation information received from the client. The image stored in the storage unit is an image before trimming, and the client can designate an arbitrary area as the trimming area.
With this configuration, an apparatus and a method for executing trimming processing in an imaging apparatus such as a monitoring camera based on an instruction from a client are realized.
Note that the effects described in the present specification are merely examples and are not limited, and may have additional effects.

It is a figure explaining the hardware structural example of a general image delivery system. It is a figure explaining the picked-up image using a wide angle lens. It is a figure explaining the example of a correction | amendment of the picked-up image using a wide angle lens. It is a figure explaining an example of the trimming process which is a cut-out process of a picked-up image. It is a figure explaining the hardware structural example of the image delivery system containing the imaging device of this indication. It is a figure which shows the flowchart explaining an example of the image photography recording process sequence by an imaging device. It is a figure explaining the structural example of an imaging | photography recording process start command. It is a sequence diagram explaining an example of an image photographing recording processing sequence. It is a figure which shows the flowchart explaining an example of the reproduction | regeneration processing sequence of a picked-up image. It is a figure explaining the structural example of the reproduction | regeneration processing start command. It is a figure explaining an example of the trimming area | region designation | designated process in a client. It is a figure explaining the structural example of a trimming area | region designation command. FIG. 10 is a sequence diagram illustrating an example of a captured image reproduction processing sequence. FIG. 10 is a sequence diagram illustrating an example of a captured image reproduction processing sequence. It is a sequence diagram explaining an example of an image photographing recording processing sequence. FIG. 10 is a sequence diagram illustrating an example of a captured image reproduction processing sequence. FIG. 10 is a sequence diagram illustrating an example of a captured image reproduction processing sequence. It is a sequence diagram explaining an example of the live streaming processing sequence of a picked-up image. It is a sequence diagram explaining an example of the live streaming processing sequence of a picked-up image.

Hereinafter, details of the imaging apparatus, the image distribution system, the image processing method, and the program of the present disclosure will be described with reference to the drawings. The description will be made according to the following items.
1. 1. General configuration example of image distribution system 2. Image distribution system according to the present disclosure 3. Processing Sequence in Image Distribution System of Present Disclosure 3-1. Image capturing and recording processing sequence by imaging apparatus 3-2. 3. Transmission of captured image of imaging device to client and reproduction processing sequence Processing sequence in the imaging apparatus when executing image recording, image reproduction, and live streaming 4-1. Image recording processing sequence 4-2. Reproduction processing of recorded image without image processing (digital PTZ) such as trimming 4-3. Reproduction processing of recorded image accompanied by image processing (digital PTZ) such as trimming 4-4. Live streaming processing of captured images without image processing (digital PTZ) such as trimming 4-5. 4. Live streaming processing of captured images with image processing (digital PTZ) such as trimming Summary of composition of this disclosure

[1. General configuration example of image distribution system]
First, a general configuration example of an image distribution system that transmits an image captured by a camera such as a monitoring camera to a client such as a PC via a network will be described with reference to FIG.

  FIG. 1 shows an imaging device (network camera) 10 such as a surveillance camera and a client (PC or the like) 20. The imaging device 10 and the client 20 are configured to transmit and receive data via the network 30.

The imaging device 10 is, for example, a surveillance camera, and includes a wide-angle lens such as a fisheye lens, and continuously captures a wide range of images around the camera, that is, captures a video that is a moving image.
For example, it is a camera used in a security system for finding a suspicious person.

As illustrated in FIG. 1, the imaging apparatus 10 includes an imaging unit 11, an image processing unit 12, a control unit (processor) 13, an encoding unit 14, a memory 15, and a communication unit 16.
The image processing unit 12 includes a distortion correction unit 12a and a trimming unit 12b.

The imaging device 10 outputs the captured image of the imaging unit 11 to the image processing unit 11. The image processing unit 12 includes a distortion correction unit 12a and a trimming unit 12b, and performs distortion correction processing and trimming processing on a captured image.
The imaging unit 11 captures a wide range of images around the camera via a wide-angle lens such as a fisheye lens. Distortion occurs in a photographed image using a wide-angle lens. An example of a captured image using a wide-angle lens will be described with reference to FIG.

  FIG. 2A shows an example of the shooting environment. In order to explain the distortion generated in the captured image in an easy-to-understand manner, the subject 50 is the ground on which a lattice pattern is drawn. The imaging device 10 provided with a wide-angle lens such as a fisheye lens is set at the upper center of the subject 50 to shoot.

  An example of an image shot in the shooting environment shown in FIG. 2A is the shot image shown in FIG. As shown in FIG. 2B, the ground grid pattern on the captured image has a curved distortion. This is image distortion caused by shooting using a wide-angle lens.

In order to remove the distortion of the captured image shown in FIG. 2B and generate an image having the same shape as the actual subject, it is necessary to perform image correction, that is, distortion removal.
An example of this correction processing will be described with reference to FIG.
FIG. 3 shows (b) a captured image similar to FIG. 2B and (c) a corrected image generated by correction processing (distortion correction) of the captured image.

  The distortion correction unit 12a of the imaging apparatus 10 illustrated in FIG. 1 performs distortion correction on the captured image in FIG. 3B, and generates a corrected image in FIG. The corrected image shown in FIG. 3C is an image having the same lattice pattern as that of the subject 50 in the photographing environment shown in FIG. The grid pattern on the ground is a restoration of the shape of the actual subject.

The distortion correction unit 12a of the image processing unit 12 of the imaging apparatus 10 illustrated in FIG. 1 performs the correction process described with reference to FIG. 3, generates a corrected image from which distortion has been removed, and outputs the corrected image to the trimming unit 12b. To do.
The trimming unit 12b executes an image clipping process. One of the purposes of this image cut-out process is to reduce the amount of data transmitted to the client. The trimming unit 12b executes a process of cutting out only a part of a predetermined area from the entire area of the captured image.

An example of the trimming process performed by the trimming unit 12b will be described with reference to FIG.
FIG. 4 shows (c) a corrected image and (d) a trimmed image.
(C) The corrected image is a corrected image shown in FIG. 3C, and is a corrected image generated by the distortion correcting unit 12a.
The trimming unit 12b sets a trimming area 52 in the corrected image, cuts out the set trimming area 52, and generates a trimmed image shown in FIG.
This (d) trimmed image becomes a transmission image to the client 20.

Other components of the imaging device 10 illustrated in FIG. 1 will be described.
The control unit (processor) 13 includes a CPU having a program execution function, for example, and executes control according to a program recorded in advance in the memory 15 to perform image capturing, correction, trimming, image transmission, and the like. Controls a series of processes executed by.

The encoding unit 14 executes an encoding process (ENCODE) of the trimmed image generated by the image processing unit 12 and generates compressed image data for transmission. By this encoding process, the amount of data transmitted via the network 30 is further reduced.
The memory 15 is used for a program executed by the control unit 13, a work area for executing the program, and a temporary recording area for a captured image.
The communication unit 16 performs data transmission / reception with the client 20 via the network 30.

In this way, the image captured by the imaging unit 11 is
(S1) Distortion correction by the distortion correction unit 12a,
(S2) Trimming processing by the trimming unit 12b;
(S3) Encoding process by the encoding unit 14,
After these series of processes (S1 to S3) are performed, the data is transmitted to the client 20 via the communication unit 16.

Next, the configuration and processing of the client 20 will be described. The client 20 is configured by, for example, a PC, a portable terminal, or the like.
As illustrated in FIG. 1, the client 20 includes a communication unit 21, a decryption unit 22, a display unit 23, a storage unit 24, a control unit 25, and an input unit 26.

The image data transmitted from the imaging device 10 is received by the communication unit 21 of the client 20 via the network 30.
The image data received by the communication unit 21 is subjected to a decoding process (DECODE) in the decoding unit 22.
The decoded image is displayed on the display unit 23. Further, it is stored in the storage unit 24.
The storage unit 24 is a non-volatile memory, and the recorded image is saved regardless of whether the client 20 is turned on or off.

The control unit (processor) 25 includes, for example, a CPU having a program execution function, and executes control according to a program recorded in advance in the storage unit 24 to receive, decode, display, and record images, etc. Controls a series of processes executed by.
The input unit 26 includes an operation unit and the like that can be operated by the user. The input unit 26 instructs various processes to be executed in the client 20 such as an image display instruction, an image reading instruction from the storage unit 24, or a command transmission to the imaging device 10. Used for input.

In the system shown in FIG. 1, trimming that is image cropping processing is performed on the imaging apparatus 10 side, and only the trimmed image is provided to the client 20.
As described above, with such a configuration, the client 20 cannot check image data in an image area other than the trimming area.

[2. About the image distribution system of the present disclosure]
Next, the configuration and processing of the image distribution system according to the present disclosure will be described with reference to FIG.
FIG. 5 shows an imaging device (network camera) 100 such as a surveillance camera and a client (PC etc.) 120 as in FIG. The imaging apparatus 100 and the client 120 are configured to transmit and receive data via the network 130.

The imaging apparatus 100 is, for example, a surveillance camera, and includes a wide-angle lens such as a fisheye lens, and continuously captures a wide range of images around the camera, that is, captures a video that is a moving image.
For example, it is a camera used for a security system such as discovery of a suspicious person.

As illustrated in FIG. 5, the imaging apparatus 100 includes an imaging unit 101, a multiplexer 102 that performs input switching processing, an encoding unit 103, a decoding unit 104, a multiplexer 105, an image processing unit 106, a storage unit interface 107, a storage unit 108, A control unit (processor) 109, a memory 110, and a communication unit 111 are included.
The image processing unit 106 includes a distortion correction unit 106a and a trimming unit 106b.

The imaging apparatus 100 inputs the captured image of the imaging unit 101 to the encoding unit 103 via the multiplexer 102.
The encoding unit 103 executes an image encoding process (ENCODE). That is, compressed image data is generated to reduce the amount of data stored in the storage unit 108. The recording image data with the data amount reduced by this encoding process is stored in the storage unit 108 via the storage unit I / F 107.

The storage unit 108 is a non-volatile memory, and stored data is saved without being erased regardless of whether the imaging apparatus is powered on or off.
The image data stored in the storage unit 108 is read out in response to a request from the client 120, for example. Specifically, for example, when an [image reproduction processing start command] is input from the client 120 to the imaging apparatus 100 via the network 130, the image data stored in the storage unit 108 is read.

The image data read from the storage unit 108 is input to the decoding unit 104.
The image data read from the storage unit 108 is encoded compressed image data.
The decoding unit 104 executes a decoding process (DECODE) of the compressed image data.
The decoded image data is input to the image processing unit 106 via the multiplexer 105.

  Note that the decoded image data generated by the decoding processing in the decoding unit 104 is image data similar to the captured image data output from the imaging unit 101. That is, it is an image photographed through a wide-angle lens such as a fisheye lens, and has the distortion described above with reference to FIG.

  The image processing unit 106 includes a distortion correction unit 106a and a trimming unit 106b. The distortion correction unit 106a executes the captured image correction process (distortion correction) described above with reference to FIG. That is, for example, a correction process for removing distortion from an image having distortion as shown in FIG. 3B is executed to generate a correction image shown in FIG.

The distortion correction unit 106a executes the correction process described with reference to FIG. 3, generates a corrected image from which distortion is removed, and outputs the corrected image to the trimming unit 106b.
The trimming unit 106b executes an image clipping process. This is the image cut-out process described above with reference to FIG.

In the configuration of FIG. 1 described above, the trimming process is executed as a process of cutting out a predetermined fixed area in the captured image.
However, the trimming unit 106b of the imaging apparatus 100 having this configuration determines the trimming area according to the trimming area designation information from the client 120 and executes trimming.

That is, the trimming process executed in the imaging apparatus 100 shown in FIG. 5 does not perform the trimming process using the trimming area as a fixed area determined in advance.
In this configuration, the client 120 can designate an arbitrary image area as the trimming area, and the trimming unit 106b determines the trimming area according to the trimming area designation information of the client 120 and executes the trimming.

The client 120 outputs a command including [trimming area designation information] which is designation information of an image clipping area to the imaging apparatus 110.
The trimming unit 106b of the imaging apparatus 100 determines a trimming area (image cutout area) according to [trimming area designation information] included in the command received from the client 120, and executes a trimming process (image cutout process).

If the imaging apparatus 100 has not received [trimming area designation information] from the client 120, the trimming process is executed according to a default trimming area defined in advance.
Alternatively, a process of setting the entire captured image as a transmission image for the client 120 without performing trimming may be performed. Thereafter, when a trimming area designation command including [trimming area designation information] indicating the trimming area determined based on the entire image is received from the client 120, trimming is performed according to [trimming area designation information] included in the command. .

The image data cut out by the trimming process in the trimming unit 106b of the imaging apparatus 100 is input to the encoding unit 103 via the multiplexer 102 in accordance with the dotted line route shown in FIG.
The encoding unit 103 executes an encoding process (ENCODE) of the trimmed image data. The amount of data is reduced by this encoding process.
The encoded image data whose data amount has been reduced is transmitted to the client 120 via the network 130 via the communication unit 111.

Another configuration of the imaging apparatus 100 illustrated in FIG. 5 will be described.
The control unit (processor) 103 includes a CPU having a program execution function, for example, and executes control according to a program recorded in advance in the storage unit 108 or the memory 110 to capture, correct, trim, and transmit an image. A series of processing executed by the imaging apparatus 100, such as command reception from the client 120 and command analysis, is controlled.

The storage unit 108 and the memory 110 are used for a program executed by the control unit 109, a work area for executing the program, a temporary recording area for a captured image, and the like.
The communication unit 111 performs data transmission / reception with the client 120 via the network 130, for example, transmission / reception of image data and commands to be transferred.

In the above description, the image captured by the imaging unit 101 has been described as being temporarily stored in the storage unit 108.
However, the captured image of the imaging unit 101 is input as it is to the image processing unit 106 via the multiplexer 105 in the preceding stage of the image processing unit 106, and distortion correction and trimming are executed in the image processing unit 106, and then the encoding unit 103. It is also possible to execute the encoding process and transmit to the client 120 via the communication unit 111.

  That is, the captured image is transmitted to the client 120 in almost real time. In this case as well, the trimming process in the trimming unit 106b of the image processing unit 106 can be executed according to [trimming area designation information] received from the client 120.

  That is, in the configuration shown in FIG. 5, there are two processing sequences (A) and (B) shown below as a sequence from image capturing in the imaging unit 101 to transmission of the captured image to the client 120.

(Processing sequence A)
(A1) Encoding the captured image in the encoding unit 103 (ENCODE),
(A2) Store the encoded image in the storage unit 108;
(A3) According to an instruction from the client 120, an image is read from the storage unit 108,
(A4) Decoding (DECODE) for an image read from the storage unit 108;
(A5) In the distortion correction unit 106a, distortion correction for the decoded image;
(A6) The trimming unit 106b trims the image after distortion correction according to [trimming area designation information] input from the client 120.
(A7) The encoding unit 103 encodes the trimmed image (ENCODE),
(A8) The communication unit 111 transmits the encoded image data to the client 120.

  In the processing sequence A, the captured image is once encoded and stored in the storage unit 108, and then the stored image in the storage unit 108 is read out in response to a request from the client 120 to perform decoding, distortion correction, trimming, and encoding. It is a processing sequence in the case of performing transmission and transmitting to the client 120.

(Processing sequence B)
(B1) In the distortion correction unit 106a, distortion correction for the captured image,
(B2) The trimming unit 106b trims the image after distortion correction according to [trimming area designation information] input from the client 120.
(B3) The encoding unit 103 encodes the trimmed image (ENCODE),
(B4) The communication unit 111 transmits the encoded image data to the client 120.

  The above processing sequence B is a processing sequence in a case where the captured image is transmitted to the client 120 after distortion correction, trimming, and encoding are performed in response to a request from the client 120 without being stored in the storage unit 108.

There are two processing sequences, A and B, when the image is transmitted from the imaging apparatus 100 to the client 120.
In any case, trimming is executed in accordance with [Trimming area designation information] included in the command received from the client 120, and the client 120 starts the client from the entire area of the image captured by the imaging unit 101 of the imaging apparatus 100. It is possible to reliably acquire an image of an arbitrary image area desired by the user.

Next, the configuration and processing of the client 120 will be described. The client 120 is configured by a mobile terminal such as a PC or a smart phone, for example.
As illustrated in FIG. 5, the client 120 includes a communication unit 121, a decryption unit 122, a display unit 123, a control unit 124, an input unit 125, and a storage unit 126.

Image data transmitted from the imaging apparatus 100 is received by the communication unit 121 of the client 120.
The image data received by the communication unit 121 is encoded image data, and the decoding unit 122 performs a decoding process (DECODE).
The decoded image is displayed on the display unit 123. Further, it is stored in the storage unit 126 as necessary.
The storage unit 126 is, for example, a non-volatile memory, and can store a recorded image regardless of whether the client 120 is turned on or off.

The control unit (processor) 124 includes a CPU having a program execution function, for example, and executes control according to a program recorded in advance in the storage unit 126 to receive, decode, display, and record an image, and further an imaging device Control of processing executed by the client 120, such as generation of commands to be output and transmission processing.
The input unit 125 is configured by a user operation unit or the like, and inputs instructions for various processes executed in the client 120 such as an image display instruction, an image read instruction from the storage unit 126, or a command transmission to the imaging apparatus 100. Done.

  In the system illustrated in FIG. 5, [trimming area designation information] set by the client 120 is transmitted to the imaging apparatus 100, and the imaging apparatus 100 performs trimming, which is image cropping processing, according to the [trimming area designation information] received from the client 120. Execute and provide the trimmed image to the client 120.

  With such a configuration, the client 120 can acquire an image by designating an arbitrary trimming area from the entire area of the captured image of the imaging apparatus 100, and can display, record, and save the image. That is, in this configuration, there is no problem that an image other than the fixed trimming area cannot be confirmed unlike the configuration of FIG. 1 described above.

[3. Processing Sequence in Image Distribution System of Present Disclosure]
Next, a processing sequence in the image distribution system of the present disclosure including the imaging device 100 described with reference to FIG. 5 and the client 120 will be described with reference to FIG.
The following two processing sequences will be described sequentially.
(1) Image capturing and recording processing sequence by imaging device (2) Transmission of captured image of imaging device to client and reproduction processing sequence

[3-1. Image capture and recording processing sequence with imaging device]
First, an image capturing and recording processing sequence by the imaging apparatus will be described.
FIG. 6 is a flowchart illustrating an image capturing / recording processing sequence performed by the imaging apparatus.
Note that the processing steps of this flow include processing executed by the imaging apparatus 100 shown in FIG. 5 and processing executed by the client 120.

Specifically, step S101 is a process executed by the client 120, and steps S102 to S105 are processes executed by the imaging apparatus 100.
Each of these processes is executed according to a program stored in a storage unit or a memory of each device under the control of the control unit 124 of the client 120 and the control unit 109 of the imaging apparatus 100.
Hereinafter, the processing of each step shown in the flow will be described sequentially.

(Step S101)
Step S101 is processing executed by the client 120 shown in FIG.
In step S <b> 101, the client 120 generates a shooting / recording process start command and transmits it to the imaging apparatus 100.
An example of the shooting / recording process start command is shown in FIG.

As shown in FIG. 7, the following information is set in the shooting / recording process start command.
(A) Command identification information (b) Recording time information

The command identification information is data for identifying the type of command. In this case, identification data indicating a shooting / recording process start command is set.
The recording time information is data indicating shooting recording start and end time information of shooting. Various settings are possible as the time information setting mode. For example,
A configuration that sets two pieces of time information, a shooting record start time and a shooting end time,
Configuration to set shooting record start time and shooting duration,
Configuration to set only the shooting end time,
For example, these settings are possible. In the configuration in which only the shooting end time is set, shooting recording is started with command reception as a trigger, and shooting recording is continued until the set shooting end time.

  In step S <b> 101, the client 120 generates a shooting / recording process start command having a data configuration illustrated in FIG. 7, for example, and transmits the command to the imaging apparatus 100.

(Step S102)
The next processing from step S102 to step S105 is processing executed by the imaging apparatus 100.
When the imaging apparatus 100 receives a shooting / recording start command having the data configuration shown in FIG. 7 from the client 120, in step S102, the imaging apparatus 100 starts up the imaging unit 101 and starts imaging processing.

(Step S103)
In step S103, the image capturing apparatus 100 inputs an image captured by the image capturing unit 101 to the encoding unit 103, and executes a captured image encoding process (ENCODE).

(Step S104)
Further, in step S <b> 104, the imaging apparatus 100 executes processing for storing the encoded image data generated by the encoding unit 103 in the storage unit 108 via the storage unit I / F 107.

(Step S105)
In step S105, the imaging apparatus 100 determines whether the end time of the shooting / recording process has been reached.
The imaging apparatus 100 refers to the recording time information set in the shooting / recording process start command received from the client 120 and determines whether or not the end time of the shooting / recording process has been reached. Although not shown in FIG. 5, the imaging device 100 has a clock function for acquiring time information. Alternatively, the time information distributed via the network 130 is acquired via the communication unit 111.

If it is determined in step S105 that the shooting and recording process end time has not come, the process returns to step S102, and the processes of steps S102 to S105 are repeated. That is, the photographing process, the encoding process, and the recording process are continuously repeated.
If it is determined in step S105 that the end time of the shooting / recording process has been reached, the shooting / recording process ends.

  In this way, the image capturing / recording process of the image by the imaging apparatus 100 is executed in accordance with the instruction from the client 120, that is, the image capturing / recording start command having the data configuration described with reference to FIG.

The flow illustrated in FIG. 6 is a flow that describes the processing of the client 120 and the imaging apparatus 100 along the time axis.
Next, communication processing executed between the client 120 and the imaging apparatus 100 in the shooting and recording processing described with reference to FIG. 6 will be described with reference to the sequence diagram shown in FIG.
Step numbers S101 to S104 shown in the sequence diagram shown in FIG. 8 correspond to the step numbers in the flowchart shown in FIG. That is, in the sequence diagram shown in FIG. 8, the same step numbers are set for the same processing steps as those in FIG.

Processing of each step in the sequence diagram shown in FIG. 8 will be described.
Step S <b> 101 is processing in which the client 120 generates a shooting / recording processing start command and transmits it to the imaging apparatus 100. It is transmitted via the network 130.
As described with reference to FIG.
(A) Command identification information (b) Recording time information This is a command in which each piece of information is set.

Subsequent processing steps S102 to S104 are executed on the imaging apparatus 100 side.
That is,
Step S102: Image photographing processing in the imaging unit 101,
Step S103: Encoding processing of the captured image in the encoding unit 103,
Step S104: recording processing of encoded image data to the storage unit 108,
These processes are executed.

The processes in steps S102 to S104 are continuously repeated until it is determined that the end time of the photographing / recording process has been reached.
Note that the process of determining whether or not the end time of the shooting / recording process has come is performed as a determination process based on the recording time information set in the shooting / recording process start command received from the client 120.

Although not shown in the flow of FIG. 5, step S <b> 106 is a notification message transmission process from the imaging apparatus 100 to the client 120 indicating the end of the shooting and recording process.
This notification process is an optional process and is not necessarily executed. It may be configured to execute in response to a request from the client side.
Specifically, an information field indicating whether or not an end notification is requested is added to the shooting / recording process start command described with reference to FIG.
The imaging apparatus refers to the setting of this information field, and when “end notification request = present” is set, notifies the client 120 of the end of the shooting / recording process as in step S106 shown in FIG. .

  In the setting of the sequence diagram illustrated in FIG. 8, the image capturing / recording process of the image by the image capturing apparatus 100 is a command transmission from the client 120 to the image capturing apparatus 100, that is, an image capturing / recording start command having the data configuration described with reference to FIG. The transmission is started by transmission, and is ended by transmission of an end notification from the imaging apparatus 100 to the client 120.

[3-2. Transmission of captured image of imaging device to client and playback processing sequence]
Next, transmission of a captured image of the imaging apparatus to the client and a reproduction processing sequence will be described.
FIG. 9 is a flowchart for explaining a transmission processing sequence and a transmission processing of a captured image of the imaging apparatus to the client.
Note that the processing steps of this flow include processing executed by the imaging apparatus 100 shown in FIG. 5 and processing executed by the client 120.

Specifically, the processes of step S201 and steps S207 to S213 are processes executed by the client 120, and the processes of steps S202 to S206 and step S214 are processes executed by the imaging apparatus 100.
Each of these processes is executed according to a program stored in a storage unit or a memory of each device under the control of the control unit 124 of the client 120 and the control unit 109 of the imaging apparatus 100.
Hereinafter, the processing of each step shown in the flow will be described sequentially.

(Step S201)
Step S201 is processing executed by the client 120 shown in FIG.
In step S <b> 201, the client 120 generates a playback process start command and transmits it to the imaging apparatus 100.
An example of the reproduction processing start command is shown in FIG.

As shown in FIG. 10, the following information is set in the playback process start command.
(A) Command identification information (b) Trimming area designation information

The command identification information is data for identifying the type of command. In this case, identification data indicating a playback process start command is set.
The trimming area designation information is data indicating a trimming area that is an image area cut out from a captured image of the imaging apparatus 100. This trimming area designation information is composed of, for example, coordinate position information of a captured image.

A specific example of input processing of trimming area designation information executed in the client 120 will be described with reference to FIG.
FIG. 11 shows a smartphone as an example of the client 120.
On the display unit of the client 120, a captured entire area image 210 captured by the imaging apparatus 100 is displayed.

Note that the captured entire area image 210 is a captured image acquired from the imaging apparatus 100 in advance. For example, an image acquired from the imaging apparatus immediately before the trimming area designation process, or an image stored in advance in the storage unit of the client 120 as a reference image for use in the trimming area designation can be used.
The entire shooting area image 210 is an image including the entire area of the shooting range of the imaging unit 101 of the imaging apparatus 100, and is an image subjected to distortion correction.

A user who operates the client 120 can display the entire photographic region image 210 on the display unit and set an arbitrary region as a trimming region.
For example, as shown in the figure, when the user's finger is brought into contact with the display unit which is a touch panel and the finger is slid from the start point A to the end point B, the start point A and the end point B as shown in the figure are apexes. A frame of the rectangular area (trimming area 220) is displayed.

As described above, the user can draw an arbitrary rectangular area on the entire photographic area image 210 by the finger slide process, and can determine the drawn rectangular area as the trimming area 220.
For example, the coordinate information of two vertices constituting the trimming area 220, which is a rectangular area drawn by the user, that is, the start point A and the end point B, is used as the configuration information of the trimming area designation information.
Starting point A (x1, y1)
End point B (x2, y2)
If these two coordinate positions are determined, the rectangular area shown in the figure, that is, the trimming area 220 can be defined.

  In step S201, the control unit 124 of the client 120 uses, for example, the two vertex coordinate information (x1, y1) and (x2, y2) shown in FIG. 11 in the field of the trimming area designation information of the reproduction processing command shown in FIG. The stored reproduction processing start command is generated and transmitted to the imaging apparatus 100 via the communication unit 121.

(Step S202)
The next processing from step S202 to step S206 is processing executed by the imaging apparatus 100.
When receiving the reproduction processing start command having the data configuration shown in FIG. 10 from the client 120, the imaging apparatus 100 reads a captured image from the storage unit 108 of the imaging apparatus 100 in step S202.
Here, the image read out from the storage unit 108 is, for example, an image that has been shot and recorded according to the flow of FIG. 5 described above.

(Step S203)
In step S <b> 203, the imaging apparatus 100 inputs an image read from the storage unit 108 to the decoding unit 104, and executes a recording image decoding process (DECODE).
The image previously stored in the storage unit 108 according to the flow of FIG. 5 is an encoded image, and the decoding process of the encoded image data is executed.

(Step S204)
Further, in step S204, the imaging apparatus 100 executes image processing on the image data decoded by the decoding unit 104.
That is, distortion correction is executed in the distortion correction unit 106a of the image processing unit 106 of the imaging apparatus 100 shown in FIG. 5, and trimming processing is executed in the trimming unit 106b.

  The distortion correction process in the distortion correction unit 106a is the distortion correction process described above with reference to FIG. That is, correction processing for removing image distortion included in a captured image using a wide-angle lens is executed.

The trimming process in the trimming unit 106b is the image cut-out process described above with reference to FIG.
In this trimming process, in order to determine which area of the image is to be cut out, that is, the trimming area, the trimming area designation information stored in the reproduction process start command received from the client 120 in step S201 is used.

In the trimming area designation information, as described above with reference to FIG. 11, two vertex coordinate information defining the trimming area, for example, the start point A (x1, y1) and the end point B (x2) shown in FIG. , Y2) are stored.
Using the two pieces of coordinate information, the trimming unit 106b sets a trimming region in the distortion correction image input from the distortion correction unit 106a, and executes a trimming process for cutting out the image of the set trimming region.
This process is the same as the image cutout process described above with reference to FIG.

(Step S205)
Next, in step S205, the imaging apparatus 100 inputs the trimmed image cut out by trimming (image cut-out processing) to the encoding unit 103, and executes the trimmed image encoding processing (ENCODE).

(Step S206)
Next, in step S206, the imaging apparatus 100 transmits the encoded trimmed image data generated by the encoding unit 103 to the client 120 via the communication unit 111.

(Step S207)
The next processing from step S207 to step S213 is processing executed by the client 120.
First, in step S207, the client 120 receives the encoded trimmed image data transmitted from the imaging apparatus 100 via the communication unit 121.

(Step S208)
Next, the client 120 inputs the received data to the decryption unit 122 and executes a decryption process (DECODE) of the received data.

(Step S209)
Next, the client 120 displays on the display unit 123 the image decoded by the decoding unit 122, that is, the trimmed image cut out according to the trimming area designation information designated by the client. Note that a process of recording an image in the storage unit 126 may be performed in accordance with an instruction from a user on the client side.

(Step S210)
Next, the client determines whether to end the image reproduction process. For example, when the reproduction of all the transmission images from the imaging apparatus 100 is completed, when a preset reproduction time is reached, or when a reproduction stop request is input by the user, it is determined that the reproduction process has ended and the process is performed. Exit.
If the reproduction process is not terminated, the process proceeds to step S211.

(Step S211)
In step S211, the client determines whether to change the currently set trimming area. Specifically, this is determination processing based on whether or not the user inputs a trimming area change request via the input unit 125 of the client 120 during image reproduction.

When a trimming area change request is input from the user during image reproduction, the determination in step S211 is Yes, and the process proceeds to step S212.
If no trimming area change request is input from the user during image reproduction, the determination in step S211 is No, the process returns to step S207, and the image reproduction processing in steps S207 to S210 is continued. In this case, the image area displayed on the display unit 123 of the client 120, that is, the trimming area is not changed.

(Step S212)
Step S212 is input processing of trimming area designation information in the client. This process is the same as the trimming area designation process described above with reference to FIG.
As described with reference to FIG. 11, the user displays the entire photographic area image 210 stored in advance in the storage unit of the client 120 on the display unit of the client 120, and slides by touching the finger on the display image. Thus, an arbitrary rectangular area is drawn on the entire photographic area image 210. The drawn rectangular area is determined as the trimming area 220.

Coordinate information of the two vertices constituting the trimming area 220, that is, the following start point A and end point B is trimming area designation information.
Starting point A (x1, y1)
End point B (x2, y2)

(Step S213)
In step S213, a trimming area designation command storing the vertex coordinate information (x1, y1) and (x2, y2) of the new trimming area determined in step S212 is generated and transmitted to the imaging apparatus 100.

An example of the trimming area designation command is shown in FIG.
As shown in FIG. 12, the following information is set in the trimming area designation command.
(A) Command identification information (b) Trimming area designation information

The command identification information is data for identifying the type of command. In this case, identification data indicating a trimming area designation command is set.
The trimming area designation information is data indicating a trimming area to be cut out from a captured image of the imaging apparatus 100. This trimming area designation information is the information designated in step S212. For example, the two vertices constituting the trimming area 220 input according to the process described with reference to FIG. 11, that is, the start point A (x1, y1). And coordinate information consisting of the end point B (x2, y2).

  In step S213, the client 120 generates a command storing the trimming area designation information, that is, a trimming area designation command having the data configuration shown in FIG.

(Step S214)
In step S <b> 214, the imaging apparatus 100 receives a trimming area designation command from the client 120.
Thereafter, the process proceeds to step S204, and trimming processing based on the new trimming area is executed according to the trimming area designation information stored in the trimming area designation command newly received from the client.

  As described above, in the process of the present disclosure, the trimming process executed by the imaging apparatus 100 is executed according to the trimming area designation information stored in the command transmitted from the client 120.

If the trimming area designation information is not stored in the playback process start command transmitted from the client 120 to the image capturing apparatus 100, it follows a predefined default trimming area, that is, a predefined default trimming area. Trimming process is executed.
Or it is good also as a structure which transmits the whole image of a picked-up image to a client, without performing a trimming process.

The flow shown in FIG. 9 is a flow explaining the processing of the client 120 and the imaging apparatus 100 along the time axis.
Next, communication processing executed between the client 120 and the imaging apparatus 100 during the image reproduction processing described with reference to FIG. 9 will be described with reference to sequence diagrams shown in FIGS. 13 and 14.
Step numbers S201 to S214 shown in the sequence diagrams shown in FIGS. 13 and 14 correspond to the step numbers in the flowchart shown in FIG. That is, in the sequence diagrams shown in FIGS. 13 and 14, the same step numbers are set for the same processing steps as those in FIG.

Processing of each step in the sequence diagrams shown in FIGS. 13 and 14 will be described.
Step S201 is processing in which the client 120 generates a playback process start command and transmits it to the imaging apparatus 100. The reproduction processing start command is transmitted via the network 130.

As described with reference to FIG.
(A) Command identification information (b) Trimming area designation information This is a command in which each piece of information is set.

Subsequent processing steps S202 to S205 are performed on the imaging device 100 side.
That is,
Step S202: image reading processing from the storage unit 108,
Step S203: Decoding processing of the read image in the decoding unit 104,
Step S204: distortion correction processing in the distortion correction unit 106a of the image correction unit 106, and trimming processing in the trimming unit 106b,
Step S205: Trimming image encoding processing in the encoding unit 103,
These processes are executed.

  Note that the setting of the trimming area in the trimming process executed in step S204 is performed based on the trimming area designation information received from the client.

The imaging apparatus 100 continuously and repeatedly executes the image transmission processing in steps S202 to S205 and step S206 until the image reproduction processing in the client ends.
Note that the image reproduction process ends based on the end of the read image from the storage unit 108 or the input of the reproduction stop from the client.
Steps S206 to S207 are transmission processing of encoded trimmed image data from the imaging apparatus 100 to the client 120.

In step S <b> 208, the client 120 inputs the reception data from the imaging apparatus 100 to the decoding unit 122 and executes a reception data decoding process (DECODE).
In step S209, the client 120 displays the image decoded by the decoding unit 122, that is, the trimmed image cut out according to the trimming area designation information designated by the client on the display unit 123.

The process after step S212 shown in FIG. 14 is a process when the determination of the trimming area change in step S211 of FIG. 9 is Yes, that is, when the trimming area change process is executed.
Step S212 is input processing of trimming area designation information in the client. As described with reference to FIG. 11, the coordinate information of the two vertices constituting the trimming area 220 of FIG. 11, that is, the following start point A and end point B, is trimming area designation information.
Starting point A (x1, y1)
End point B (x2, y2)

In steps S213 to S214, a trimming area designation command is generated and transmitted to the imaging apparatus 100.
The trimming area designation command is set with the following information as described with reference to FIG.
(A) Command identification information (b) Trimming area designation information

The processes in steps S204 to S209 below are the same as the processes described with reference to FIG.
However, the trimming process executed by the imaging apparatus 100 in step S204 is executed according to new trimming area designation information stored in the trimming area designation command transmitted from the client to the imaging apparatus in steps S213 to S214.

  In this way, the client 120 can change the trimming area during image reproduction. This trimming area changing process can be executed any number of times. That is, by sending a new trimming area setting command, the trimming area can be changed any number of times at an arbitrary timing.

As described above, in the configuration of the present disclosure, the image data transmitted from the imaging apparatus 100 to the client 120 is an image of a trimming area determined according to an instruction from the client.
As the trimming area, an arbitrary area can be designated from the entire area of the captured image on the client side.

In the imaging recording process, the imaging apparatus 100 stores the entire captured image in the storage unit 108 of the imaging apparatus 100 without performing trimming, as described with reference to the flow shown in FIG.
Thereafter, in response to a playback request from the client 120, the imaging apparatus 100 reads the image from the storage unit 108, executes trimming according to the trimming area designation information included in the command received from the client, and sends the trimming result to the client. Send.

  As described above, in the configuration of the present disclosure, the image of the entire captured image is stored in the storage unit 108 of the imaging apparatus 100, and the client makes a trimming request by designating an arbitrary region from the entire captured image region. Is possible. Therefore, an image area that cannot be confirmed does not occur as in the configuration described with reference to FIG. 1, and all areas can be confirmed.

[4. Processing sequence inside the imaging device during image recording, image playback, and live streaming]
Next, referring to the sequence diagrams shown in FIG. 15 and subsequent figures, a sequence of processing that is executed in each component in the imaging apparatus when the image recording process, the image reproduction process, and the live streaming process are executed. Will be described.
The sequence of each process below will be described.
(4-1) Image recording processing sequence (FIG. 15)
(4-2) Reproduction processing of recorded image without image processing (digital PTZ) such as trimming (FIG. 16)
(4-3) Reproduction processing of recorded image with image processing (digital PTZ) such as trimming (FIG. 17)
(4-4) Live streaming processing of captured image without image processing (digital PTZ) such as trimming (FIG. 18)
(4-5) Live streaming processing of captured image with image processing (digital PTZ) such as trimming (FIG. 19)
Digital PTZ (D-PTZ) means image processing executed by the image processing unit 106, such as trimming processing of an image captured by the imaging unit 101 of the imaging device 100.

[4-1. Image recording processing sequence]
First, an image capturing and recording processing sequence by the imaging apparatus will be described.
FIG. 15 is a sequence diagram illustrating an image capturing / recording processing sequence performed by the imaging apparatus 100.
In the sequence diagram shown in FIG.
A user operating the client 120, the client 120, and the imaging apparatus 100 are shown from the left.
Furthermore, regarding the imaging device 100, each component of the imaging device is shown individually. From the left. The communication unit 111, the control unit 109, the decoding unit 104, the storage unit 107, the encoding unit 103, the image processing unit 106, and the imaging unit 101. These correspond to the components of the imaging apparatus 100 described above with reference to FIG.

Processing of each step in the sequence diagram shown in FIG. 15 will be described.
(Steps S301 to S302)
In steps S301 to S302, the user inputs an image shooting and recording start request for the imaging apparatus 100 to the client 120 such as a smartphone, and the client 120 generates a shooting and recording process start command according to the user input information. This is a process of transmitting to the imaging device 100. The command is received by the communication unit 111 of the imaging apparatus 100 via the network 130 and notified to the control unit 109.
As described with reference to FIG.
(A) Command identification information (b) Recording time information This is a command in which each piece of information is set.

Subsequent processing steps S303 to S307 are executed on the imaging device 100 side.
(Step S303)
The control unit 109 outputs an operation start request to the imaging unit 101, the encoding unit 103, and the storage unit 107.

(Steps S304 to S307)
In response to an operation start request input from the control unit 109, the imaging unit 101 starts shooting processing and outputs a shot image to the encoding unit 103.
The encoding unit 103 executes compression (encoding) processing of the captured image input from the imaging unit 101, outputs the generated compressed image to the storage unit 107, and the storage unit 107 stores the compressed image.

The processes in steps S304 to S307 are repeatedly executed continuously until it is determined that the end time of the photographing / recording process has been reached.
Note that the process of determining whether or not the end time of the shooting / recording process has been reached is a determination process based on the recording time information set in the shooting / recording process start command received from the client 120.

[4-2. Reproduction processing of recorded images without image processing such as trimming (digital PTZ)]
Next, with reference to FIG. 16, a sequence of a recorded image reproduction process without image processing (digital PTZ) such as trimming will be described.

The sequence diagram shown in FIG. 16 also shows the user who operates the client 120 from the left, the client 120, and the imaging device 100, as in FIG.
Furthermore, regarding the imaging device 100, each component of the imaging device is shown individually. From the left. The communication unit 111, the control unit 109, the decoding unit 104, the storage unit 107, the encoding unit 103, the image processing unit 106, and the imaging unit 101. These correspond to the components of the imaging apparatus 100 described above with reference to FIG.

Processing in each step of the sequence diagram shown in FIG. 16 will be described.
(Steps S321 to S322)
In steps S321 to S322, a user inputs a reproduction start request for a captured image of the imaging apparatus 100 to a client 120 such as a smartphone, and the client 120 generates a reproduction processing start command in accordance with the user input information and performs imaging. This is processing to be transmitted to the device 100. The command is received by the communication unit 111 of the imaging apparatus 100 via the network 130 and notified to the control unit 109.
As described with reference to FIG.
(A) Command identification information (b) Trimming area designation information This is a command in which each piece of information is set.
However, the sequence diagram shown in FIG. 16 is an image reproduction request without trimming processing, and “(b) trimming area designation information” of the reproduction processing start command stores an invalid value, for example, all 0.

Subsequent processing steps S323 to S325 are executed on the imaging device 100 side.
(Steps S323 to 325)
In step S323, the control unit 109 outputs an operation start request to the storage unit 107.
In steps S 324 to S 325, an image is read from the storage unit 107, and the read image is transmitted to the client 120 via the communication unit 111. Note that the image read from the storage unit 107 and transmitted is a past photographed image and a compressed image.
The compressed image is transmitted to the client 120, and the client 120 performs decoding and reproduction.

[4-3. Reproduction processing of recorded image with image processing (digital PTZ) such as trimming]
Next, with reference to FIG. 17, a sequence of recorded image reproduction processing accompanied with image processing (digital PTZ) such as trimming will be described.

The sequence diagram shown in FIG. 17 also shows the user who operates the client 120 from the left, the client 120, and the imaging device 100, as in FIG.
Furthermore, regarding the imaging device 100, each component of the imaging device is shown individually. From the left. The communication unit 111, the control unit 109, the decoding unit 104, the storage unit 107, the encoding unit 103, the image processing unit 106, and the imaging unit 101. These correspond to the components of the imaging apparatus 100 described above with reference to FIG.

The processing of each step in the sequence diagram shown in FIG. 17 will be described.
(Steps S341 to S342)
In steps S341 to S342, the user inputs a reproduction start request accompanied with image processing (digital PTZ) such as trimming of a captured image of the imaging apparatus 100 to the client 120 such as a smartphone, and the client 120 uses the user input information as the user input information. In response to this, a reproduction process start command is generated and transmitted to the imaging apparatus 100. The command is received by the communication unit 111 of the imaging apparatus 100 via the network 130 and notified to the control unit 109.
As described with reference to FIG.
(A) Command identification information (b) Trimming area designation information This is a command in which each piece of information is set.
The sequence diagram shown in FIG. 17 is an image reproduction request accompanied by trimming processing, and the designated region information of the image is stored in “(b) Trimming region designation information” of the reproduction processing start command. The specification information is initially set to a default initial value, and can be changed at any time during reproduction.

Subsequent processing steps S343 to S351 are executed on the imaging device 100 side.
(Steps S343 to 346)
In step S343, the control unit 109 outputs an operation start request to the image processing unit 106, the encoding unit 103, the storage unit 107, and the decoding unit 104.
The image read from the storage unit 107 in step S344 is decoded by the decoding unit 104 in step S345, and is output to the image processing unit 106 in step S346.

(Steps S347 to 349)
In step S347, the image processing unit 106 performs image processing such as image trimming processing. Trimming is executed in accordance with “(b) Trimming area designation information” of the reproduction processing start command.
An image after predetermined image processing such as trimming is defined as a digital PTZ (D-PTZ) image. The digital PTZ (D-PTZ) image generated by the image processing unit 106 is input to the encoding unit 103 in step S348 and compressed (encoded) in step S349.

(Steps S350-351)
In steps S350 to S351, the digital PTZ (D-PTZ) compressed image is transmitted to the client 120 via the communication unit 111, and the client 120 performs decoding / reproduction processing.

[4-4. Live streaming processing of captured images without image processing (digital PTZ) such as trimming]
Next, with reference to FIG. 18, a sequence of live streaming processing of a captured image without image processing (digital PTZ) such as trimming will be described.

The sequence diagram shown in FIG. 18 also shows the user who operates the client 120 from the left, the client 120, and the imaging device 100, as in FIG.
Furthermore, regarding the imaging device 100, each component of the imaging device is shown individually. From the left. The communication unit 111, the control unit 109, the decoding unit 104, the storage unit 107, the encoding unit 103, the image processing unit 106, and the imaging unit 101. These correspond to the components of the imaging apparatus 100 described above with reference to FIG.

The processing of each step in the sequence diagram shown in FIG. 18 will be described.
(Steps S401 to S402)
In steps S401 to S402, the user inputs a live viewing start request for a captured image of the imaging apparatus 100 to the client 120 such as a smartphone, and the client 120 generates a live viewing processing start command according to the user input information. This is a process of transmitting to the imaging device 100. The command is received by the communication unit 111 of the imaging apparatus 100 via the network 130 and notified to the control unit 109.
The live viewing start command has the same configuration as the playback processing start command described above with reference to FIG.
(A) Command identification information (b) Trimming area designation information This is a command in which each piece of information is set.
However, the sequence diagram shown in FIG. 18 is an image live streaming request without trimming processing, and “(b) trimming area designation information” of the command stores an invalid value, for example, all 0s.

Subsequent processing steps S403 to S408 are executed on the imaging apparatus 100 side.
(Steps S403 to 405)
In step S <b> 403, the control unit 109 outputs an operation start request to the imaging unit 101 and the encoding unit 103.
In step S <b> 404, the imaging unit 101 starts shooting processing, and in step S <b> 405, the captured image is output to the encoding unit 103.

(Steps S406 to 408)
In step S406, the encoding unit 103 compresses (encodes) the captured image input from the imaging unit 101, and generates a compressed image.
In steps S <b> 407 to S <b> 408, the compressed image generated by the encoding unit 103 is transmitted to the client 120 via the communication unit 111.
The compressed image is transmitted to the client 120, and the client 120 performs decoding and reproduction.

[4-5. Live streaming processing of captured images with image processing (digital PTZ) such as trimming]
Next, with reference to FIG. 19, a sequence of live streaming processing of a captured image accompanied by image processing (digital PTZ) such as trimming will be described.

The sequence diagram shown in FIG. 19 also shows the user who operates the client 120 from the left, the client 120, and the imaging device 100, as in FIG.
Furthermore, regarding the imaging device 100, each component of the imaging device is shown individually. From the left. The communication unit 111, the control unit 109, the decoding unit 104, the storage unit 107, the encoding unit 103, the image processing unit 106, and the imaging unit 101. These correspond to the components of the imaging apparatus 100 described above with reference to FIG.

The process of each step in the sequence diagram shown in FIG. 19 will be described.
(Steps S421 to S422)
In steps S421 to S422, the user inputs a live streaming viewing request accompanied by image processing (digital PTZ) such as trimming of a captured image of the imaging apparatus 100 to the client 120 such as a smartphone, and the client 120 receives the user input information. In response, the live viewing process start command is generated and transmitted to the imaging apparatus 100. The command is received by the communication unit 111 of the imaging apparatus 100 via the network 130 and notified to the control unit 109.
The live viewing start command has the same configuration as the playback processing start command described above with reference to FIG.
(A) Command identification information (b) Trimming area designation information This is a command in which each piece of information is set.
The sequence diagram shown in FIG. 19 is a live streaming request for an image accompanied by a trimming process, and valid image region designation information is stored in the command “(b) Trimming region designation information”. The specification information is initially set to a default initial value, and can be changed at any time during reproduction.

Subsequent processing steps S423 to S430 are executed on the imaging device 100 side.
(Steps S423 to 426)
In step S423, the control unit 109 outputs an operation start request to the imaging unit 101, the image processing unit 106, and the encoding unit 103.
In step S <b> 42, the imaging unit 101 starts shooting processing, and in step S <b> 425, the captured image is output to the image processing unit 106.

(Steps S426 to 428)
In step S426, the image processing unit 106 performs image processing such as image trimming processing. Trimming is executed in accordance with “(b) Trimming area designation information” of the reproduction processing start command.
An image after predetermined image processing such as trimming is defined as a digital PTZ (D-PTZ) image. The digital PTZ (D-PTZ) image generated by the image processing unit 106 is input to the encoding unit 103 in step S427 and compressed (encoded) in step S428.

(Steps S429 to 430)
In steps S429 to S430, the digital PTZ (D-PTZ) compressed image is transmitted to the client 120 via the communication unit 111, and the decoding reproduction process is performed in the client 120.

[5. Summary of composition of the present disclosure]
As described above, the embodiments of the present disclosure have been described in detail with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present disclosure. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present disclosure, the claims should be taken into consideration.

The technology disclosed in this specification can take the following configurations.
(1) an imaging unit;
An image processing unit that performs image processing on a captured image of the imaging unit;
A communication unit that transmits the processed image generated by the image processing unit to the client;
The image processing unit
An imaging apparatus that performs a trimming process for cutting out a partial area from the captured image, and determines an image cut-out area in the trimming process according to trimming area designation information received from the client.

  (2) The imaging apparatus includes a storage unit that stores a captured image of the imaging unit, the storage unit stores image data of the entire region of the captured image before trimming processing, and the image processing unit The imaging apparatus according to (1), wherein trimming processing is performed on an image stored in the storage unit in accordance with a request from a client.

  (3) The communication unit receives a processing request command from the client, and the image processing unit determines a trimming region according to the trimming region designation information included in the processing request command. The imaging device described.

  (4) The imaging apparatus according to (3), wherein the processing request command is a reproduction start command transmitted to the imaging apparatus when the client reproduces a captured image.

  (5) The imaging apparatus according to (3), wherein the processing request command is a trimming area designation command that is transmitted to the imaging apparatus when the client designates a trimming area.

  (6) The image processing unit includes a distortion correction unit that performs distortion correction of the captured image, and a trimming unit that performs a trimming process of cutting out a partial region from the distortion correction image generated by the distortion correction unit. ) To (5).

  (7) The imaging unit includes a wide-angle lens and is configured to capture a wide-area image around the imaging unit via the wide-angle lens, and the distortion correction unit is included in an image captured via the wide-angle lens. The imaging apparatus according to (6), wherein correction is performed to remove distortion.

  (8) The imaging apparatus includes an encoding unit that encodes the processed image generated by the image processing unit, and the communication unit transmits the encoded image generated by the encoding unit to the client ( The imaging device according to any one of 1) to (7).

  (9) The imaging device includes an encoding unit that performs an encoding process of a captured image of the imaging unit to generate an encoded image, a storage unit that stores the encoded image generated by the encoding unit, A decoding unit that decodes the encoded image stored in the storage unit to generate a decoded image; and the image processing unit executes image processing on the decoded image generated by the decoding unit. (8) The imaging device according to any one of the above.

  (10) The imaging device according to (9), wherein the decoding process in the decoding unit and the image process in the image processing unit are executed in accordance with a reproduction process start command received from the client.

  (11) The imaging apparatus according to (9), wherein the encoding process of the captured image and the storage process of the storage unit in the encoding unit are executed in accordance with a shooting record start command received from the client.

  (12) The imaging device according to any one of (1) to (11), wherein the imaging device is a surveillance camera.

  (13) The imaging device according to any one of (1) to (12), wherein the imaging device is a camera used for a security system.

(14) An image distribution system having an imaging device and a client,
The client
Trimming area designation information that is designation information of an image cut-out area from a captured image of the imaging apparatus is transmitted to the imaging apparatus,
The imaging device
An imaging unit;
An image processing unit that performs image processing on a captured image of the imaging unit;
A communication unit that transmits the processed image generated by the image processing unit to the client;
The image processing unit
An image distribution system for determining an image cutout area from the photographed image according to trimming area designation information received from the client and executing a trimming process.

(15) An image processing method executed in the imaging apparatus,
The image processing unit executes an image processing step of performing image processing on the captured image of the imaging unit,
In the image processing step,
Perform a trimming process to cut out a partial area from the captured image,
An image processing method in which, in the trimming process, an image cut-out area is determined according to trimming area designation information received from the client.

(16) A program for executing image processing in the imaging apparatus,
Let the image processing unit execute image processing on the captured image of the imaging unit,
In the image processing,
A trimming process for cutting out a partial area from the captured image is executed,
In the trimming process, a program for determining an image cutout area according to trimming area designation information received from the client.

  The series of processing described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run. For example, the program can be recorded in advance on a recording medium. In addition to being installed on a computer from a recording medium, the program can be received via a network such as a LAN (Local Area Network) or the Internet and can be installed on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

As described above, according to the configuration of an embodiment of the present disclosure, an apparatus and a method for executing trimming processing in an imaging apparatus such as a monitoring camera based on an instruction from a client are realized.
Specifically, the imaging unit, a storage unit that stores a captured image of the imaging unit, an image processing unit that performs image processing on the stored image of the storage unit, and a processed image generated by the image processing unit are transmitted to the client. The imaging apparatus includes a communication unit, and the image processing unit executes a trimming process of a captured image. The image processing unit determines an image clipping region according to trimming region designation information received from the client. The image stored in the storage unit is an image before trimming, and the client can designate an arbitrary area as the trimming area.
With this configuration, an apparatus and a method for executing trimming processing in an imaging apparatus such as a monitoring camera based on an instruction from a client are realized.

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Imaging part 12 Image processing part 12a Distortion correction part 12b Trimming part 13 Control part (processor)
14 Encoding unit 15 Memory 16 Communication unit 20 Client (PC etc.)
21 Communication Unit 22 Decoding Unit 23 Display Unit 24 Storage Unit 25 Control Unit (Processor)
26 Input Unit 30 Network 100 Imaging Device 101 Imaging Unit 102 Multiplexer (MUX)
103 Coding section 104 Decoding section 105 Multiplexer (MUX)
106 Image processing unit 106a Distortion correction unit 106b Trimming unit 107 Control unit (processor)
108 Memory 109 Memory I / F
110 Storage unit 111 Communication unit 120 Client (PC etc.)
121 communication unit 122 decoding unit 123 display unit 124 control unit (processor)
125 Input unit 126 Storage unit 130 Network

Claims (16)

An imaging unit;
An image processing unit that performs image processing on a captured image of the imaging unit;
A communication unit that transmits the processed image generated by the image processing unit to the client;
The image processing unit
An imaging apparatus that performs a trimming process for cutting out a partial area from the captured image, and determines an image cut-out area in the trimming process according to trimming area designation information received from the client. The imaging device
A storage unit for storing a captured image of the imaging unit;
The storage unit stores image data of the entire captured image area before trimming processing,
The imaging apparatus according to claim 1, wherein the image processing unit performs a trimming process on an image stored in the storage unit in accordance with a request from the client. The communication unit receives a processing request command from the client,
The imaging apparatus according to claim 1, wherein the image processing unit determines a trimming region according to trimming region designation information included in the processing request command.   The imaging apparatus according to claim 3, wherein the processing request command is a reproduction start command that is transmitted to the imaging apparatus when the client reproduces a captured image.   The imaging apparatus according to claim 3, wherein the processing request command is a trimming area designation command that is transmitted to the imaging apparatus when the client designates a trimming area.   The image processing unit according to claim 1, further comprising: a distortion correction unit that performs distortion correction of the captured image; and a trimming unit that performs a trimming process of cutting out a partial region from the distortion correction image generated by the distortion correction unit. Imaging device. The imaging unit includes a wide-angle lens, and is configured to capture a wide-area image around the imaging unit via the wide-angle lens.
The imaging apparatus according to claim 6, wherein the distortion correction unit performs correction for removing distortion included in an image captured through the wide-angle lens. The imaging device
An encoding unit that encodes the processed image generated by the image processing unit;
The imaging device according to claim 1, wherein the communication unit transmits the encoded image generated by the encoding unit to a client. The imaging device
An encoding unit that executes an encoding process of a captured image of the imaging unit to generate an encoded image;
A storage unit for storing the encoded image generated by the encoding unit;
A decoding unit that generates a decoded image by decoding the encoded image stored in the storage unit;
The imaging apparatus according to claim 1, wherein the image processing unit executes image processing on the decoded image generated by the decoding unit.   The imaging apparatus according to claim 9, wherein the decoding processing in the decoding unit and the image processing in the image processing unit are executed according to a reproduction processing start command received from the client.   The imaging apparatus according to claim 9, wherein the encoding process of the captured image and the storage process of the storage unit in the encoding unit are executed in response to a recording start command received from the client.   The imaging apparatus according to claim 1, wherein the imaging apparatus is a surveillance camera.   The imaging apparatus according to claim 1, wherein the imaging apparatus is a camera used for a security system. An image distribution system having an imaging device and a client,
The client
Trimming area designation information that is designation information of an image cut-out area from a captured image of the imaging apparatus is transmitted to the imaging apparatus,
The imaging device
An imaging unit;
An image processing unit that performs image processing on a captured image of the imaging unit;
A communication unit that transmits the processed image generated by the image processing unit to the client;
The image processing unit
An image distribution system for determining an image cutout area from the photographed image according to trimming area designation information received from the client and executing a trimming process. An image processing method executed in the imaging apparatus,
The image processing unit executes an image processing step of performing image processing on the captured image of the imaging unit,
In the image processing step,
Perform a trimming process to cut out a partial area from the captured image,
An image processing method in which, in the trimming process, an image cut-out area is determined according to trimming area designation information received from the client. A program for executing image processing in an imaging apparatus;
Let the image processing unit execute image processing on the captured image of the imaging unit,
In the image processing,
A trimming process for cutting out a partial area from the captured image is executed,
In the trimming process, a program for determining an image cutout area according to trimming area designation information received from the client.

Images