TWI752465B - Image processing system, image processing method, and program - Google Patents

Abstract

An object of the present invention is to provide an image processing system, an image processing method, and a program capable of suppressing image distortion regardless of the posture of the person being photographed. The image processing system (12) of the present invention includes: an image acquisition unit (121) that acquires image data of a subject captured by the camera unit (11) of the lobby walkie-talkie (1); and an image processing unit (122), which performs image processing on the captured image. An image processing unit (122) performs viewpoint conversion distortion correction in which the viewpoint of the captured image is changed by a conversion angle (θb) with respect to the subject by deforming the captured image. The conversion angle (θb) is greater than 0 degrees and smaller than the angle of the lobby intercom (1) relative to the vertical plane, that is, the inclination angle θ1.

Description

Image processing system, image processing method and program

The present disclosure generally relates to an image processing system, an image processing method, and a program, and more particularly, relates to an image processing system, an image processing method, and a program for performing image processing on a captured image of a walkie-talkie device.

Conventionally, there has been an image conversion device used for a surveillance system or the like for crime prevention (for example, refer to Patent Document 1 (Japanese Patent Laid-Open No. 2010-224691 )). The image conversion device described in Patent Document 1 performs processing of converting a distorted circular image obtained from a camera using a fisheye lens into a regular image from above with less distortion.

A walkie-talkie device having an imaging section such as the camera of Patent Document 1 captures a visitor or the like to generate a captured image. In recent years, the walkie-talkie device installed in the lobby of a condominium or the like has been installed on the upper surface of a stage or the like in an inclined state in consideration of the convenience of visitors. That is, the front surface of the walkie-talkie device in which the operation part, the screen, the imaging part, etc. are arranged is inclined with respect to the vertical plane, and the imaging part images the person from the obliquely downward direction. Therefore, the imaging unit faces the person who is leaning forward in order to operate the walkie-talkie device, and the person can be photographed from the front. On the other hand, the camera section is directed towards the person standing upright in front of the intercom device, and the person standing upright is photographed obliquely.

When the imaging unit captures a person leaning forward obliquely from below, the image conversion device of Patent Document 1 can generate a regular top-view image with less distortion. However, when the image pickup unit captures an upright person from obliquely below, the amount of distortion of the top-view regular image generated by the image conversion device increases.

The present disclosure is made in view of the above-mentioned problems, and an object of the present disclosure is to provide an image processing system, an image processing method and a program which can suppress image distortion regardless of the posture of the person being photographed.

An image processing system according to an aspect of the present disclosure includes: an image acquisition unit that acquires data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; and an image processing unit that acquires data on the captured image Implement image processing. The image processing unit performs viewpoint conversion distortion correction in which the viewpoint of the captured image is changed at a conversion angle with respect to the subject by deforming the captured image. The above-mentioned conversion angle is greater than 0 degrees, and is smaller than the angle of the above-mentioned interphone device relative to the vertical plane, that is, the inclination angle.

An image processing system of one aspect of the present disclosure includes: an image acquisition unit that acquires data of a captured image captured by an imaging unit of a walkie-talkie device; an image processing unit that performs image processing on the captured image; and an information acquisition section, which acquires input information. The image processing unit corrects the distortion of the captured image based on the input information.

An image processing method of one aspect of the present disclosure includes: an image acquisition step of acquiring data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; and an image processing step of obtaining the captured image Implement image processing. The image processing step performs viewpoint conversion distortion correction for changing the viewpoint of the captured image with respect to the subject at a conversion angle by deforming the captured image. The above-mentioned conversion angle is greater than 0 degrees, and is smaller than the angle of the above-mentioned interphone device relative to the vertical plane, that is, the inclination angle.

An image processing method of one aspect of the present disclosure includes: an image acquisition step of acquiring data of an image captured by an imaging unit of a walkie-talkie device; an information acquisition step of acquiring input information; and an image processing step of Image processing is performed on the above-mentioned captured image. The above-mentioned image processing step corrects the distortion of the above-mentioned captured image based on the above-mentioned input information.

A program of one aspect of the present disclosure is a program for causing a computer to execute the above-mentioned image processing method.

Hereinafter, embodiments of the present disclosure will be described. The following embodiment is only one of various embodiments of the present disclosure. In addition, in the following embodiment, as long as the object of cost disclosure can be achieved, various changes can be made according to the design and the like.

(first embodiment) (1) Outline of the walkie-talkie system First, the outline of the walkie-talkie system 10 according to the first embodiment will be described with reference to FIG. 1 .

As shown in FIG. 1 , the walkie-talkie system 10 according to the first embodiment includes a hall walkie-talkie (walkie-talkie device) 1 , a master walkie-talkie 2 , and a control device 3 . The walkie-talkie system 10 is a walkie-talkie system suitable for use in facilities such as condominiums, commercial facilities, offices, or shops, for example. In the first embodiment, a walkie-talkie system for complex housing is exemplified.

The walkie-talkie master unit 2 is, for example, an indoor master unit installed in a house (indoor) of a complex house. The lobby walkie-talkie 1 is, for example, a walkie-talkie sub-unit installed in a common space (lobby) of a condominium. The walkie-talkie main unit 2 and the walkie-talkie slave unit 1 are configured to be able to communicate with each other via the control device 3 . Thereby, in the walkie-talkie system 10 of the first embodiment, the communication between the walkie-talkie main unit 2 and the walkie-talkie slave unit 1 is possible.

(2) Details of the walkie-talkie system Hereinafter, the details of the walkie-talkie system 10 according to the first embodiment will be described.

(2-1) The composition of the walkie-talkie in the lobby First, the configuration of the walkie-talkie 1 will be described. The lobby walkie-talkie 1 corresponds to the walkie-talkie device of the present disclosure. The lobby walkie-talkie 1 includes, as shown in FIG. 1 , an imaging unit 11 and an image processing device 120 as an image processing system 12 . The image processing device 120 performs image processing on an image (captured image) captured by the imaging unit 11 . The lobby walkie-talkie 1 further includes a slave control unit 13 , a slave communication unit 14 , a slave operation unit 15 , and a slave communication unit 16 .

The imaging unit 11 is a camera that has an imaging element 111 and a lens 112 and is used to photograph a subject such as the upper body of a visitor. In the first embodiment, the imaging unit 11 is assumed to be a camera that captures moving images, but the imaging unit 11 may be a camera (still camera) that captures still images. In the first embodiment, the imaging unit 11 is assumed to be a color camera that captures color images, but the imaging unit 11 may be a monochrome camera that captures monochrome images.

The imaging element 111 is a two-dimensional image sensor such as a CCD (Charge Coupled Devices: Charge Coupled Device) image sensor, or a CMOS (Complementary Metal-Oxide Semiconductor: Complementary Metal Oxide Semiconductor) image sensor. The imaging unit 11 forms an image of the light from the object on the imaging surface (light-receiving surface) of the imaging element 111 through the lens 112 , and converts the light from the object into an electrical signal (imaging signal) by the imaging element 111 . The image (photographed image) data captured by the imaging unit 11 is output from the imaging unit 11 as an imaging signal. Here, the imaging unit 11 is configured to generate a digital imaging signal in the YUV format from an output signal of the imaging element 111 and output the digital imaging signal. The lens 112 is, for example, a fish-eye lens (super wide-angle lens) having a viewing angle of 180 degrees or more.

The image processing device 120 acquires the data (image pickup signal) of the captured image from the imaging unit 11, and performs distortion correction processing to be described later on the captured image. The image processing device 120 sets the captured image subjected to distortion correction processing as a corrected image, and generates corrected image data. The image processing device 120 is implemented by, for example, an ISP (Image Signal Processor) or a PFGA (Field-Programmable Gate Array) or the like.

The slave control unit 13 includes, for example, a microcomputer having a processor and a memory. Then, when the processor executes the program stored in the memory, the microcomputer functions as the slave control unit 13 . Here, the program executed by the processor is pre-recorded in the memory of the microcomputer, and can also be provided by being recorded in a non-transitory recording medium such as a memory card, or provided through an electrical communication line such as the Internet.

The slave control unit 13 is electrically connected to the slave communication unit 14 , the slave operation unit 15 , and the slave communication unit 16 , and can monitor and control the slave communication unit 14 , the slave operation unit 15 , and the slave communication unit 16 of each action.

The slave communication unit 14 is a communication interface for communicating with the main unit 2 of the walkie-talkie. The slave communication unit 14 is electrically connected to the control device 3 via the transmission path W1. The control device 3 is further connected to one or more walkie-talkie master units 2 via the transmission path W2. The transmission paths W1 and W2 are two-wire transmission paths including, for example, twisted pair wires. In other words, the slave communication unit 14 is electrically connected to one or more master walkie-talkies 2 via the control device 3 . The slave communication unit 14 transmits the corrected image signal including the corrected image data generated by the image processing device 120 to the main unit 2 of the walkie-talkie. Furthermore, in order to transmit audio signals and control signals bidirectionally between the lobby intercom 1 and the main intercom 2 , the slave communication unit 14 is configured to be capable of bidirectional communication with the main intercom 2 .

The slave operation unit 15 accepts various operations. Various operations include operations by visitors, operations by builders, and operations by administrators. For example, the slave operation unit 15 accepts an operation for calling (call operation) from a visitor. The call operation is the operation of inputting or selecting the household number, etc. (for example, key operation). When a call operation is performed on the slave operation unit 15 , the slave control unit 13 transmits a control signal (calling signal) for calling a resident in the house from the slave communication unit 14 to the walkie-talkie main unit 2 . In addition, when the lobby walkie-talkie 1 is provided with a touch panel display, the touch panel display can also serve as the function of the slave operation unit 15 .

The handset communication unit 16 includes a speaker and a microphone, and the microphone collects the voice of the visitor and emits the voice of the occupant from the speaker. The slave control unit 13 transmits a voice signal including the voice information of the visitor from the slave communication unit 14 to the main unit 2 of the walkie-talkie. The slave control unit 13 outputs a voice signal including the voice information of the resident received by the slave communication unit 14 to the slave communication unit 16, and emits the voice of the resident from the speaker.

(2-2) The composition of the main unit of the walkie-talkie Next, the configuration of the main unit 2 of the walkie-talkie will be described. The main unit 2 of the walkie-talkie includes a main unit control unit 21 , a main unit communication unit 22 , a display unit 23 , a main unit operation unit 24 , and a main unit communication unit 25 as shown in FIG. 2 .

The host computer control unit 21 includes, for example, a microcomputer having a processor and a memory. Then, when the processor executes the program stored in the memory, the microcomputer functions as the master control unit 21 . Here, the program executed by the processor is pre-recorded in the memory of the microcomputer, and can also be provided by being recorded in a non-transitory recording medium such as a memory card, or provided through an electrical communication line such as the Internet.

The main unit control unit 21 is electrically connected to the main unit communication unit 22, the display unit 23, the main unit operation unit 24, and the main unit communication unit 25, and can monitor and control the main unit communication unit 22, the display unit 23, the main unit operation unit 24, and the main unit communication unit 25 of the actions.

The master communication unit 22 is a communication interface for communicating with the lobby intercom 1 . The master communication unit 22 is electrically connected to the lobby intercom 1 via the control device 3 . The master communication unit 22 is configured to be capable of bidirectional communication with the lobby walkie-talkie 1 in order to enable two-way transmission of audio signals and control signals between the walkie-talkie master 2 and the hall walkie-talkie 1 .

The display unit 23 displays an image based on the corrected image signal transmitted from the lobby intercom 1 . In the first embodiment, the main unit 2 of the walkie-talkie is provided with a touch panel display. Therefore, the touch panel display functions as the display unit 23 and the main unit operation unit 24 . The parent machine control unit 21 causes the display unit 23 to display the corrected image based on the corrected image data received by the parent machine communication unit 22 .

The main machine operation unit 24 accepts operations by the occupants of the dwelling. When a specific operation (such as a key operation) is performed on the main unit operating unit 24 while the main unit communication unit 22 is receiving a call signal from the lobby intercom 1, the interphone main unit 2 and the lobby intercom 1 can be in a state of being able to communicate.

The main unit communication unit 25 includes a speaker and a microphone, and the microphone collects the voice of the resident and emits various audio frequencies such as the voice of the visitor from the speaker. When the main unit control unit 21 receives the call signal and the corrected image signal from the lobby intercom 1 via the main unit communication unit 22 , it outputs the sound data of the calling sound to the speaker of the main unit communication unit 25 , and outputs the corrected image data to the display unit 23 . Thereby, the main unit control unit 21 causes the speaker of the main unit communication unit 25 to output a calling sound, and causes the display unit 23 to display the corrected image. Furthermore, if the main unit 2 of the walkie-talkie and the lobby walkie-talkie 1 are in a callable state, the master unit control unit 21 transmits a voice signal including the information of the occupant's voice from the master unit communication unit 22 to the hall walkie-talkie 1 . Furthermore, the parent machine control unit 21 outputs a voice signal including the voice information of the visitor received by the parent machine communication unit 22 to the parent machine communication unit 25, and causes the visitor's voice to be emitted from the speaker.

(2-3) Control device When the control device 3 receives a control signal (calling signal) from the lobby walkie-talkie 1, it transmits it to the calling walkie-talkie master 2. When the control device 3 receives the corrected image signal from the hall walkie-talkie 1, it sends it to the master walkie-talkie 2 of the transmission destination. The control device 3 transmits and receives the audio data of the visitor and the audio data of the occupant when the intercom 1 and the main intercom 2 in the lobby are in a state of being able to communicate. Furthermore, the control device 3 performs locking and unlocking of, for example, a door installed in a hall according to the operation of the intercom master unit 2 .

(2-4) Setting of the walkie-talkie in the lobby FIG. 3 is a schematic diagram showing an installation example of the lobby intercom 1 according to the first embodiment. The lobby intercom 1 is installed in the common space 5 of the collective residence.

The lobby walkie-talkie 1 further includes a casing 100 . The casing 100 is in the shape of a thin rectangular parallelepiped. Inside the casing 100 , various components of the hall walkie-talkie 1 including the imaging unit 11 and the image processing device 120 are accommodated. On the surface (front surface) of the casing 100 , the imaging unit 11 , the handset operation unit 15 , and the handset communication unit 16 are arranged.

In the common space 5 of the complex house, an installation stand 4 formed in a rectangular parallelepiped shape long in the vertical direction (vertical direction) is installed. The mounting table 4 has a mounting surface 41 on which the hall walkie-talkie 1 is mounted. The mounting surface 41 is inclined with respect to the vertical plane, and the inclination angle is θ1. Therefore, in the state where the lobby intercom 1 is mounted on the mounting surface 41, the lobby intercom 1 (the casing 100) is also inclined at the inclination angle θ1 with respect to the vertical plane. That is, the imaging unit 11 , the handset operation unit 15 , and the handset communication unit 16 face the visitor 6 from diagonally below in front of the visitor 6 .

By arranging the hall walkie-talkie 1 obliquely, it is easy for the visitor 6 to view the sub-machine operation part 15 disposed on the surface of the casing 100 and to easily operate the sub-machine operation part 15 .

The imaging unit 11 is a camera for photographing the visitor 6 of the residents who visit the complex. The imaging unit 11 is located in front of the visitor 6 who operates the hall walkie-talkie 1 . In a state where the lobby walkie-talkie 1 is mounted on the mounting table 4 , the imaging unit 11 photographs the visitor 6 from obliquely below. When the visitor 6 operates the lobby intercom 1, at least the face of the visitor 6 is reflected in the image, and the imaging area A1 (field of view) of the imaging unit 11 is set in front of the lobby intercom 1.

(2-5) Image processing device (image processing system) Hereinafter, the distortion correction processing of the image processing device 120 (image processing system 12 ) will be described. The image processing device 120 includes, as shown in FIG. 1 , an image acquisition unit 121 , an image processing unit 122 , a memory unit 123 , a mode switching unit 124 , a degree selection unit 125 , and an information acquisition unit 126 .

The image acquisition unit 121 acquires data (imaging signals) of the captured image from the imaging unit 11 . The image processing unit 122 performs distortion correction processing on the captured image.

In the first embodiment, since the lens 112 is a fish-eye lens, lens distortion of the lens 112 occurs in the captured image. Lens distortion is distortion that expands from the center of the image to the outside (radiation direction), and the distortion is greater on the outside than on the inside of the image. For example, if the imaging unit 11 captures the grid-shaped graph C1 in FIG. 4A from the front, as shown in FIG. 4B in the captured image G11 , the grid-shaped graph C1 reflected in the grid-shaped graph C1 is expanded and distorted as a grid pattern D1 as shown in FIG. 4B . . A plurality of horizontal lines (lines extending in the horizontal direction in the common space 5 ) C11 and a plurality of vertical lines (lines extending in the vertical direction in the common space 5 ) C12 in the graph C1 are respectively straight lines, and the horizontal lines C11 and the vertical lines C12 are mutually Orthogonal. On the other hand, the plurality of horizontal lines D11 and the plurality of vertical lines D12 of the grid pattern D1 are distorted so as to bulge outward from the center of the captured image G11, respectively.

Therefore, the image processing unit 122 reduces the lens distortion of the captured image G11 by performing lens distortion correction on the captured image G11.

FIG. 5A shows a lens-corrected image G21 after correcting the longitudinal lens distortion of the captured image G11. In the lens correction image G21, the grid-shaped graph C1 is reflected as a grid pattern D21. The plurality of horizontal lines D21 of the grid pattern D2 are respectively distorted by expanding from the center of the lens correction image G21 to the outside, but the distortions of the plurality of vertical lines D22 of the grid pattern D2 are respectively reduced, and are almost straight lines. The vertical lines D22 are substantially parallel to each other.

FIG. 5B shows a lens-corrected image G22 for correcting the two-dimensional lens distortion in the vertical and horizontal directions of the captured image G11. In the lens correction image G22, the grid-shaped graph C1 is mapped to the grid pattern D3. Distortions of the plurality of horizontal lines D31 and the plurality of vertical lines D32 of the grid pattern D3 are respectively reduced and almost straight. The plurality of horizontal lines D31 are substantially parallel to each other, and the plurality of vertical lines D32 are substantially parallel to each other.

Therefore, as shown in FIG. 6 , if the hall walkie-talkie 1 is installed along a vertical plane such as the wall 7 and the camera unit 11 captures the standing visitor 6 from the front, the lens-corrected image is an easy-to-view image with reduced distortion. For example, in the lens-corrected image in which the longitudinal lens distortion is corrected, the longitudinal distortion is reduced, and the visitor 6 reflected on the lens-corrected image can be seen standing upright. In addition, in the following description, the standing visitor 6 may be referred to as the visitor 61 .

However, due to the installation state of the intercom 1 in the lobby and the posture of the visitor, problems such as thinning of the top of the image and deterioration of the image balance occur.

Hereinafter, a first comparative example which is different from the first embodiment will be described. In the first comparative example, the lens correction process was performed only on the captured image. As shown in FIG. 3 , the lobby intercom 1 installed obliquely images the visitor 61 , and as a first comparative example, when only the lens correction process is performed on the captured image, the lens correction image G24 shown in FIG. 8A is generated. In the lens-corrected image G24, the upper body of the visitor 61 is reflected as the subject region R24. In the lens-corrected image G24, as the lower end (first end) of the lens-corrected image G24 approaches the upper end (second end), the top becomes thinner, and the head becomes smaller than the body of the subject region R24. When the lens correction image G24 is displayed on the display unit 23 , the occupant of the house has a strong sense of incongruity with the subject region R24 whose tip is tapered, and it becomes difficult to confirm the visitor 6 .

For example, in the hall walkie-talkie 1 installed obliquely, the imaging unit 11 captures a grid-shaped graph C1 (refer to FIG. 4A ) arranged in the vertical direction. The image processing unit 122 performs longitudinal lens distortion correction on the captured image of the graph C1, and generates a lens-corrected image G23 shown in FIG. 7 . In the lens correction image G23, the grid-shaped graph C1 is mapped to the grid pattern D4. The plurality of horizontal lines D41 of the grid pattern D4 are respectively distorted so as to bulge outward from the center of the lens correction image G23. Distortions of the plurality of vertical lines D42 of the grid pattern D4 are respectively reduced and almost straight. However, the plurality of vertical lines D42 are inclined so as to approach each other as they travel from the lower end to the upper end, respectively. That is, the plurality of vertical lines D42 extend radially so as to separate from each other as they proceed from the upper end to the lower end. Therefore, in the lens-corrected image G23, as it progresses from the lower end to the upper end, the tip thinning occurs in the longitudinal direction.

On the other hand, as shown in FIG. 9 , if the lobby walkie-talkie 1 installed obliquely captures a visitor 6 leaning forward to operate the lobby walkie-talkie 1, and as the first comparative example, only the lens correction processing is performed on the captured image, resulting in The lens correction image G25 shown in FIG. 8B. In addition, in the following description, the client 6 leaning forward may be called the client 62 in some cases. In this case, the face of the visitor 62 is directly opposite to the imaging unit 11, and in the lens-corrected image G25, the upper body of the visitor 62 is reflected as the subject area R25. In the lens-corrected image G25, the balance of the head and the body of the subject region R25 is improved compared to the balance of the subject region R24.

In order to suppress the above-mentioned tip thinning, the image processing unit 122 performs viewpoint conversion distortion correction in addition to vertical lens distortion correction on the captured image. The viewpoint transition distortion correction rotates the captured image by a conversion angle θb with respect to the viewpoint of the subject by deforming the captured image (see FIG. 9 ). Specifically, the captured image is deformed so that the relative angle θa between the imaging unit 11 and the subject is changed by the conversion angle θb. In FIGS. 3 and 9 , the relative angle is the angle between the imaging unit 11 and the subject when the left-right direction of the visitor 6 (the normal direction of the paper in FIGS. 3 and 9 ) is the central axis. In FIGS. 3 and 9 , since the direction of the imaging unit 11 depends on the inclination angle θ1 of the installation surface 41 , the angle formed between the installation surface 41 and the upper body of the subject, ie, the visitor 6 , is the relative angle θa. In FIG. 3 , the conversion angle θb=0 degrees, and the relative angle θa=θ1. In FIG. 9 , the conversion angle θb=θ1, and the relative angle θa is 0 degrees.

Hereinafter, a second comparative example which is different from the first embodiment will be described. In the second comparative example, the image processing unit 122 performs viewpoint conversion distortion correction such that the conversion angle θb=θ1 and the relative angle θa is 0 degrees. In this case, the imaging unit 11 captures the standing visitor 61, and the image processing unit 122 generates the corrected image G31 shown in FIG. 10A by correcting the lens distortion and the viewpoint conversion distortion. In the corrected image G31, the upper body of the visitor 61 is reflected as the subject area R31. In the corrected image G31, the balance between the head and the body of the subject region R31 is improved compared to the balance of the subject region R24 (see FIG. 8A ). However, when the imaging unit 11 captures the forward-leaning visitor 62, the image processing unit 122 generates the corrected image G32 shown in FIG. 10B through lens distortion correction and viewpoint conversion distortion correction. In the corrected image G32, the upper body of the visitor 62 is reflected as the subject area R32. In the corrected image G32, the head of the subject region R32 is larger than the body, and the balance between the head and the body of the subject region R32 is deteriorated.

Therefore, the image processing unit 122 of the first embodiment sets the conversion angle θb to 0<θb<θ1, and performs viewpoint conversion distortion correction so that the relative angle θa is larger than 0 degrees and smaller than the inclination angle θ1. In this case, if the imaging unit 11 captures the standing visitor 61, the image processing unit 122 generates the corrected image G33 shown in FIG. 11A by correcting the lens distortion and the viewpoint conversion distortion. In the corrected image G33, the upper body of the visitor 61 is reflected as the subject area R33. In the corrected image G33, the balance of the head and the body of the subject region R33 is improved compared to the balances of the subject regions R24 (see FIG. 8A ) and the subject regions R32 (see FIG. 10B ). Furthermore, when the imaging unit 11 captures the image of the visitor 62 leaning forward, the image processing unit 122 generates a corrected image G34 shown in FIG. 11B through lens distortion correction and viewpoint conversion distortion correction. In the lens-corrected image G34, the upper body of the visitor 62 is reflected as the subject area R34. In the corrected image G34, the balance of the head and the body of the subject region R34 is improved compared to the balances of the subject regions R24 (see FIG. 8A ) and the object regions R32 (see FIG. 10B ).

Therefore, since the image processing unit 122 sets the conversion angle θb to be 0<θb<θ1, the image distortion can be suppressed regardless of the posture of the visitor 6 captured. In addition, the conversion angle θb is preferably θb=θ1/2.

FIG. 12 shows a corrected image G35 obtained by performing the viewpoint conversion distortion correction (see FIGS. 11A and 11B ) of the image processing unit 122 of the first embodiment on the lens corrected image G23 of FIG. 7 . When θb=θ1/2, in the corrected image G35, the grid-shaped graph C1 is reflected as the grid pattern D5. The plurality of horizontal lines D51 of the grid pattern D5 are respectively distorted so as to bulge outward from the center of the corrected image G35. Distortions of the plurality of vertical lines D52 of the grid pattern D5 are respectively reduced and almost straight. Furthermore, the plurality of vertical lines D52 are respectively parallel to each other. Therefore, in the corrected image G35, tip thinning is suppressed.

The image processing unit 122 executes at least one of a first process of extending the image in the horizontal direction and a second process of extending the image in the vertical direction as viewpoint conversion distortion correction. Each stretch amount of the first process and the second process corresponds to the correction amount of the viewpoint conversion distortion correction. Then, the correction amount gradually increases from the lower end of the captured image to the upper end. That is, the correction amount is set asymmetrically in the longitudinal direction (vertical direction) of the captured image. In addition, it is preferable to set the correction amount to be symmetrical in the lateral direction (left-right direction) of the captured image.

Furthermore, it is preferable that the image processing unit 122 of the first embodiment performs one coordinate transformation (projection processing) on the captured image, thereby collectively executing both the lens distortion correction and the viewpoint conversion distortion correction.

The memory portion 123 is a memory capable of rewriting data, preferably a non-volatile memory. The memory unit 123 is configured to record the captured image acquired from the imaging unit 11 . In addition, the memory unit 123 is configured to record correction parameters used by the image processing unit 122 for lens distortion correction and viewpoint conversion distortion correction. The correction parameter is a parameter related to the correction amount of the lens distortion correction corresponding to the lens 112 and the correction amount of the viewpoint conversion distortion correction corresponding to the conversion angle θb, and is displayed as data of at least one of the correction amount and the row and row of the coordinate conversion. . When performing lens distortion correction and viewpoint conversion distortion correction, the image processing unit 122 reads out correction parameters from the memory unit 123, and performs lens distortion correction and viewpoint conversion distortion correction using the correction parameters.

In addition, the image processing unit 122 preferably performs color tone correction on the captured image. Tone correction includes correction of the color, brightness and shading of the captured image. For example, when the image processing unit 122 performs color tone correction, face recognition processing is performed on the captured image, and the color of the captured image is determined so that both the face area and the background area in the captured image are brighter. , a parameter related to at least one of brightness and shade. The image processing unit 122 performs color tone correction based on the determined parameters on the captured image.

The image processing method of the image processing system 12 (image processing device 120 ) of the first embodiment is shown in the flowchart of FIG. 13 . The image processing method includes an image acquisition step S1 and an image processing step S2. The image acquisition step S1 acquires the data of the captured image captured by the imaging unit 11 of the lobby walkie-talkie 1 . The image processing step S2 performs image processing on the captured image. Then, in the image processing step S2, viewpoint conversion distortion correction is performed by deforming the captured image to rotate the captured image by a conversion angle θb with respect to the viewpoint of the subject. The conversion angle θb is larger than 0 degrees and smaller than the inclination angle θ1 of the installation surface 41 of the hall walkie-talkie with respect to the vertical plane.

(3) Variation example Hereinafter, a modified example will be described. In addition, the modified example described below can be combined with the above-mentioned 1st Embodiment suitably, and can be applied.

(3-1) The first modification example The image processing apparatus 120 preferably further includes a mode switching unit 124 (see FIG. 1 ). The mode switching unit 124 switches the operation mode of the image processing unit 122 to the first mode or the second mode. The switching of the operation mode of the mode switching unit 124 can be performed manually or automatically.

When switching the operation mode manually, a builder, an administrator, etc. perform an operation mode setting operation on the sub-machine operation unit 15 . The slave control unit 13 instructs the mode switching unit 124 to switch the operation mode according to the setting operation of the slave operation unit 15 . When a construction worker, a manager, etc. install the lobby intercom 1 at an incline as shown in FIG. 3 , the operation mode is set to the first mode. When a construction worker, a manager, etc. install the lobby intercom 1 along a vertical surface as shown in FIG. 6 , the operation mode is set to the second mode.

When switching the operation mode automatically, the lobby walkie-talkie 1 is provided with a tilt sensor 17 (refer to FIG. 1 ). The tilt sensor 17 detects the tilt angle θ1 and transmits a tilt detection signal including the information of the tilt angle θ1 to the image processing device 120 . In the image processing device 120 , the information acquisition unit 126 acquires the tilt detection signal as input information, and transmits the tilt detection signal to the mode switching unit 124 . The mode switching unit 124 switches the operation mode of the image processing unit 120 to the first mode or the second mode based on the magnitude of the inclination angle θ1. The mode switching unit 124 sets the operation mode to the second mode when the inclination angle θ1 is equal to or greater than 0 degrees and less than the threshold value. The mode switching unit 124 sets the operation mode to the first mode when the inclination angle θ1 is greater than or equal to the threshold value and less than or equal to 90 degrees.

In addition, the mode switching unit 124 can acquire the captured image of the grid-shaped graph C1 placed vertically as a test image, and switch the operation mode of the image processing unit 122 based on the degree of deformation of the grid pattern reflected on the test image. for the 1st mode or the 2nd mode. If the deformation degree of the grid pattern does not reach the predetermined value, the mode switching unit 124 sets the operation mode to the second mode. When the degree of deformation of the grid pattern is equal to or greater than a predetermined value, the mode switching unit 124 sets the operation mode to the first mode.

When the operation mode is the first mode, the image processing unit 122 performs both viewpoint conversion distortion correction and lens distortion correction. When the operation mode is the second mode, the image processing unit 122 performs lens distortion correction instead of viewpoint conversion distortion correction.

Therefore, if the lobby walkie-talkie 1 is installed obliquely, the image processing unit 122 performs both viewpoint conversion distortion correction and lens distortion correction, thereby suppressing image distortion regardless of the posture of the visitor 6 being photographed. Furthermore, if the lobby intercom 1 is installed along the vertical plane, since the image processing unit 122 does not perform viewpoint conversion distortion correction, the computational load of image processing can be reduced compared with the first mode.

(3-2) Second variation example The image processing device 120 preferably further includes a degree selection unit 125 (see FIG. 1 ). The degree selection unit 125 selects the correction degree of the viewpoint conversion distortion correction when the image processing unit 122 performs the viewpoint conversion distortion correction.

In this modified example, the memory unit 123 stores the correction parameters for correction of the viewpoint transformation distortion corresponding to the plurality of transformation angles θb, respectively. For example, as shown in FIG. 14 , the storage unit 123 stores respective correction parameters for three types of viewpoint conversion distortion correction corresponding to the plurality of conversion angles θb1 to θb3, respectively. The magnitude relationship of the conversion angles θb1 to θb3 is θb1<θb2<θb3. The correction parameter corresponding to the conversion angle θb1 corresponds to the first correction degree, the correction parameter corresponding to the conversion angle θb2 corresponds to the second correction degree, and the correction parameter corresponding to the conversion angle θb3 corresponds to the third correction degree. The first correction degree is the lowest correction degree, and the third correction degree is the highest correction degree.

In this modification, the selection of the correction degree by the degree selection unit 125 is performed manually.

In the case where the selection of the correction degree is performed manually, the builder, the administrator, etc. perform the selection operation of the correction degree on the sub-machine operation unit 15 . The slave control unit 13 instructs the degree selection unit 125 to select the correction degree in accordance with the selection operation of the slave operation unit 15 . As the inclination angle θ1 is larger, a construction worker, a manager, or the like selects a higher correction degree among the first correction degree to the third correction degree. The degree selection unit 125 transmits the selection result to the image processing unit 122 .

The image processing unit 122 reads out the correction parameter of the conversion angle θb corresponding to the selected correction degree from the memory unit 123 . When the first correction degree is selected, the image processing unit 122 reads out the correction parameter of the conversion angle θb1 from the memory unit 123 . When the second correction degree is selected, the image processing unit 122 reads out the correction parameter of the conversion angle θb2 from the memory unit 123 . When the third correction degree is selected, the image processing unit 122 reads out the correction parameter of the conversion angle θb3 from the memory unit 123 . The image processing unit 122 uses the read correction parameters.

In addition, the storage unit 123 can memorize the respective correction parameters for correction of a plurality of viewpoint conversion distortions corresponding to a plurality of inclination angles θ1 of the mounting surface 41, respectively. For example, the storage unit 123 stores the correction parameters for three types of viewpoint conversion distortion correction corresponding to the plurality of tilt angles θ11 to θ13 (see FIG. 4 ), respectively. The magnitude relationship of the inclination angles θ11 to θ13 is θ11<θ12<θ13. The correction parameter corresponding to the inclination angle θ11 corresponds to the first correction degree, the correction parameter corresponding to the inclination angle θ12 corresponds to the second correction degree, and the correction parameter corresponding to the inclination angle θ13 corresponds to the third correction degree.

As the inclination angle θ1 is larger, a construction worker, a manager, or the like selects a higher correction degree among the first correction degree to the third correction degree. The degree selection unit 125 transmits the selection result to the image processing unit 122 . The image processing unit 122 reads out the correction parameter of the conversion angle θb corresponding to the selected correction degree from the memory unit 123 .

As described above, in this modification, since the correction amount of viewpoint conversion distortion correction is set by manual operation of the builder or administrator, it is possible to set the optimum correction amount while checking the actual correction image. As a result, irrespective of the magnitude of the inclination angle θ1 , it is possible to suppress a sense of incongruity related to the balance between the head and the body of the subject region in the corrected image.

In addition, the number of the above-mentioned correction levels is not limited to three of the first correction level to the third correction level, and may be two or four or more.

(3-3) The third modification example In the second modification, the selection of the correction degree may be performed automatically. In the case where the selection of the correction degree is performed automatically, it is preferable that the lobby walkie-talkie 1 is provided with the tilt sensor 17, and the image processing device 120 is provided with the information acquisition unit 126 (refer to FIG. 1). The tilt sensor 17 detects the tilt angle θ1 and transmits a tilt detection signal including information of the tilt angle θ1 to the image processing device 120 . In the image processing device 120 , the information acquisition unit 126 acquires the tilt detection signal as input information, and transmits the tilt detection signal to the degree selection unit 125 . The degree selection unit 125 selects the correction degree of the viewpoint conversion distortion correction according to the inclination angle θ1 when the image processing unit 122 performs the viewpoint conversion distortion correction. The degree selection unit 125 selects the higher correction degree among the first correction degree to the third correction degree as the inclination angle θ1 increases, and transmits the selection result to the image processing unit 122 .

The image processing unit 122 reads out the correction parameter of the conversion angle θb corresponding to the selected correction degree from the memory unit 123 . When the first correction degree is selected, the image processing unit 122 reads out the correction parameter of the conversion angle θb1 from the memory unit 123 . When the second correction degree is selected, the image processing unit 122 reads out the correction parameter of the conversion angle θb2 from the memory unit 123 . When the third correction degree is selected, the image processing unit 122 reads out the correction parameter of the conversion angle θb3 from the memory unit 123 . The image processing unit 122 uses the read correction parameters.

In addition, the image processing unit 122 can recognize the inclination angle θ1 based on the inclination detection signal. Therefore, the image processing unit 122 can set the conversion angle θb to be 1/2 of the inclination angle θ1. In this case, the memory unit 123 stores in advance a formula or table for obtaining the correction parameter from the conversion angle θb, and the image processing unit 122 uses the formula or table of the memory unit 123 to obtain the correction parameter from the conversion angle θb.

In addition, a builder, a manager, or the like can perform an input operation for inputting the information of the installation height H1 of the imaging section 11 as the height position information of the imaging section 11 on the sub-machine operation section 15 . When the slave control unit 13 receives the information of the setting height H1 of the input operation of the slave operation unit 15 , it transmits the information of the setting height H1 to the information acquiring unit 126 . The information acquisition unit 126 acquires the information of the setting height H1 as input information, and transmits the information of the setting height H1 to the image processing unit 122 . The image processing unit 122 can obtain the magnitude of the conversion angle θb based on the inclination angle θ1 and the installation height H1.

In addition, the image processing unit 122 preferably corrects the distortion of the captured image without changing the viewing angle of the captured image. That is, the image processing unit 122 performs the viewpoint conversion distortion correction and the lens distortion correction so that the viewing angle does not change due to the viewpoint conversion distortion correction and the lens distortion correction.

The image processing method of the image processing system 12 (image processing device 120 ) of this modification is shown in the flowchart of FIG. 15 . The image processing method includes an image acquisition step S11, an information acquisition step S12, and an image processing step S13. The image acquisition step S11 acquires image data captured by the camera unit 11 of the lobby walkie-talkie 1 . The information acquisition step S12 acquires input information. The image processing step S13 performs image processing on the captured image. And, the image processing step corrects the above-mentioned distortion of the captured image based on the input information.

As described above, in this modification, the correction amount corresponding to the inclination angle θ1 can be automatically set. As a result, irrespective of the magnitude of the inclination angle θ1, it is possible to suppress a sense of incongruity related to the balance between the head and the body of the subject region in the corrected image.

In addition, when the image processing unit 122 performs the viewpoint conversion distortion correction, if the viewpoint conversion distortion correction is performed considering the installation height H1 in addition to the inclination angle θ1, the distortion of the corrected image can be further suppressed.

(3-4) Other Variations Among the above-described embodiments and modifications, the various embodiments and modifications of the present disclosure are merely examples. As long as the above-described embodiment and each modification can achieve the purpose of cost disclosure, various modifications can be made in accordance with design and the like. In addition, the same functions as the image processing system 12 (image processing device 120 ) can be embodied by an image processing method, a computer program, or a non-transitory recording medium in which the program is recorded.

The execution subject of the image processing system 12 or the image processing method of the present disclosure includes a computer system. A computer system has a processor and memory as hardware. By causing the processor to execute the program recorded in the memory of the computer system, the function as the execution body of the image processing system 12 or the image processing method of the present disclosure is realized. The program can be pre-recorded in the memory of the computer system, and can also be provided through an electrical communication line. In addition, the program can be provided by being recorded on a non-transitory recording medium such as a memory card, optical disc, and hard disk that can be read by a computer. The processor of the computer system is composed of one or a plurality of electronic circuits including a semiconductor integrated circuit (IC: Integrated Circuit) or a large-scale integrated circuit (LSI: Large Scale Integrated Circuit). A plurality of electronic circuits can be concentrated on one chip, or can be distributed on a plurality of chips. A plurality of wafers can be collected in one device, or distributed in a plurality of devices.

The computer system, that is, the image processing system 12 may be a system composed of one or a plurality of computers. For example, at least a part of the functions of the image processing system 12 may be implemented by the cloud (cloud computing).

(Second Embodiment) Hereinafter, the walkie-talkie system 10A and the image processing system (or image information output system) 12A of the present embodiment will be described with reference to FIGS. 16 to 21 .

(1) Outline As shown in FIG. 17, the walkie-talkie system 10A of the present embodiment includes a hall walkie-talkie 1A as a walkie-talkie device, a plurality of (two in the example of the figure) two main sets of walkie-talkies, and a plurality of (two in the example of the figure) entrance doors machine 9A, and control device 3A. In this embodiment, the case where the walkie-talkie system 10A is used for a complex house as a facility F1 will be described as an example. In addition, each of the plurality of walkie-talkie base units corresponds to the information terminal of the present disclosure.

The lobby walkie-talkie 1A is installed in, for example, a common part such as a lobby of a condominium. In addition to the shared part, the collective housing also has a plurality of exclusive parts (households). The main unit 2A of the walkie-talkie is installed in the dedicated sections for each dedicated section. The door sub-machine 9A is installed in the door outside the house, for example, for each dedicated part.

The lobby walkie-talkie 1A includes an imaging unit 11A and an image processing device 120A as an image processing system 12A. The lobby walkie-talkie 1A performs processing related to the image (camera image) captured by the imaging unit 11A.

The lobby walkie-talkie 1A and the walkie-talkie master 2A are configured to be able to communicate with each other via the control device 3A. Thereby, in the walkie-talkie system 10A of the present embodiment, the lobby walkie-talkie 1A and the walkie-talkie master 2A can communicate with each other. Furthermore, the main unit 2A of the walkie-talkie and the sub-unit 9A of the entrance door are configured to be able to communicate with each other.

Moreover, the lobby intercom 1A and the control device 3A are connected to a network NT1 such as the Internet via a gateway not shown. Thereby, the lobby intercom 1A and the control device 3A can communicate with the server 8A via the network NT1.

When the hall walkie-talkie 1A detects a person (for example, a visitor) in the common section, it starts imaging the person. When the lobby walkie-talkie 1A accepts an operation for calling from a visitor, it transmits a control signal (calling signal) for calling the occupant in the private section to the master walkie-talkie 2A via the control device 3A. When the lobby walkie-talkie 1A can communicate with the walkie-talkie master 2A by operating on the walkie-talkie master 2A, it sends the visitor's voice data to the walkie-talkie master 2A via the control device 3A. When the lobby walkie-talkie 1A receives the voice data of the operator of the walkie-talkie master 2A via the control device 3A, it outputs the voice (operator's voice) based on the received data.

The image processing unit 120A performs image processing on the captured image according to the purpose of the image (captured image) captured by the imaging unit 11A. In this embodiment, image authentication is included as the first use, and display of the main unit 2A of the walkie-talkie is included as the second use.

In the state where the intercom master 2A receives a control signal (calling signal) from the lobby intercom 1A (that is, the state where the visitor calls via the lobby intercom 1A), the occupants in the exclusive department communicate with the lobby intercom 1A or unlock the door of the lobby operation.

When the control device 3A receives a control signal (calling signal) from the hall walkie-talkie 1A, it transmits the received control signal to the calling walkie-talkie master 2A. The control device 3A transmits and receives the audio data of the visitor and the audio data of the operator when the intercom 1A in the lobby and the main intercom 2A are in a state of being able to communicate. Furthermore, the control device 3A performs, for example, locking and unlocking of a door installed in a hall in accordance with the operation of the intercom master unit 2A.

The lobby walkie-talkie 1A transmits the captured image to the server 8A (see FIG. 17 ) when the operation for calling is not accepted from the visitor after the image of the person is started. The lobby intercom 1A performs the unlocking process of the lobby door according to the result of the authentication process of the server 8A.

When the hall sub-unit 9A receives an operation for calling from the visitor, it transmits a control signal (calling signal) for calling the occupant in the private section to the main unit 2A of the walkie-talkie. The hall sub-unit 9A transmits the voice data of the visitor to the main intercom unit 2A when the communication between the main intercom unit 2A and the main intercom unit 2A is possible by the operation of the main unit 2A. When the hall sub-unit 9A receives the voice data of the operator of the walkie-talkie main unit 2A, it outputs the voice (the operator's voice) based on the received data.

The server 8A uses the image captured by the lobby intercom 1A to authenticate the captured person (eg, a visitor, a resident returning from an outing). The server 8A sends the authentication result to the lobby walkie-talkie 1A via the network NT1. In this embodiment, as an example of authentication using an image (image authentication), the server 8A performs face authentication.

(2) Composition (2-1) Intercom in the lobby As shown in FIG. 16 , the lobby walkie-talkie 1A includes an image processing device 120A as an image processing system 12A, an imaging unit 11A, a sensor unit 18A, a first communication unit 19A, a second communication unit 14A, an operation unit 15A, and a communication unit. part 16A and control part 13A.

The lobby walkie-talkie 1A has, for example, a microcomputer including a processor and a memory. Then, when the processor executes the program stored in the memory, the microcomputer functions as the control unit 13A. The program executed by the processor is pre-recorded in the memory of the microcomputer, and can also be provided by being recorded in a non-transitory recording medium such as a memory card, or provided through an electrical communication line such as the Internet.

The imaging unit 11A has an imaging element such as a CCD (Charge Coupled Devices) image sensor or a two-dimensional image sensor such as a CMOS (Complementary Metal-Oxide Semiconductor) image sensor, and is used to capture a subject (visitor). , etc.) cameras. The lens used by the imaging unit 11A for photographing is, for example, a fisheye lens. In this embodiment, the imaging area (field of view) of the imaging unit 11A is set in front of the lobby walkie-talkie 1A so that at least the face of the visitor can be photographed. In addition, in the present embodiment, the imaging unit 11A is a camera that captures a moving image, but may be a camera that captures a still image. Furthermore, in the present embodiment, the imaging unit 11A is a camera that captures a color image, but may be a camera that captures a monochrome image.

The sensor portion 18A is, for example, a human sensor that detects the approach of a person.

The first communication unit 19A also includes a communication interface for communicating with the server 8A via the network NT1. The first communication unit 19A is configured to be capable of bidirectional communication with the server 8A so as to transmit at least images captured by the lobby intercom 1A, image information, and authentication information between the lobby intercom 1A and the server 8A.

The second communication unit 14A includes a communication interface for communicating with the main unit 2A of the walkie-talkie via the control device 3A. The second communication unit 14A is configured to be capable of bidirectional communication with the main walkie-talkie unit 2A so that audio signals and control signals can be bidirectionally transmitted between the hall walkie-talkie 1A and the main walkie-talkie unit 2A.

The operation unit 15A accepts an operation for calling from a visitor. When a specific operation (such as a key operation) is performed on the operation unit 15A, a control signal (call signal) for calling a resident in the exclusive unit (household) is transmitted to the main unit of the walkie-talkie via the second communication unit 14A and the control device 3A 2A.

The communication unit 16A includes a speaker and a microphone, and is configured to be able to communicate with the main unit 2A of the walkie-talkie.

The control unit 13A receives the detection result from the sensor unit 18A. The control unit 13A receives the operation information received by the operation unit 15A from the operation unit 15A.

When the detection result of the sensor 18A indicates that a person has been detected, the control unit 13A activates the imaging unit 11A to start imaging. That is, the control unit 13A starts to supply power to the imaging unit 11A.

The control unit 13A determines whether to cause the server 8A to perform authentication based on the captured image, or to cause the walkie-talkie main unit 2A to display an image based on the captured image. Specifically, the control unit 13A determines to display an image based on the captured image on the main intercom 2A when a call operation is received within a predetermined time after the image capturing by the image capturing unit 11A starts. The control unit 13A judges that the server 8A displays the authentication based on the captured image when the call operation is not accepted within a certain period of time after the image capturing by the image capturing unit 11A starts.

When the control unit 13A displays an image based on the captured image on the main intercom 2A, it transmits a control signal (call signal) to the main intercom 2A via the second communication unit 14A and the control device 3A. When the control unit 13A is in a state of being able to communicate with the main walkie-talkie unit 2A, if the voice data (audio signal) of the visitor is received from the microphone of the communication unit 16A, the received audio signal is sent to the main intercom unit 2A. When the control unit 13A is in a state of being able to communicate with the intercom base unit 2A, if it receives the operator's voice signal from the intercom base unit 2A, it outputs the received audio signal to the speaker of the communication unit 16A. At this time, the speaker outputs sound based on the received sound signal.

The control unit 13A receives the authentication result from the server 8A via the first communication unit 19A. The control part 13A controls the opening and closing operation of the door of the common part according to the authentication result. Specifically, when the authentication is successful, the control unit 13A sends an unlocking signal for opening the door of the common unit to the control device 3A. On the other hand, when the authentication fails, the control unit 13A does not transmit the unlocking signal for opening the door of the common unit to the control device 3A. That is, the control part 13A maintains the state of the door of the common part in the closed state.

The image processing device 120A performs image processing on the captured image according to the purpose of the image (captured image) captured by the imaging unit 11A. Specifically, when the server 8A performs authentication, the image processing device 120A generates an image for authentication based on the image captured by the imaging unit 11A as the first image information, and uses the generated image for authentication as the first image information. Sent to server 8A. The image processing device 120A generates an image for display as the second image information based on the image captured by the camera unit 11A when displaying the main intercom 2A, and transmits the generated image for display to the main intercom 2A .

(2-2) Image processing device As shown in FIG. 16 , an image processing apparatus 120A serving as an image processing system 12A includes an image acquisition unit 121A, an image processing unit 122A, a first output unit 1271 , and a second output unit 1272 . The image processing unit 122A includes a first processing unit 1221 and a second processing unit 1222 .

The image acquisition unit 121A receives an image (captured image) captured by the imaging unit 11A from the imaging unit 11A. The image acquisition unit 121A performs processing (common processing) commonly performed on the received captured image to generate an image for authentication and a process to generate an image for display. For example, the image acquisition unit 121A performs processing such as RGB Bayer and two-stage exposure on the received captured image.

The first processing unit 1221 is realized by one integrated circuit, and generates an image for authentication (first image information) based on the captured image. Specifically, the first processing unit 1221 performs processing of at least one of distortion correction and color tone correction on the captured image. Distortion correction includes lens distortion correction and viewpoint conversion distortion correction. Tone correction includes correction of the color, brightness and shading of the captured image.

The first processing unit 1221 captures the face region of the person included in the captured image.

When performing distortion correction, the first processing unit 1221 determines parameters for distortion correction according to the extracted face region. Specifically, the first processing unit 1221 determines a parameter for distortion correction so that the face of the person included in the captured image is sharpened. For example, when the hall walkie-talkie 1A is installed on a sloping table, that is, when the imaging unit 11A is installed obliquely with respect to the vertical direction, and when a person is in a state of leaning forward, the camera is close to the imaging unit 11A, so the image is captured. The face area becomes larger. In this case, the imaging unit 11A is in a state of facing the face of the person, and when the face is located in the center of the captured image, the distortion in the face area is small. Therefore, the first processing unit 1221 determines the parameters so as to perform lens distortion correction for correcting the distortion around the captured image. The first processing unit 1221 performs distortion correction based on the determined parameters on the captured image.

Furthermore, when the hall walkie-talkie 1A is installed on the inclined table, and the image is taken with the person standing upright, as shown in FIG. 18A , it can be seen that the top of the visitor U1 is thin (the top (head) is thin). Therefore, the small face area is not suitable for authentication. Therefore, the first processing unit 1221 corrects the distortion shown in FIG. 18A (refer to FIG. 18B ) by performing viewpoint conversion distortion correction on, for example, the captured image (image in which the tip of the visitor is tapered). In FIG. 18B , the thinning of the top end is suppressed for the client U1, so that the face area of the client U1 is larger than the former. Therefore, by setting the corrected image as an image for authentication, the accuracy of authentication can be improved.

In addition, when the imaging unit 11A is provided obliquely with respect to the vertical direction, the first processing unit 1221 can perform viewpoint conversion correction when imaging is performed in a state where the person is leaning forward. In addition, in the case where the lobby walkie-talkie 1A is mounted on a tilting table, the first processing unit 1221 can perform lens distortion correction when photographing with a person standing upright.

When the first processing unit 1221 performs color tone correction, it determines at least one of the color, brightness, and shading of the captured image so that the face area is relatively brighter than the area other than the face area (background area). related parameters. The first processing unit 1221 performs color tone correction based on the determined parameters on the captured image.

The first processing unit 1221 generates an image for authentication by performing at least one of the above-described distortion correction and color tone correction on the captured image. The first processing unit 1221 transmits the generated image for authentication to the server 8A via the first output unit 1271 and the first communication unit 19A.

The first output unit 1271 transmits the authentication image generated by the first processing unit 1221 to the server 8A via the first communication unit 19A. Specifically, it is provided at the output port of the integrated circuit that realizes the first processing unit 1221 . The first output port 1271 outputs the image for authentication received from the first processing unit 1221 to the first communication unit 19A connected to the first output unit 1271 .

The second processing unit 1222 is realized by an integrated circuit different from the integrated circuit that realizes the first processing unit 1221, and generates an image for display (second image information) based on the captured image. Specifically, the second processing unit 1222 performs processing of at least one of distortion correction and color tone correction on the captured image. The distortion correction includes lens distortion correction and viewpoint conversion distortion correction, as in the case of the first processing unit 1221 . As in the case of the first processing unit 1221, the color tone correction includes correction of the color, brightness, and shading of the captured image.

The second processing unit 1222 captures the face area of the person included in the captured image.

When the second processing unit 1222 performs distortion correction, the parameters for distortion correction are determined according to the extracted face region. Specifically, the second processing unit 1222 determines parameters for distortion correction so that the entire captured image can be clearly seen. For example, when the lobby walkie-talkie 1A is installed on a tilting table, and when a person leans forward to take a picture, the face area to be photographed becomes larger due to the proximity of the imaging unit 11A. In this case, the face of the person is large, and the clarity of the entirety including the background is poor. Therefore, the second processing unit 1222 determines the parameters so as to perform distortion correction for projecting the captured image in the vertical direction, that is, viewpoint conversion distortion correction. The second processing unit 1222 performs distortion correction based on the determined parameters on the captured image. In addition, in the case where the lobby walkie-talkie 1A is installed on the inclined table, the second processing unit 1222 can also perform lens distortion correction when taking pictures with the person leaning forward.

In addition, when the imaging unit 11A is provided obliquely with respect to the vertical direction, the second processing unit 1222 can perform viewpoint conversion correction and lens distortion correction when photographing with a person standing upright.

When the second processing unit 1222 performs color tone correction, it determines parameters related to at least one of the color, brightness, and shading of the captured image so that both the face area and the background area are bright. The second processing unit 1222 performs color tone correction based on the determined parameters on the captured image.

The second processing unit 1222 generates an image for display by performing at least one of the above-described distortion correction and color tone correction on the captured image. The second processing unit 1222 transmits the generated image for display to the main walkie-talkie unit 2A, which is the calling party, via the second output unit 1272 and the second communication unit 14A.

The second output unit 1272 transmits the display image generated by the second processing unit 1222 to the main walkie-talkie unit 2A via the second communication unit 14A and the control device 3A. Specifically, it is provided at the output port of the integrated circuit that realizes the second processing unit 1222 . The second output unit 1272 outputs the display image received from the second processing unit 1222 to the second communication unit 14A connected to the second output unit 1272 .

In the present embodiment, the captured images can be processed differently from each other. For example, the first processing unit 1221 performs distortion correction and the second processing unit 1222 performs color tone correction.

In addition, the same processing (distortion correction) may be performed on the captured image with different parameters. For example, the first processing unit 1221 performs lens distortion correction as distortion correction, and the second processing unit 1222 performs viewpoint transformation distortion correction as distortion correction. . Alternatively, the same processing (tone correction) may be performed on the captured image using mutually different parameters. For example, the first processing unit 1221 performs brightness correction as tone correction, and the second processing unit 1222 performs gradation correction as tone correction.

In addition, the first processing unit 1221 may perform at least one of distortion correction and color tone correction only on the extracted face region. When the first processing unit 1221 extracts a plurality of face regions, the first processing unit 1221 may perform a correction corresponding to the face region on each of the plurality of face regions. For example, the first processing unit 1221 performs lens distortion correction on a face region existing at the edge of the captured image among the plurality of face regions. The first processing unit 1221 performs color tone correction on a face region existing in the center of the captured image among the plurality of face regions.

Likewise, the second processing unit 1222 may perform at least one of distortion correction and color tone correction only on the extracted face region. When the second processing unit 1222 extracts a plurality of face regions, the second processing unit 1222 may perform a correction corresponding to the face region on each of the plurality of face regions. For example, the second processing unit 1222 performs lens distortion correction on a face region existing at the edge of the captured image among the plurality of face regions. The second processing unit 1222 performs color tone correction on a face region existing in the center of the captured image among the plurality of face regions.

(2-3) Walkie-talkie base unit As shown in FIG. 17 , the main unit 2A of the walkie-talkie includes a first communication unit 22A, a second communication unit 26A, an operation unit 24A, a communication unit 25A, a display unit 23A, and a control unit 21A.

The main unit 2A of the walkie-talkie has, for example, a microcomputer including a processor and a memory. Then, when the processor executes the program stored in the memory, the microcomputer functions as the control unit 21A. The program executed by the processor is pre-recorded in the memory of the microcomputer, and can also be provided by being recorded in a non-transitory recording medium such as a memory card, or provided through an electrical communication line such as the Internet.

The first communication unit 22A is a communication interface for communicating with the control device 3A. The first communication unit 22A is connected to the lobby walkie-talkie 1A via the control device 3A. For example, the first communication unit 22A receives an image transmitted from the lobby intercom 1A. Furthermore, the first communication unit 22A is configured to be capable of bidirectional communication with the lobby intercom 1A so that audio signals and control signals can be bidirectionally transmitted to and from the lobby intercom 1A.

The second communication unit 26A is a communication interface for communicating with the hall slave 9A. The second communication unit 26A is electrically connected to the hall slave 9A via a two-wire transmission path including, for example, a twisted pair cable.

The operation part 24A accepts the operation of the occupant in the exclusive part. In a state where the first communication unit 22A receives a control signal (call signal) for calling the occupant from the lobby intercom 1A, if a specific operation is performed on the operation unit 24A, the intercom base unit 2A and the lobby intercom 1A can communicate with each other. state. Here, the specific operation is, for example, a key operation. When the first communication unit 22A receives a call signal from the lobby intercom 1A, or in a state where a call is possible with the lobby intercom 1A, if an unlocking operation is performed on the door installed in the common area, the unlocking signal is sent to the control device 3A.

Furthermore, in the state where the second communication unit 26A receives a control signal (call signal) for calling the occupant from the hall sub-unit 9A, if a specific operation (such as a key operation) is performed on the operation unit 24A, the intercom master unit 2A communicates with the main unit 2A. 9A of the hall handsets are in a state of being able to talk.

The communication unit 25A includes a speaker and a microphone, and is configured to be able to communicate with the hall walkie-talkie 1A and the hall sub-unit 9A.

The display unit 23A displays the image transmitted from the lobby walkie-talkie 1A.

When the control unit 21A receives the control signal (call signal) and the image (display image here) captured by the camera unit 11A from the lobby walkie-talkie 1A via the first communication unit 22A, it outputs the voice data of the calling sound to the communication unit 25A The speaker outputs the received display image to the display unit 23A. Thereby, the control part 21A makes the speaker of the communication part 25A output a calling sound, and displays the image for display on the display part 23A.

When the control unit 21A receives a control signal (calling signal) from the hall slave 9A via the second communication unit 26A, it causes the speaker of the communication unit 25A to output a calling sound.

From the above, the main unit 2A of the walkie-talkie is an information terminal that communicates with the walkie-talkie 1A in the hall, and displays an image for display.

(2-4) Foyer handset The hall sub-machine 9A has, for example, a microcomputer including a processor and a memory. Then, by the processor executing the program stored in the memory, the microcomputer functions as the hall slave 9A. The program executed by the processor is pre-recorded in the memory of the microcomputer, and can also be provided by being recorded in a non-transitory recording medium such as a memory card, or provided through an electrical communication line such as the Internet.

The hall sub-unit 9A transmits a control signal (calling signal) for calling the occupant in the private section to the intercom main unit 2A when the visitor accepts the operation for calling.

The hall sub-unit 9A includes a speaker and a microphone, and is configured to be able to communicate with the main unit 2A of the walkie-talkie.

In addition, the hall sub-machine 9A may have an imaging unit. In this case, the hall sub-unit 9A transmits the image captured by the camera unit to the walkie-talkie main unit 2A. The walkie-talkie master 2A displays the image received from the hall slave 9A on the display unit 23A.

(2-5) Server The server 8A includes a communication unit 81A, a storage unit 82A, and an authentication unit 83A.

The server 8A has, for example, a microcomputer including a processor and a memory. And the microcomputer functions as the server 8A by executing the program stored in the memory by the processor. Here, the program executed by the processor is pre-recorded in the memory of the microcomputer, and can also be provided by being recorded in a non-transitory recording medium such as a memory card, or provided through an electrical communication line such as the Internet.

The communication unit 81A is a communication interface for communicating with the lobby walkie-talkie 1A via the network NT1.

The memory section 82A can select ROM (Read Only Memory), RAM (Random Access Memory), or EEPROM (Electrically Erasable Programmable Read Only Memory) and other components.

The memory unit 82A stores the facial images of the occupants of each dedicated unit and the facial images of visitors registered in advance by the occupants.

The authentication unit 83A receives the image for authentication transmitted from the image processing device 120A from the lobby intercom 1A via the communication unit 81A.

The authentication unit 83A performs a face authentication process of comparing the received image with each image stored in the memory unit 82A.

The authentication unit 83A determines that the authentication is successful when the memory unit 82A stores an image that matches the received image. The authentication unit 83A judges that the authentication has failed when an image matching the received image is not stored in the memory unit 82A. Here, "consistency" is not completely consistent, but also includes the case where the degree of consistency is within the allowable error range.

In the case where the authentication image received from the image processing device 120A includes a plurality of faces, the authentication unit 83A stores in the memory unit 82A each of the faces included in the authentication image. When there is a matching image in the memory, it is judged that the authentication is successful. The authentication unit 83A judges that the authentication has failed when the memory unit 82A does not store an image that matches at least one face among the plurality of faces.

The authentication unit 83A transmits the authentication result to the lobby walkie-talkie 1A via the communication unit 81A.

(2-6) Control device The control device 3A will be described. The control device 3A has a communication interface for communicating with the hall walkie-talkie 1A and the walkie-talkie master 2A.

When the control device 3A receives the call signal, the image for display, and the voice signal of the visitor's voice from the lobby intercom 1A, it transmits it to the main intercom 2A. When the control device 3A receives the voice signal of the voice accepted by the main intercom 2A, it transmits it to the lobby intercom 1A.

The control device 3A further controls the locking and unlocking of the door of the common part. For example, when the control device 3A receives the unlocking signal from the main unit 2A of the walkie-talkie, it unlocks the door of the common part, and locks the door after a certain time has elapsed. In addition, when the control device 3A receives the unlock signal from the lobby intercom 1A, it unlocks the door of the common part, and locks the door after a predetermined time has elapsed. In addition, the control of locking and unlocking the door of the common part can be performed by the lobby intercom 1A.

(3) Action (3-1) Action of the walkie-talkie in the lobby Here, the operation of the lobby walkie-talkie 1A will be described with reference to FIG. 19 .

The imaging unit 11A of the lobby walkie-talkie 1A starts imaging when the sensor unit 18A detects a person (step S21).

After the imaging unit 11A starts capturing images, the control unit 13A determines whether the operation unit 15A accepts a call operation within a predetermined time (step S22).

When it is determined that the operation unit 15A has accepted the call operation within a certain period of time ("Yes" in step S22), the lobby intercom 1A performs call processing (step S23). Specifically, the lobby walkie-talkie 1A transmits a control signal (call signal) and an image for display to the master walkie-talkie 2A. Then, when the control unit 13A is in a state of being able to communicate with the intercom base unit 2A, if it receives the visitor's voice data (audio signal) from the microphone of the communication unit 16A, it sends the received audio signal to the intercom base unit. 2A. When the control unit 13A is in a state of being able to communicate with the main intercom 2A, if it receives an operator's voice signal from the main intercom 2A, it outputs the received audio signal to the speaker of the communication unit 16A. At this time, the speaker outputs sound based on the received sound signal.

When it is determined that the operation unit 15A has not yet accepted the call operation (“No” in step S22 ), the control unit 13A determines whether or not a predetermined time has elapsed (step S24 ).

When it is determined that the predetermined time has not elapsed (NO in step S24 ), the process waits for the operation of the operation unit 15A to be accepted.

When the control unit 13A determines that the predetermined time has elapsed (YES in step S24), the image processing device 120A transmits the image for authentication to the server 8A (step S25).

After sending the image, the control unit 13A receives the authentication result of the authentication performed by the server 8A (step S26).

After the call processing in step S23, the control device 3A of the walkie-talkie system 10A performs an opening and closing process according to the operation of the walkie-talkie main unit 2A (step S27). Specifically, when the control device 3A receives the unlocking signal from the main unit 2A of the walkie-talkie, it unlocks the door of the common part, and locks the door after a predetermined time has elapsed. In addition, after receiving the authentication result in step S26, the control device 3A also performs the opening and closing process of step S27 according to the instruction of the lobby intercom 1A. Specifically, when the control device 3A receives the unlock signal from the lobby intercom 1A, the door of the common part is unlocked, and after a predetermined time has elapsed, the door is locked.

(3-2) Operation example 1 Here, an example of operation at the time of authentication will be described with reference to FIG. 20 .

The imaging unit 11A of the lobby walkie-talkie 1A starts imaging when the sensor unit 18A detects a person (step S31 ).

The imaging unit 11A outputs the captured image (captured image) to the image processing device 120A (step S32).

The image acquisition unit 121A of the image processing device 120A receives the captured image and performs common processing (step S33). The image acquisition unit 121A performs processing such as RGB Bayer and two-stage exposure on the received captured image.

The image acquisition unit 121A outputs the image subjected to the common processing to the first processing unit 1221 (step S34).

The first processing unit 1221 performs the first image processing on the image subjected to the common processing (step S35). Specifically, the first processing unit 1221 captures the face region of the person included in the captured image. The first processing unit 1221 performs at least one of distortion correction and color tone correction according to the extracted face region to generate an image for authentication. In addition, when a plurality of face regions are extracted, the first processing unit 1221 performs at least one of distortion correction and tone correction for each extracted face region according to the face region, and generates an authentication image.

The first output unit 1271 transmits the authentication image generated by the first processing unit 1221 to the server 8A via the network NT1 (step S36).

The server 8A performs authentication processing using the authentication image received from the lobby intercom 1A (step S37).

After performing the authentication process, the server 8A transmits the authentication result to the lobby intercom 1A. The control unit 13A of the lobby walkie-talkie 1A controls the opening and closing of the door of the common unit based on the authentication result (refer to steps S26 and S27 in FIG. 19 ).

(3-3) Action example 2 Here, an operation example when the main unit 2 of the walkie-talkie device 2 displays an image will be described with reference to FIG. 21 .

The imaging unit 11A of the lobby walkie-talkie 1A starts imaging when the sensor unit 18A detects a person (step S41 ).

The imaging unit 11A outputs the captured image (captured image) to the image processing device 120A (step S42).

The image acquisition unit 121A of the image processing apparatus 120A receives the captured image and performs common processing (step S43). The image acquisition unit 121A performs processing such as RGB Bayer and two-stage exposure on the received captured image.

The image acquisition unit 121A outputs the image subjected to the common processing to the second processing unit 1222 (step S44).

The second processing unit 1222 performs second image processing on the image subjected to the common processing (step S45). Specifically, the second processing unit 1222 captures the face region of the person included in the captured image. The second processing unit 1222 performs at least one of distortion correction and color tone correction according to the extracted face area and the number thereof, and generates an image for display. In addition, when capturing a plurality of face regions, the second processing unit 1222 performs at least one of distortion correction and tone correction according to the face region for each captured face region, and generates a display image.

The second output unit 1272 transmits the display image generated by the second processing unit 1222 to the main walkie-talkie unit 2A via the control device 3A (step S46).

The main unit 2A of the walkie-talkie performs display processing (step S47). Specifically, the main unit 2A of the walkie-talkie causes the display unit 23A to display the display image received from the lobby walkie-talkie 1A, and performs call processing (see step S26 in FIG. 19 ). If the control device 3A receives the unlock signal from the main unit 2A of the walkie-talkie, it will unlock the door of the common part, and after a certain time has elapsed, lock the door (refer to step S27 in FIG. 19 ).

(4) Advantages As described above, the image processing apparatus 120A includes the first processing unit 1221 and the second processing unit 1222, and based on the image captured by the image capturing unit 11A, generates images (images for authentication and display) according to the application image).

With this configuration, even when images captured by the imaging unit 11A are used for a plurality of applications, images most suitable for each application can be provided.

In addition, each of the first processing unit 1221 and the second processing unit 1222 is realized by an integrated circuit different from each other. Therefore, of course, images corresponding to various purposes can be generated at different timings, and parallel processing can be easily realized even at the same timing.

Furthermore, each of the first processing unit 1221 and the second processing unit 1222 sets different parameters for the same processing in order to generate an image corresponding to each application. In addition, each of the first processing unit 1221 and the second processing unit 1222 executes different processing in order to generate an image corresponding to each use. Therefore, each of the first processing unit 1221 and the second processing unit 1222 can generate an appropriate image corresponding to each application.

(5) Variation example Hereinafter, a modified example will be described. In addition, the modification examples described below can be applied in combination with the above-mentioned second embodiment as appropriate.

(5-1) The first modification example In the second embodiment, the radio base unit 2A will be described as an example of the information terminal of the present disclosure. However, the information terminal is not limited to the main unit 2A of the walkie-talkie. The information terminal may be a terminal capable of communicating with the hall walkie-talkie 1A (walkie-talkie device), such as a management room host computer installed in the administrator's room, a smart phone or a tablet terminal used by residents.

Information terminals such as smart phones and tablet terminals can also be configured to communicate with the lobby walkie-talkie 1A via the walkie-talkie master 2A. Alternatively, information terminals such as smart phones and tablet terminals may be configured to be able to communicate via the control device 3A and the network NT1.

(5-2) Second modification example In the second embodiment, the second processing unit 1222 of the image processing apparatus 120A is configured to receive an instruction for processing to generate an image for display based on the operation of the main body 2A of the walkie-talkie as an information terminal. Alternatively, the second processing unit 1222 may be configured to receive a change in the content of the processing for generating the display image from the main unit 2A of the walkie-talkie. For example, when the second processing unit 1222 performs gradation correction as the color tone correction in the process of generating the display image, the main unit 2A of the walkie-talkie accepts an instruction to change to brightness correction as the color tone correction.

In addition, the second processing unit 1222 can receive an instruction of processing change from an information terminal such as a smart phone used by a resident.

The first processing unit 1221 may be configured so as not to accept changes in the content of the processing for generating the display image from terminals such as the main unit of the walkie-talkie 2A and the smartphone. Thereby, it is possible to maintain a high-precision image with respect to the image for authentication.

(5-3) The third modification example In the second embodiment, as the use of the image captured by the imaging unit 11A, the case for authentication and the case for display are described as examples, but the use is not limited to these.

The image captured by the imaging unit 11A can also be used for other purposes. For example, the image captured by the imaging unit 11A can also be used for monitoring. When the image is used for monitoring, the image processing device 120A performs at least one of distortion correction and color tone correction on the image captured by the imaging unit 11A to make a person's face clearer, as in the case of authentication.

Furthermore, the sending destination of the image processed by the image processing device 120A is not limited to any one of the server 8A and the main unit 2A of the walkie-talkie. You can also change the sending destination of the image according to the usage. For example, when the image captured by the imaging unit 11A is used for monitoring, the transmission destination of the image processed by the image processing device 120A may be the master computer of the management room installed in the management room.

(5-4) Fourth modification example In the second embodiment, the image processing apparatus 120A is configured to include two processing units (the first processing unit 1221 and the second processing unit 1222 ) configured by different integrated circuits, but it is not limited to this configuration .

The image processing apparatus 120A may be provided with three or more processing units constituted by mutually different integrated circuits. For example, in the case where the image processing apparatus 120A includes three processing units composed of different integrated circuits, the first processing unit 1221 generates an image for authentication, the second processing unit 1222 generates an image for display, and the third processing unit 1222 generates an image for display. The processing unit generates an image for object detection. Here, the object detection means, for example, detecting a situation where a person enters or leaves a room. The third processing unit performs at least one of distortion correction and color tone correction on the image captured by the imaging unit 11A to make the shape of a person clear.

In addition to object detection, the third processing unit may generate images for recording, crime prevention, and the like as an image application.

(5-5) Fifth modification example In the second embodiment, the lobby walkie-talkie 1A has a configuration including two communication units (19A of the first communication unit and 14A of the second communication unit), but it is not limited to this configuration.

The lobby walkie-talkie 1A may be configured to include one communication unit. In this case, the first processing unit 1221 transmits the generated authentication image to the server 8A via the one communication unit and the control device 3A. The second processing unit 1222 transmits the generated display image to the main walkie-talkie unit 2A via the one communication unit and the control device 3A.

(5-6) Sixth Variation Example In the second embodiment, it is premised that image display and authentication are performed at separate timings, and image display and authentication may be performed at the same timing. That is, the first processing unit 1221 and the second processing unit 1222 can operate in parallel. Since the first processing unit 1221 and the second processing unit 1222 are constituted by mutually different integrated circuits, they can be easily operated in parallel.

(5-7) Seventh Variation Example In the second embodiment, face authentication has been described as an example of image authentication. Image authentication is not limited to face authentication. For example, the image authentication may be the contour authentication of a person's face, or the brilliance authentication of the pupil. For example, in the case of performing contour authentication, the image processing device 120A performs at least one of distortion correction and color tone correction on the image captured by the imaging unit 11A to make the contour of the face clear.

(5-8) Eighth Variation Example Hereinafter, other modification examples are listed.

In the second embodiment, the image acquisition unit 121A has a configuration in which common processing related to image processing is performed, but it is not limited to this configuration. Common processing may be performed individually by each of the first processing unit 1221 and the second processing unit 1222 .

Furthermore, in the second embodiment, the first processing unit 1221 and the second processing unit 1222 are constituted by mutually different integrated circuits, but one integrated circuit may be constituted by a dual core including a plurality of arithmetic processing circuits.

In addition, in the second embodiment, the hall walkie-talkie 1A is provided with the image processing device 120A, but it is not limited to this configuration. The image processing device 120A may also be provided in the casing separately from the lobby intercom 1A. That is, the image processing apparatus 120A can be installed in the walkie-talkie system 10A as a different apparatus from the hall walkie-talkie 1A. When the image processing apparatus 120A is installed in the walkie-talkie system 10A as a different apparatus from the hall walkie-talkie 1A, the image processing apparatus 120A can acquire images from the hall sub-machine 9A having an imaging department. In this case, the hall sub-unit 9A having the camera unit corresponds to the walkie-talkie device of the present disclosure.

In addition, the image processing device 120A may be installed in the hall sub-machine 9A having an imaging unit. In this case, the hall sub-machine 9A generates an image corresponding to the purpose based on the image captured by the imaging unit it has. Therefore, it can also be applied to a walkie-talkie system in which the image processing device 120A is installed in an independent house. In this case, the hall sub-unit 9A having the camera unit corresponds to the walkie-talkie device of the present disclosure.

In addition, the image processing device 120A can also be applied to systems other than the walkie-talkie system 10A. For example, the image processing device 120A can be applied to a system that performs authentication and image display.

In addition, the main unit 2A of the walkie-talkie may display the authentication image when the authentication image is generated.

Moreover, the lobby intercom 1A and the server 8A can communicate via the network NT1 and the control device 3A.

(5-9) Other Variations The above-described embodiment is only one of various implementations of the present disclosure. As long as the above-described embodiment can achieve the purpose of cost disclosure, various changes can be made in accordance with the design and the like. In addition, the same function as the image processing system 12A can be embodied in an image information output method, a computer program, a non-transitory recording medium for recording the program, and the like. An image information output method of one aspect includes an image acquisition step, a first processing step, a second processing step, a first output step, and a second output step. The image acquisition step acquires an image captured by the imaging unit 11A of the walkie-talkie device (the lobby walkie-talkie 1A) having the imaging unit 11A. The first processing step generates first image information for use in the first application based on the image. The second processing step generates second image information for the second use based on the image. The first output step outputs the first image information. The second output step outputs the second image information. A program of one form is a program for making a computer system function as the above-mentioned image information output method.

In addition, the respective configurations of the first embodiment, the second embodiment, and the respective modifications can be appropriately combined with each other.

(Summarize) As described above, the image processing system (12, 12A) of the first aspect includes an image acquisition unit (121, 121A) that acquires the image captured by the imaging unit (11, 11A) of the walkie-talkie device (1, 1A). photographed image data of a subject; and an image processing unit (122, 122A) that performs image processing on the photographed image. The image processing unit (122, 122A) performs viewpoint transformation distortion correction in which the viewpoint of the captured image changes by the transformation angle (θb) relative to the subject by deforming the captured image. The conversion angle (θb) is larger than 0 degree and smaller than the angle of the walkie-talkie device (1) with respect to the vertical plane, that is, the inclination angle (θ1).

The above-mentioned image processing system (12, 12A) can suppress image distortion irrespective of the posture of the visitor (6, U1) to be photographed.

Moreover, the image processing system (12, 12A) of the second aspect of the embodiment is the same as that of the first aspect, wherein the image processing unit (122, 122A) is preferably the image processing unit (122, 122A) to further perform the image processing unit (11, 11A). Lens distortion correction for reducing lens distortion produced by the lens (112) in a captured image.

The above-mentioned image processing system (12, 12A) can correct the lens distortion caused by the lens, thereby further suppressing the image distortion.

Moreover, it is preferable that the image processing system (12) of the 3rd aspect of embodiment is further provided with the mode switching part (124) like the 2nd aspect. The mode switching unit (124) switches the operation mode of the image processing unit (122) between a first mode for performing viewpoint conversion distortion correction and lens distortion correction, and a second mode for performing lens distortion correction without performing viewpoint conversion distortion correction either.

The above-mentioned image processing system (12) can suppress image distortion regardless of the posture of the visitor (6) to be photographed as long as the walkie-talkie device (1) is installed obliquely. Moreover, if the walkie-talkie device (1) is installed along the vertical surface, the computational load of image processing can be reduced compared with the first mode.

Moreover, the image processing system (12) of the fourth aspect of the embodiment is as any one of the first to third aspects, and preferably further includes a degree selection unit (125). The image processing unit (122) may perform a plurality of viewpoint conversion distortion corrections corresponding to each of the plurality of conversion angles (θb) as viewpoint conversion distortion corrections. The degree selection unit (125) selects any one of a plurality of degrees corresponding to the plurality of conversion angles ( θb1 to θb3 ) as a set degree. An image processing unit (122) performs viewpoint transition distortion correction corresponding to a set degree among the plurality of viewpoint transition distortion corrections.

The above-mentioned image processing system (12) can suppress the distortion of the image irrespective of the size of the tilt angle θ1.

Furthermore, the image processing system (12) of the fifth aspect of the embodiment is as in any one of the first to third aspects, wherein the image processing unit (122) is preferably determined according to the inclination angle (θ1) Conversion angle (θb).

The above-mentioned image processing system (12) can suppress the distortion of the image irrespective of the size of the tilt angle θ1.

Furthermore, the image processing system (12) of the sixth aspect of the embodiment includes: an image acquisition unit (121) that acquires data of a captured image captured by the imaging unit (11) of the walkie-talkie device (1); the image A processing unit (122) performs image processing on the captured image; and an information acquisition unit (126) acquires input information. An image processing unit (122) corrects the distortion of the captured image based on the input information.

The above-mentioned image processing system (12) can suppress the distortion of the image regardless of the posture of the visitor (6) to be photographed.

Furthermore, in the image processing system (12) of the seventh aspect of the embodiment, as in the sixth aspect, it is preferable that the image processing unit (122) corrects the distortion of the captured image without changing the viewing angle of the captured image.

The above-mentioned image processing system (12) can suppress the distortion of the image without reducing the range of the space that can be confirmed in the captured image.

In addition, the image processing system (12) of the eighth aspect of the embodiment is the sixth or seventh aspect, wherein preferably the input information includes the angle of the walkie-talkie device (1) relative to the vertical plane, that is, the inclination angle (θ1) information.

The above-mentioned image processing system (12) can suppress the distortion of the image according to the size of the tilt angle (θ1).

Furthermore, in the image processing system (12) of the ninth aspect of the embodiment, as in any one of the sixth to eighth aspects, it is preferable that the input information includes the state where the walkie-talkie device (1) is installed. Height position information of the camera part (11).

The above-mentioned image processing system (12) can suppress image distortion according to the height position of the imaging unit (11).

Furthermore, the image processing system (12) of the tenth aspect of the embodiment is any one of the first to ninth aspects, wherein preferably the walkie-talkie device (1) is a hall walkie-talkie (1) of a facility.

The above-mentioned image processing system (12) can suppress image distortion regardless of the posture of the visitor (6) visiting the hall.

In addition, the image processing system (12) of the eleventh aspect of the embodiment is as in any one of the first to tenth aspects, wherein the image processing unit (122) is preferably an image processing unit (122) to further perform the color tone of the captured image Correction.

The image processing system (12) described above makes it easy to see images.

Furthermore, the image processing system (12A) of the twelfth aspect of the embodiment is the same as the first aspect, wherein the image processing unit (122A) preferably includes a first processing unit (1221) and a second processing unit (1222) . The first processing unit (1221) generates first image information used for the first purpose based on the captured image. The second processing unit (1222) generates second image information used for the second purpose based on the captured image. At least one of the first processing unit (1221) and the second processing unit (1222) performs viewpoint conversion distortion correction.

In the above image processing system (12A), the first processing unit (1221) generates first image information for use in the first application, and the second processing unit (1222) generates second image information for use in the second application. Therefore, even when the image captured by the imaging unit (11) is used in a plurality of applications, an image most suitable for each application can be provided.

Furthermore, the image processing system (12A) of the thirteenth aspect of the embodiment is the same as the sixth aspect, wherein the image processing unit (122A) preferably includes a first processing unit (1221) and a second processing unit (1222) . The first processing unit (1221) generates first image information used for the first purpose based on the captured image. The second processing unit (1222) generates second image information used for the second purpose based on the captured image.

In the above image processing system (12A), the first processing unit (1221) generates first image information for use in the first application, and the second processing unit (1222) generates second image information for use in the second application. Therefore, even when the image captured by the imaging unit (11) is used in a plurality of applications, an image most suitable for each application can be provided.

In addition, the image processing system (12A) of the fourteenth aspect of the embodiment is preferably further provided in the twelfth or thirteenth aspect, further comprising: a first output unit (1271) that outputs the first image information; and The second output unit (1272) outputs the second image information.

In the above-mentioned image processing system (12A), even when the image captured by the imaging unit (11) is used for a plurality of purposes, an image most suitable for each purpose can be provided.

In addition, the image processing system (12A) of the fifteenth aspect of the embodiment is any one of the twelfth to fourteenth aspects, wherein preferably the first processing unit (1221) and the second processing unit (1222) ) are treated differently.

The above-mentioned image processing system (12A) can perform appropriate processing according to the usage.

In addition, the image processing system (12A) of the sixteenth aspect of the embodiment is any one of the twelfth to fourteenth aspects, wherein preferably the first processing unit (1221) and the second processing unit (1222) ) with different parameters for the same processing.

In the above-mentioned image processing system (12A), suitable processing according to the application can be performed.

In addition, the image processing system (12A) of the seventeenth aspect of the embodiment is any one of the twelfth to sixteenth aspects, wherein preferably the first processing unit (1221) and the second processing unit (1222) ) each performs processing including tone correction.

In the above-mentioned image processing system (12A), suitable processing according to the application can be performed.

In addition, the image processing system (12A) of the eighteenth aspect of the embodiment is any one of the twelfth to seventeenth aspects, wherein preferably the first processing unit (1221) and the second processing unit (1222) ) The processing performed by each of them is performed for a partial area of the captured image.

The above-mentioned image processing system (12A) can perform appropriate processing corresponding to the use purpose to the required area when using the image.

In addition, in the image processing system (12A) of the nineteenth aspect of the embodiment, as in the eighteenth aspect, it is preferable that the first processing unit (1221) be used when a plurality of regions exist as a part of the captured image. Each of the second processing unit (1222) performs processing corresponding to each area.

In the above-mentioned image processing system (12A), when an image is used, appropriate processing corresponding to the use purpose can be performed for each of the plurality of areas required.

Furthermore, in the image processing system (12A) of the twentieth aspect of the embodiment, as in any one of the twelfth to nineteenth aspects, preferably the first use includes image authentication using the first image information .

In the above-mentioned image processing system (12A), the first processing unit (1221) generates the first image information by performing processing suitable for image authentication on the image captured by the imaging unit (11A), so that the image can be improved Accuracy of certification.

Furthermore, in the image processing system (12A) of the 21st aspect of the embodiment, as in the 20th aspect, it is preferable that the image authentication includes face authentication.

The above image processing system (12A) can improve the accuracy of face authentication.

In addition, the image processing system (12A) of the 22nd aspect of the embodiment is any one of the 12th to 21st aspects, wherein preferably the second use includes an information terminal (for example, an information terminal that communicates with a walkie-talkie device) Display on the main unit of the walkie-talkie (2A).

In the above image processing system (12A), the second processing unit (1222) generates second image information by performing processing suitable for image display on the image captured by the imaging unit (11A), so that the information terminal can be Display images that are easy for readers to see.

Furthermore, in the image processing system (12A) of the 23rd aspect of the embodiment, as in the 22nd aspect, preferably the first application includes image authentication using the first image information. The first processing unit ( 1221 ) accepts an instruction to change the processing content without operating the information terminal (for example, the main unit 2A of the walkie-talkie). The second processing unit (1222) accepts an instruction to change the processing content based on the operation of the information terminal.

The above-mentioned image processing system (12A) can change the second image information generated by the second processing unit (1222) according to the instruction of the viewer to be easy for the viewer to see. Furthermore, since the first image information used for image authentication cannot be changed, high-precision authentication can be maintained.

In addition, the image processing system (12A) of the 24th aspect of the embodiment is any one of the 12th to 23rd aspects, wherein preferably the first processing unit (1221) and the second processing unit (1222) ) are composed of different integrated circuits.

The above-mentioned image processing system (12A) can also provide an image most suitable for each application when the image captured by the imaging unit (11A) is used for a plurality of applications.

Furthermore, the image processing method according to the twenty-fifth aspect of the embodiment includes: an image acquisition step (S1) for acquiring a captured image of the subject captured by the imaging unit (11, 11A) of the walkie-talkie device (1, 1A) and the image processing step (S2), which performs image processing on the captured image. The image processing step ( S2 ) performs viewpoint transformation distortion correction in which the viewpoint of the captured image is changed with respect to the subject by the transformation angle (θb) by deforming the captured image. The conversion angle (θb) is larger than 0 degree and smaller than the angle of the walkie-talkie device (1) with respect to the vertical plane, that is, the inclination angle (θ1).

The above-mentioned image processing method can suppress image distortion irrespective of the posture of the client (6, U1) being photographed.

In addition, the image processing method of the twenty-sixth aspect of the embodiment includes: an image acquisition step (S11) for acquiring data of the captured image captured by the imaging unit (11, 11A) of the walkie-talkie device (1, 1A); information An acquisition step ( S12 ) for acquiring input information; and an image processing step ( S13 ) for performing image processing on the captured image. The image processing step (S13) corrects the distortion of the captured image based on the input information.

The above-mentioned image processing method can suppress image distortion irrespective of the posture of the client (6, U1) being photographed.

Furthermore, the program of the twenty-seventh aspect of the embodiment enables a computer to execute the image processing method of the twenty-fifth or twenty-sixth aspect.

The above procedure can suppress image distortion irrespective of the posture of the client (6, U1) being photographed.

Furthermore, the image processing system (12A) according to the twenty-eighth aspect of the embodiment includes an image acquisition unit (121A), a first processing unit (1221), a second processing unit (1222), and a first output unit (1271) and the second output unit (1272). An image acquisition unit (121A) acquires an image captured by an imaging unit (11A) of a walkie-talkie device (for example, a hall walkie-talkie 1A) having an imaging unit (11A). The first processing unit (1221) generates first image information (eg, an image for authentication) to be used for the first purpose based on the image. The second processing unit (1222) generates second image information (eg, display image) used for the second purpose based on the image. The first output unit (1271) outputs the first image information. The second output unit (1272) outputs the second image information.

In the above image processing system (12A), the first processing unit (1221) generates first image information used for the first purpose, and the second processing unit (1222) generates second image information used for the second purpose. Therefore, even when the image captured by the imaging unit (11A) is used for a plurality of applications, an image most suitable for each application can be provided.

Furthermore, in the image processing system (12A) of the twenty-ninth aspect, as in the twenty-eighth aspect, it is preferable that the first processing unit (1221) and the second processing unit (1222) perform different processing.

The above-mentioned image processing system (12A) can perform appropriate processing according to the usage.

Furthermore, in the image processing system (12A) of the 30th aspect, as in the 28th aspect, preferably the first processing unit (1221) and the second processing unit (1222) perform the same processing with different parameters.

In the above-mentioned image processing system (12A), suitable processing according to the application can be performed.

In addition, the image processing system (12A) of the 31st aspect is any one of the 28th to 30th aspects, preferably each of the first processing unit (1221) and the second processing unit (1222) The processing performed includes distortion correction.

In the image processing system (12A) described above, appropriate corrections can be performed according to the application.

Furthermore, in the image processing system (12A) of the 32nd aspect, as in any one of the 28th to 31st aspects, preferably the first processing unit (1221) and the second processing unit (1222) are each The processing performed by the user includes tone correction.

The above-mentioned image processing system (12A) can perform appropriate correction according to the usage.

In addition, the image processing system (12A) of the 33rd aspect is any one of the 28th to 32nd aspects, preferably each of the first processing unit (1221) and the second processing unit (1222) The processing performed is performed for a partial area of the captured image.

The above-mentioned image processing system (12A) can perform appropriate processing corresponding to the usage to the required area when using the captured image.

Furthermore, in the image processing system (12A) of the thirty-fourth aspect, as in the thirty-third aspect, it is preferable that the first processing unit (1221) and the second processing unit (1221) and the second processing unit (1221) and the second Each of the processing units (1222) performs processing corresponding to each area.

In the above-mentioned image processing system (12A), when a captured image is used, an appropriate process corresponding to the use purpose can be performed for each of the plurality of regions required.

Furthermore, in the image processing system (12A) of the thirty-fifth aspect, according to any one of the twenty-eighth to the thirty-second aspects, preferably the first use includes image authentication using the first image information.

In the above-mentioned image processing system (12A), the first processing unit (1221) generates the first image information by performing processing suitable for image authentication on the image captured by the imaging unit (11A), so that the image can be improved Accuracy of certification.

Furthermore, the image processing system (12A) of the thirty-sixth aspect is as in the thirty-fifth aspect, wherein preferably the image authentication includes face authentication.

In the above image processing system (12A), the accuracy of face authentication can be improved.

Also, the image processing system (12A) of the thirty-seventh aspect is any one of the twenty-eighth to thirty-sixth aspects, wherein preferably the second use includes an information terminal (for example, an interphone master ( 2A)) in the display.

In the above-mentioned image processing system (12A), the second processing unit (1222) generates second image information by performing processing suitable for the display of the captured image on the image captured by the imaging unit (11A), so that the information can be The terminal displays images that are easy for readers to see.

Furthermore, in the image processing system (12A) of the thirty-eighth aspect, as in the thirty-seventh aspect, preferably the first use includes image authentication using the first image information. The first processing unit (1221) does not accept an instruction to change the processing content based on the operation of the information terminal (for example, the main unit 2A of the walkie-talkie). The second processing unit (1222) accepts an instruction to change the processing content based on the operation of the information terminal.

In the above-mentioned image processing system (12A), the second image information generated by the second processing unit (1222) can be changed to be easy for the viewer to see according to the instruction of the viewer. Furthermore, since the first image information used for image authentication cannot be changed, high-precision authentication can be maintained.

In addition, the image processing system (12A) of the thirty-ninth aspect is any one of the twenty-eighth to thirty-eighth aspects, wherein preferably the first processing part (1221) and the second processing part (1222) are different The integrated circuit composition.

The above-mentioned image processing system (12A) can also provide an image most suitable for each application when the image captured by the imaging unit (11A) is used for a plurality of applications.

Furthermore, the image information output method of the 40th aspect includes an image acquisition step, a first processing step, a second processing step, a first output step, and a second output step. The image acquisition step acquires an image captured by an imaging unit ( 11A ) of a walkie-talkie device (for example, a hall walkie-talkie 1A) having an imaging unit 11A. The first processing step generates first image information for use in the first application based on the image. The second processing step generates second image information for the second use based on the image. The first output step outputs the first image information. The second output step outputs the second image information.

The above-mentioned image information output method can also provide an image most suitable for each application when the image captured by the imaging unit (11A) is used for a plurality of applications.

Moreover, the program of the 41st aspect is a program for causing a computer to execute the image information output method of the 40th aspect.

The above-mentioned program can also provide the most suitable image for each application when the image captured by the imaging unit (11A) is used for a plurality of applications.

Furthermore, the image processing system (12A) of the 42nd aspect includes an image acquisition unit (121A), a first processing unit (1221), a second processing unit (1222), a first output unit (1271), and a second processing unit (1271) Output (1272). An image acquisition unit (121A) acquires an image. The first processing unit (1221) generates first image information suitable for face authentication based on the image. The second processing unit (1222) generates second image information suitable for viewing and display based on the image. The first output unit (1271) outputs the first image information. The second output unit (1272) outputs the second image information.

The above-mentioned image processing system (12A) can also provide an image most suitable for each application when the image captured by the imaging unit (11A) is used for a plurality of applications.

1: lobby walkie-talkie (walkie-talkie device) 1A: Lobby walkie-talkie (walkie-talkie device) 2: Walkie-talkie mother machine 2A: Walkie-talkie host (information terminal) 3: Control device 3A: Control device 4: Mounting table 5: Common space 6: Visitors 7: Wall 8A: Server 9A: Foyer handset (walkie-talkie device) 10: Walkie-talkie system 10A: Walkie-talkie system 11: Camera Department 11A: Camera Department 12: Image processing system 12A: Image Processing System 13: Slave Control Department 13A: Control Department 14: Slave Communication Department 14A: 2nd Communications Department 15: Slave operation department 15A: Operation Department 16: Slave phone call department 16A: Communication Department 17: Tilt sensor 18A: Sensor section 19A: 1st Communications Department 21: Main machine control department 21A: Control Department 22: Main engine communication department 22A: 1st Communications Department 23: Display part 23A: Display part 24: Mother machine operation department 24A: Operation Department 25: Main phone communication department 25A: Communication Department 26A: 2nd Communications Department 41: Mounting surface 61: Visitors 62: Visitors 81A: Communications Department 82A: Memory Department 83A: Certification Department 100: Shell 111: Camera element 112: Lens 120: Image processing device 120A: Image processing device 121: Image acquisition department 121A: Image Acquisition Section 122: Image Processing Department 122A: Image Processing Department 123: Memory Department 124: Mode switching section 125: Degree Selection Department 126: Information Acquisition Department 1221: 1st Processing Division 1222: Second Processing Division 1271: 1st output 1272: 2nd output part A1: Camera area C1: Chart C11: Horizontal line C12: Vertical line D1: Grid Pattern D2: Grid Pattern D3: Grid Pattern D4: Grid Pattern D5: Grid Pattern D11: horizontal line D12: Vertical line D21: horizontal line D22: Vertical line D31: horizontal line D32: Vertical line D41: horizontal line D42: Vertical line D51: horizontal line D52: Vertical line F1: Facilities G11: Camera image G21: Lens Correction Image G22: Lens Correction Image G23: Lens Correction Image G24: Lens Correction Image G25: Lens Correction Image G31: Correct Image G32: Correct Image G33: Correct Image G34: Correct Image G35: Correct Image H1: set height NT1: Network R24: Subject area R25: Subject area R31: Subject area R32: Subject area R33: Subject area R34: Subject area S1: Image acquisition step S2: Image processing step S11: Image acquisition step S12: Information acquisition steps S13: Image processing step S21～S27: Steps S31～S37: Steps S41～S47: Steps U1: Visitors W1: Transmission path W2: Teleportation path θa: relative angle θb: conversion angle θb1～θb3: Conversion angle θ1: Inclination angle θ11～θ13: Inclination angle

FIG. 1 is a block diagram showing the configuration of a walkie-talkie system including the image processing system of the first embodiment. FIG. 2 is a block diagram showing the constitution of the main unit of the walkie-talkie included in the above-mentioned walkie-talkie system. FIG. 3 is a side view showing a hall walkie-talkie set obliquely and a visitor standing upright in the above-mentioned walkie-talkie system. FIG. 4A is a diagram showing a grid-like graph used in the above-described image processing system. FIG. 4B is a diagram showing a photographed image of a grid-shaped graph. FIG. 5A is a diagram showing a lens-corrected image for correcting the longitudinal lens distortion in the above-mentioned captured image. FIG. 5B is a diagram showing a lens-corrected image for correcting the two-dimensional lens distortion in the above-mentioned captured image. FIG. 6 is a side view showing a state in which the above-mentioned hall walkie-talkie is installed along a vertical plane. FIG. 7 is a diagram showing a lens-corrected image of a grid-shaped graph captured in the first comparative example. FIG. 8A is a diagram showing a lens-corrected image of an upright client in the first comparative example. FIG. 8B is a diagram showing a lens-corrected image of a forward-leaning client in the first comparative example. FIG. 9 is a side view showing a hall walkie-talkie set obliquely and a visitor leaning forward in the above-mentioned walkie-talkie system. FIG. 10A is a diagram showing a lens-corrected image of an upright client in the second comparative example. FIG. 10B is a diagram showing a lens-corrected image of a forward-leaning client in the second comparative example. FIG. 11A is a diagram showing a lens-corrected image of the above-mentioned upright client. FIG. 11B is a diagram showing a lens-corrected image of a client leaning forward. FIG. 12 is a diagram showing a corrected image of the above-mentioned grid-shaped graph. FIG. 13 is a flowchart showing the above-mentioned image processing method. Fig. 14 is a side view showing the obliquely installed lobby intercom and the visitor in the second and third modifications described above. FIG. 15 is a flow chart showing the image processing method in the third modification above. Fig. 16 is a diagram illustrating the configuration of a hall walkie-talkie as a walkie-talkie device according to the second embodiment. FIG. 17 is a diagram illustrating the configuration of a walkie-talkie system including the above-mentioned hall walkie-talkie. FIG. 18A is a diagram showing an image captured by the camera of the above-mentioned lobby intercom. FIG. 18B is a diagram showing a state when the image shown in FIG. 18A is corrected to an image for authentication. FIG. 19 is a diagram illustrating the operation of the above-mentioned lobby intercom. FIG. 20 is a diagram illustrating the operation at the time of authentication in the above-mentioned walkie-talkie system. Fig. 21 is a diagram for explaining the operation of the above-mentioned walkie-talkie system at the time of display.

1: lobby walkie-talkie

2: Walkie-talkie mother machine

3: Control device

10: Walkie-talkie system

11: Camera Department

12: Image processing system

13: Slave Control Department

14: Slave Communication Department

15: Slave operation department

16: Slave phone call department

17: Tilt sensor

111: Camera element

112: Lens

120: Image processing device

121: Image acquisition department

122: Image Processing Department

123: Memory Department

124: Mode switching section

125: Degree Selection Department

126: Information Acquisition Department

W1: Transmission path

W2: Teleportation path

Claims (31)

An image processing system comprising: an image acquisition unit that acquires data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; an image processing unit that performs image processing on the captured image; and a mode switching unit; and the image processing unit performs viewpoint conversion distortion correction for changing the viewpoint of the captured image with respect to the subject at a conversion angle by deforming the captured image, and The lens distortion correction for reducing the lens distortion generated in the above-mentioned captured image; the above-mentioned conversion angle is greater than 0 degrees, and is smaller than the angle of the walkie-talkie device relative to the vertical plane, that is, the inclination angle; and the above-mentioned mode switching part The operation mode of the image processing unit is switched to either a first mode for performing the viewpoint conversion distortion correction and the lens distortion correction, and a second mode for performing the lens distortion correction without performing the viewpoint conversion distortion correction. An image processing system comprising: an image acquisition unit that acquires data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; an image processing unit that performs image processing on the captured image; and a degree selection unit; and the image processing unit performs deformation of the captured image by deforming the captured image Correction of viewpoint conversion distortion by changing the viewpoint with respect to the subject's viewpoint; the conversion angle is greater than 0 degrees and smaller than the angle of the walkie-talkie device relative to the vertical plane, that is, the inclination angle; and the image processing unit can perform corresponding A plurality of viewpoint conversion distortion corrections at a plurality of the above-mentioned conversion angles are used as the above-mentioned viewpoint conversion distortion correction; the above-mentioned degree selection unit selects any one of the plurality of degrees corresponding to the above-mentioned plurality of conversion angles as the setting degree. The image processing unit performs viewpoint conversion distortion correction corresponding to the above-mentioned set degree among the plurality of viewpoint conversion distortion corrections. An image processing system comprising: an image acquisition unit that acquires data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; and an image processing unit that performs image processing on the captured image and the above-mentioned image processing unit performs a viewpoint conversion distortion correction in which the viewpoint of the above-mentioned captured image relative to the above-mentioned subject is changed by a conversion angle by deforming the above-mentioned captured image; the above-mentioned conversion angle is greater than 0 degrees and less than 0 degrees. The angle of the walkie-talkie device relative to the vertical plane is an inclination angle; and the image processing unit determines the conversion angle according to the inclination angle. An image processing system comprising: an image acquisition unit that acquires data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; and an image processing unit that performs image processing on the captured image; and the image processing unit transforms the captured image with respect to a viewpoint of the subject at a conversion angle by deforming the captured image The viewpoint conversion distortion correction; the conversion angle is greater than 0 degrees, and is smaller than the angle of the walkie-talkie device relative to the vertical plane, that is, the inclination angle; and the image processing unit is provided with: a first processing unit that generates a first image information used in the first application; and a second processing unit that generates second image information used in the second application based on the captured image; and the first processing unit and the second processing unit have At least one of them performs the above-mentioned viewpoint conversion distortion correction. The image processing system according to claim 2 or 3, wherein the image processing unit further performs lens distortion correction for reducing lens distortion caused by the lens of the imaging unit in the captured image. The image processing system of claim 5, further comprising: a mode switching unit that switches the operation mode of the image processing unit to a first mode that performs the viewpoint conversion distortion correction and the lens distortion correction, and does not perform the above Any one of the second modes in which the viewpoint conversion distortion correction is performed to perform the above-mentioned lens distortion correction. The image processing system of claim 1, further comprising a degree selection unit; and The image processing unit may perform a plurality of viewpoint conversion distortion corrections corresponding to a plurality of the conversion angles as the viewpoint conversion distortion correction; the degree selection unit selects one of the plurality of degrees corresponding to the plurality of conversion angles. Any degree is set as a set degree; the image processing unit performs viewpoint conversion distortion correction corresponding to the set degree among a plurality of viewpoint conversion distortion corrections. The image processing system of claim 1, wherein the image processing unit determines the conversion angle according to the tilt angle. An image processing system comprising: an image acquisition unit that acquires data of a captured image captured by an imaging unit of a walkie-talkie device; an image processing unit that performs image processing on the captured image; and an information acquisition unit , which obtains input information; and the image processing unit corrects the distortion of the captured image based on the input information; and the image processing unit includes: a first processing unit that generates a first use based on the captured image used first image information; and a second processing unit that generates second image information used for the second purpose based on the above-mentioned captured image. The image processing system of claim 9, wherein The image processing unit corrects the distortion of the captured image without changing the viewing angle of the captured image. The image processing system according to claim 9 or 10, wherein the input information includes information about the angle of the walkie-talkie device relative to the vertical plane, that is, the inclination angle. The image processing system according to claim 9 or 10, wherein the input information includes height position information of the imaging unit in a state where the walkie-talkie device is installed. The image processing system according to any one of claims 1, 9 and 10, wherein the above-mentioned walkie-talkie device is a hall walkie-talkie of a facility. The image processing system according to any one of claims 1, 9, and 10, wherein the image processing unit further performs the tone correction of the captured image. The image processing system according to claim 4 or 9, further comprising: a first output unit that outputs the first image information; and a second output unit that outputs the second image information. The image processing system according to claim 4 or 9, wherein the first processing unit and the second processing unit perform different processing. The image processing system of claim 4 or 9, wherein the first processing unit and the second processing unit perform the same processing with different parameters. The image processing system according to claim 4 or 9, wherein the processing performed by the first processing unit and the second processing unit each includes color tone correction. The image processing system according to claim 4 or 9, wherein the processing performed by each of the first processing unit and the second processing unit is performed on a partial area of the captured image. The image processing system of claim 19, wherein when there are a plurality of areas in the captured image as the partial areas, the first processing unit and the second processing unit each perform processing corresponding to each area. The image processing system of claim 4 or 9, wherein the first use includes image authentication using the first image information. The image processing system of claim 21, wherein the image authentication includes face authentication. The image processing system of claim 4 or 9, wherein the second use includes display by an information terminal communicating with the walkie-talkie device. The image processing system of claim 23, wherein the first use includes image authentication using the first image information; and the first processing unit does not accept an instruction to change the processing content based on the operation of the information terminal; 2. The processing unit accepts an instruction to change the processing content based on the operation of the above-mentioned information terminal. The image processing system according to claim 4 or 9, wherein the first processing unit and the second processing unit are constituted by different integrated circuits. An image processing method, comprising: an image acquisition step of acquiring data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; an image processing step of performing image processing on the captured image; and a mode switching step; and the image processing step performs viewpoint conversion distortion correction for changing the viewpoint of the captured image with respect to the subject by a conversion angle by deforming the captured image, and The lens distortion correction for reducing the lens distortion generated in the above-mentioned captured image; the above-mentioned conversion angle is greater than 0 degrees, and less than the angle of the walkie-talkie device relative to the vertical plane, that is, the inclination angle; and the above-mentioned mode switching step, which will The operation mode of the above-mentioned image processing steps is switched to carry out Any one of the first mode of the viewpoint conversion distortion correction and the lens distortion correction, and the second mode of the lens distortion correction without performing the viewpoint conversion distortion correction. An image processing method, comprising: an image acquisition step of acquiring data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; an image processing step of performing image processing on the captured image; and a degree selection step; and the image processing step performs viewpoint conversion distortion correction in which the viewpoint of the captured image relative to the subject is changed at a conversion angle by deforming the captured image; the conversion angle is greater than 0 degree, and is smaller than the angle of the walkie-talkie device relative to the vertical plane, that is, the inclination angle; and the above-mentioned image processing step can perform a plurality of viewpoint conversion distortion corrections corresponding to a plurality of the above-mentioned conversion angles, as the above-mentioned viewpoint conversion distortion correction; the above degree The selecting step selects any one of the plurality of degrees corresponding to the plurality of conversion angles as the set degree; the image processing step performs the viewpoint conversion distortion correction corresponding to the set degree among the plurality of viewpoint conversion distortion corrections. An image processing method comprising: an image acquisition step of acquiring data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; and an image processing step of performing image processing on the captured image ;and The above-mentioned image processing step performs viewpoint conversion distortion correction by deforming the above-mentioned captured image to change the viewpoint of the above-mentioned captured image relative to the above-mentioned subject at a conversion angle; the above-mentioned conversion angle is greater than 0 degrees and smaller than the above-mentioned walkie-talkie. The angle of the device relative to the vertical plane is the inclination angle; and the image processing step determines the conversion angle according to the inclination angle. An image processing method comprising: an image acquisition step of acquiring data of a captured image of a subject captured by an imaging unit of a walkie-talkie device; and an image processing step of performing image processing on the captured image and the above-mentioned image processing step performs viewpoint conversion distortion correction in which the viewpoint of the above-mentioned photographed image relative to the above-mentioned subject is changed by a conversion angle by deforming the above-mentioned captured image; the above-mentioned conversion angle is greater than 0 degrees and less than 0 degrees. The angle of the walkie-talkie device with respect to the vertical plane is an inclination angle; and the image processing step includes: a first processing step for generating first image information used for a first purpose based on the captured image; and a second processing step of generating second image information for use in a second application based on the captured image; and performing the viewpoint conversion distortion correction in at least one of the first processing step and the second processing step. An image processing method comprising: An image acquisition step, which acquires data of a captured image captured by the imaging unit of the walkie-talkie device; an information acquisition step, which acquires input information; and an image processing step, which performs image processing on the above-mentioned captured image; The image processing step corrects the distortion of the captured image based on the input information; and the image processing step includes: a first processing step for generating first image information for a first purpose based on the captured image; and The second processing step is to generate second image information used for the second purpose based on the above-mentioned captured image. A program for causing a computer to execute the image processing method of any one of claims 26 to 30.

Images