JP2023128978A - Image processing device, image processing method, image processing program, and recording medium - Google Patents

Abstract

To provide an image processing device capable of performing appropriate image processing for a subject in a captured image.SOLUTION: An image processing device includes: an imaging information acquisition unit that obtains imaging information including an image captured by an imaging device, an imaging position of the image, and an imaging direction thereof; an attribute identification unit that identifies an attribute of a ground object in the captured image based on the imaging information and map information including information indicating the position of a ground object and the attribute of the ground object; and a target area identification unit that identifies an area where a ground object whose attribute identified by the attribute specification unit is a predetermined attribute is included in the captured image, as a target area for which image processing is to be performed.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing device, an image processing method, an image processing program, and a recording medium, and particularly relates to an image processing device, an image processing method, an image processing program, and a recording medium that perform image processing based on map information.

2. Description of the Related Art Apparatuses have been proposed that perform processing to protect personal information regarding objects captured in images taken by imaging devices.

For example, Patent Document 1 describes an imaging method in which a person's face, a predetermined object, etc. is detected as personal information included in an image taken by a camera through image recognition, and image processing is performed to make the personal information indistinguishable. An apparatus is disclosed.

Japanese Patent Application Publication No. 2021-177597

For example, one issue is that if an image of a private residence's nameplate is published on the Internet through social networking services (SNS), it may violate the privacy of the owner or resident of the residence. .

In order to prevent the above-mentioned invasion of privacy, it is possible to perform image processing such as making the image blurry, but this would also apply image processing to areas in the image that do not require image processing, making it difficult to restore the original image. One problem is that information in the original image may be excessively lost, such as landscape information being lost.

The present invention has been made in view of the above points, and one of the objects of the present invention is to provide an image processing device that can perform appropriate image processing on a subject in a photographed image. .

The invention according to claim 1 includes a photographing information acquisition unit that acquires photographing information including a photographed image photographed by an imaging device, a photographing position and a photographing direction of the photographed image, and a position of a feature and an attribute of the feature. an attribute specifying section that specifies an attribute of a feature in the photographed image based on map information including information indicating , and the photographing information; The image processing apparatus is characterized in that it includes a target area specifying unit that specifies an area in which a feature having a predetermined attribute is photographed as a target area for image processing.

The invention according to claim 12 is an image processing method executed by an image processing device, the method comprising: acquiring photographic information including a photographed image photographed by an imaging device, a photographing position and a photographing direction of the photographed image; an attribute specifying step of identifying the attribute of the feature in the captured image based on the acquisition step, map information including information indicating the location of the feature and the attribute of the feature, and the imaging information; The present invention is characterized by including a target area specifying step of specifying, in the image, an area in which a feature having a predetermined attribute specified by the attribute specifying unit is shown as a target area for image processing. This is an image processing method.

The invention according to claim 13 is an image processing program executed by an image processing device including a computer, wherein the computer includes a photographed image photographed by an imaging device, a photographing position and a photographing direction of the photographed image. a photographic information acquisition step of acquiring photographic information; identifying the attribute of the feature in the photographed image based on the photographic information and map information including information indicating the position of the feature and the attribute of the feature; an attribute specifying step; a target area specifying step of specifying, in the photographed image, an area in which a feature whose attribute is a predetermined attribute specified by the attribute specifying unit is captured as a target area for image processing; This is an image processing program for executing.

The fourteenth aspect of the invention provides a photographic information acquisition step of acquiring photographic information including a photographic image photographed by an imaging device, a photographing position and a photographing direction of the photographic image, in an image processing apparatus including a computer; an attribute specifying step of identifying an attribute of a feature in the captured image based on map information including information indicating the location of the feature and an attribute of the feature, and the imaging information; A computer-readable computer that stores an image processing program that executes a target area identifying step of identifying an area in which a feature whose attribute is a predetermined attribute identified by the identifying unit as a target area for image processing. It is a storage medium.

1 is a diagram showing an image processing system 100 according to an example. 1 is a diagram showing a front seat portion of a vehicle in which a terminal device according to an embodiment is mounted. FIG. 2 is a block diagram showing the configuration of a terminal device according to an embodiment. It is a figure showing an example of information included in photographing information concerning an example. FIG. 1 is a block diagram showing the configuration of an image processing device according to an embodiment. It is a figure showing an example of map information concerning an example. FIG. 2 is a top view showing the positional relationship between a photographing position and features around a vehicle according to an embodiment. FIG. 3 is a diagram showing an example of a reproduced image according to an example. FIG. 7 is a diagram illustrating an example of setting attributes used to specify a target area according to an embodiment. It is a figure showing an example of the photographed image concerning an example. FIG. 3 is a diagram showing an example of a photographed image after image processing according to the example. 7 is a flowchart illustrating an example of a routine executed by the image processing apparatus according to the embodiment when identifying attributes of image processing. 7 is a flowchart illustrating an example of a routine executed by the image processing apparatus according to the embodiment when identifying a target area for image processing. 2 is a flowchart illustrating an example of a routine executed by the image processing apparatus according to the embodiment when performing image processing. 2 is a flowchart illustrating an example of a routine executed by the image processing apparatus according to the embodiment when performing image processing. It is a figure showing an example of the photographed image concerning an example. FIG. 3 is a diagram showing an example of a photographed image after image processing according to the example. 2 is a flowchart illustrating an example of a routine executed by the image processing apparatus according to the embodiment when performing image processing. FIG. 3 is a diagram showing an example of a photographed image after image processing according to the example.

Examples of the present invention will be described in detail below. In the following description and accompanying drawings, substantially the same or equivalent parts are designated by the same reference numerals.

The configuration of an image processing system 100 including an image processing device 10 according to an embodiment will be described with reference to the accompanying drawings.

FIG. 1 is a diagram schematically showing an image processing system 100. As shown in FIG. 1, the image processing system 100 includes an image processing device 10 and a terminal device 20 mounted on a vehicle M.

The image processing device 10 is a server device that communicates with the terminal device 20 via the network NW. The image processing device 10 and the terminal device 20 are capable of mutually transmitting and receiving data using a communication protocol such as TCP/IP, for example. Connection between the terminal device 20 and the network NW can be made, for example, by mobile communication such as 4G (4th Generation) or 5G (5th Generation), or wireless communication such as Wi-Fi (registered trademark).

The image processing device 10 receives an image taken by a camera attached to the vehicle M from the terminal device 20, and performs processing such as processing to blur the image for privacy protection or processing to emphasize a landmark in the image. A target area to be processed is specified, predetermined image processing is performed on the target area in the image, and the image after the image processing is stored.

The terminal device 20 transmits an image taken by a camera attached to the vehicle M to the image processing device 10.

For example, when a user of the image processing system 100 publishes an image on the Internet via SNS (Social Networking Service) or an electronic map, the image processing system 100 provides an image that has been processed to protect privacy. It is a system that

For example, the user of the image processing system 100 can publish an image in which a nameplate of a private residence is reflected, and the image is processed to make the part where the nameplate is blurred, and the nameplate can be viewed. There is no need to infringe on the privacy of the owners or residents of the listed homes.

Further, for example, if the user of the image processing system 100 does not want the location of the photograph to be specified, processing may be performed on an image in which a commercial facility is reflected to make the portion of the image in which the commercial facility is blurred. Images can be made public and users' own privacy can be protected.

FIG. 2 is a diagram showing the front seat portion of the vehicle M. As shown in FIG. 2, the terminal device 20 is placed in the center console of the vehicle M. In this embodiment, the terminal device 20 has a route generation function and a navigation function (so-called car navigation function). Note that the terminal device 20 itself does not need to have a car navigation function. In that case, the terminal device 20 may be connected to a car navigation device mounted on the vehicle M so as to be able to acquire position information of the vehicle M.

The front camera 11 is provided on the dashboard DB. The front camera 11 uses the traveling direction of the vehicle M as its photographing direction. When the traveling direction of the vehicle M is set as the front, the front camera 11 is arranged so as to be able to take images of a range including the front, right front, and left front of the vehicle M.

The front camera 11 may be provided at any location on the vehicle M as long as it can capture images in front of the vehicle M. For example, the front camera 11 may be disposed on the back of the rearview mirror RM, that is, on the surface facing the windshield FG, or may be disposed on the outer surface (exterior surface) of the vehicle M, for example, on the hood or the front bumper. Good too.

The front camera 11 is communicably connected to the terminal device 20 and can transmit signals of captured images to the terminal device 20.

The GNSS receiver 13 is a device that receives signals from GNSS (Global Navigation Satellite System) satellites (GNSS signals). The GNSS receiver 13 is built into the front camera 11, for example.

Note that the GNSS receiver 13 may be placed anywhere within the vehicle M as long as it can receive GNSS signals. For example, the GNSS receiver 13 may be placed near the front camera 11. For example, the GNSS receiver 13 may be placed on the dashboard DB.

The GNSS receiver 13 is communicably connected to the terminal device 20 and can transmit the received GNSS signal to the terminal device 20. The terminal device 20 acquires current position information of the vehicle M using the GNSS signal.

Note that the GNSS receiver 13 may be a receiver compatible with Real Time Kinematic (RTK) positioning, and may be capable of receiving highly accurate position information obtained by correcting position information by GNSS.

The touch panel display 15 consists of a touch panel and a display. Touch panel display 15 is communicatively connected to terminal device 20 . The touch panel display 15 displays images supplied from the terminal device 20. The touch panel of the touch panel display 15 transmits a signal indicating an input operation by touching the touch panel to the terminal device 20.

For example, the touch panel display 15 transmits a signal indicating a display and an input operation related to route guidance for the vehicle M to the terminal device 20.

For example, the touch panel display 15 transmits to the terminal device 20 a signal indicating an input operation for the user to make settings regarding image processing in the image processing system 100. For example, the settings are settings for image processing, such as blurring processing to protect privacy or processing to emphasize parts that are desired to stand out. Further, the setting is, for example, a setting of a privacy protection mode that differs depending on the object of privacy protection, such as a mode of protecting the privacy of the owner of a feature such as a private residence, or protection of the privacy of the user of the image processing system 100.

Note that, for example, the terminal device 20 may receive an input operation for performing settings related to image processing by voice via a microphone (not shown) provided in the vehicle M.

The configuration and functions of the terminal device 20 installed in the vehicle M will be described with reference to FIGS. 3 and 4.

FIG. 3 is a block diagram showing the configuration of the terminal device 20. As shown in FIG. As shown in FIG. 3, the terminal device 20 is configured such that each part is connected via a system bus 21.

The input unit 23 is an interface that communicably connects the terminal device 20 and equipment provided in the vehicle M.

The input unit 23 is an interface that connects the terminal device 20 and the front camera 11 in a communicable manner.

The input unit 23 is an interface that connects the terminal device 20 to the GNSS receiver 13. The terminal device 20 receives the GNSS signal from the GNSS receiver 13 and acquires information indicating the current position of the vehicle M.

The input unit 23 is an interface that connects the terminal device 20 and the touch panel of the touch panel display 15 in a communicable manner.

The input unit 23 is an interface that connects the terminal device 20 to the gyro sensor GY and the acceleration sensor AC in a communicable manner. The gyro sensor GY is a sensor capable of detecting, for example, the lateral angle (posture), angular velocity, or angular acceleration of the vehicle M. Acceleration sensor AC is a sensor that can detect acceleration in the traveling direction of vehicle M, that is, in the longitudinal direction when viewed from above the vehicle M. Furthermore, the acceleration sensor AC is capable of detecting, for example, acceleration in the lateral direction (width direction) perpendicular to the traveling direction of the vehicle M.

The terminal device 20 determines the traveling direction of the vehicle M based on, for example, the transition of the current position information of the vehicle M acquired by the GNSS receiver 13 and the direction of acceleration indicated by the sensor signals of the gyro sensor GY and the acceleration sensor AC. Get the information shown.

Further, the terminal device 20 uses, for example, information indicating the traveling direction of the vehicle M to acquire information indicating the photographing direction of the image photographed by the front camera 11.

The storage unit 25 is configured with, for example, a hard disk device, an SSD (solid state drive), a flash memory, etc., and stores various programs executed in the terminal device 20. Note that the various programs may be obtained, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. That is, the various programs stored in the storage unit 25 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

A map information database (not shown) is constructed in the storage unit 25. The map information database stores map information used by the terminal device 20 to generate a route for the vehicle M.

The control unit 27 includes a CPU (Central Processing Unit) 27A, a ROM (Read Only Memory) 27B, a RAM (Random Access Memory) 27C, and functions as a computer. The CPU 27A reads and executes various programs stored in the ROM 27B and the storage unit 25, thereby realizing various functions.

The control unit 27 obtains an image, which is a moving image or a still image, of the scenery in front of the vehicle M from the front camera 11 via the input unit 23 .

The control unit 27 acquires current position information of the vehicle M from the GNSS receiver 13 via the input unit 23 . The current position information is, for example, information in which information indicating time and information indicating the position of vehicle M at that time are associated. The information indicating the position of the vehicle M may be information indicating the position on a map.

Further, the control unit 27 generates photographing information by associating the image acquired from the front camera 11 with information indicating the photographing conditions including the photographing position and photographing direction when the image was photographed.

The control unit 27 acquires information indicating the photographing position, which is the position of the camera 11 when the image was photographed. For example, the control unit 27 acquires a GNSS signal from a GNSS receiver, and acquires information indicating the photographing position of the image based on the received GNSS signal.

For example, the control unit 27 acquires the position information of the vehicle M at the time when the image was photographed from the GNSS receiver 13, and sets the position indicated by the position information as the photographing position. For example, when the mounting position of the front camera 11 and the mounting position of the GNSS receiver 13 are far apart, the control unit 27 may calculate the photographing position by correcting the difference in the mounting positions.

Further, the photographing position may include a position in the height direction in addition to a position in the horizontal direction. For example, the position in the height direction is determined by the vehicle height of the vehicle M and the mounting position of the front camera 11, and information indicating the position in the height direction is stored in the storage unit 25 in association with the front camera 11. You can leave it there.

For example, the control unit 27 acquires a QZSS signal (Quasi-Zenith Satellite System) (quasi-zenith satellite system, Michibiki) signal from a GNSS receiver, and based on the received QZSS signal, the control unit 27 acquires a height that indicates the photographing position of the image. Accurate location information may be obtained.

Further, for example, the control unit 27 may obtain a more accurate photographing position by using RTK positioning to obtain information obtained by correcting the position information by GNSS.

The control unit 27 acquires the photographing direction, for example, based on the direction of travel of the vehicle M, which is acquired based on the direction of the front camera 11 and the acceleration of the vehicle M when the image was photographed. For example, when the front camera 11 is fixed so as to face forward of the vehicle M, the control unit 27 may obtain the traveling direction of the vehicle M as the photographing direction.

For example, if the front camera 11 is attached at a variable angle with respect to the traveling direction of the vehicle M, the control unit 27 acquires information indicating the orientation of the front camera 11 at the time of photographing from the front camera 11, and The photographing direction may be acquired based on the traveling direction of M and the orientation of the front camera 11.

Further, the control unit 27 may acquire information indicating the angle of view of the front camera 11 along with the image of the front camera 11 in addition to the photographing position and the photographing direction. That is, the photographing information may include information indicating the angle of view at which the image was photographed. For example, information indicating the angle of view is held in the front camera 11 and transmitted to the terminal device 20 together with the image.

FIG. 4 is a diagram illustrating an example of photographing information generated by the control unit 27 of the terminal device 20. In the example shown in FIG. 4, for one image among the plurality of images acquired by the front camera 11, the frame number of the image includes the photographing position, photographing direction, and angle of view when the image was acquired. are associated.

In the example shown in FIG. 4, the photographing position is expressed as three-dimensional coordinates including the position in the height direction. Note that the photographing position is not limited to three-dimensional coordinates, but may be two-dimensional coordinates.

The photographing direction is expressed, for example, as an azimuth with north set at 0°.

The angle of view is expressed, for example, by angles in the horizontal direction, vertical direction, and diagonal direction. The angle of view included in the photographic information is not limited to this, and may be, for example, only the angle in the horizontal direction.

The control unit 27 transmits the generated photographic information to the image processing device 10. The control unit 27 may sequentially transmit the photographing information for each frame as illustrated in FIG. 4, or may transmit the photographing information every predetermined number of frames or every predetermined time.

Referring again to FIG. 3, the communication unit 29 is a network adapter such as a NIC (Network Interface Card) connected to a wireless device (not shown). The communication unit 29 communicates with the image processing device 10 according to instructions from the control unit 27 . For example, the communication unit 29 performs communication when the terminal device 20 transmits photographing information to the image processing device 10.

The output unit 31 is connected to a display of a touch panel display 15 provided in the vehicle M. The output unit 31 is an interface for supplying various information to the display according to commands from the control unit 27.

For example, the output unit 31 supplies information for displaying a navigation screen to the display. Further, for example, the output unit 31 supplies the display with information for displaying a screen that accepts input of settings related to image processing, such as setting a privacy protection mode by the image processing device 10. For example, the output unit 31 may supply the image captured by the front camera 11 to the display.

Next, the configuration and functions of the image processing device 10 will be explained with reference to FIGS. 5 to 9.

FIG. 5 is a block diagram showing the configuration of the image processing device 10. As shown in FIG. As shown in FIG. 5, the image processing device 10 is a server device in which each part is connected via a system bus 41.

The mass storage device 43 is, for example, a storage device configured with a hard disk device, an SSD (solid state drive), a flash memory, or the like. The mass storage device 43 stores various programs executed by the image processing device 10. Note that various programs may be obtained from, for example, another server device via a network, or may be recorded on a recording medium and read via various drive devices.

For example, the mass storage device 43 stores an image processing program that the image processing device 10 executes when identifying a target area in an image that is to be subjected to image processing.

For example, the mass storage device 43 stores an image processing program that the image processing device 10 executes when performing image processing on a target area in an image.

For example, the large-capacity storage device 43 stores image processing that is performed when the image processing device 10 performs processing to blur an area in an image in which characters that meet predetermined conditions that have a high probability of infringing privacy are shown. Memorize the program. Characters that have a high probability of infringing on privacy are, for example, characters whose height from the ground is less than a predetermined height or whose size is less than a predetermined size, or characters that match words related to people's names, place names, etc. It is.

The mass storage device 43 stores a database in which data necessary for executing various programs is stored.

The mass storage device 43 includes a map information database (hereinafter referred to as map information DB) 43A. The map information DB 43A stores map information including information indicating the position of a feature and the attribute of the feature. In the map information stored in the map information DB 43A, position information of terrestrial objects and information indicating attributes of each terrestrial object are associated with each other.

For example, the map information stored in the map information DB 43A is map information in which position information of a feature is expressed in three dimensions. Note that in the map information, the position information of the terrestrial objects may be represented in two dimensions.

In this embodiment, the map information is used when specifying the target area for image processing based on the attributes of the features in the photographed image.

FIG. 6 is a diagram showing an example of information indicating attributes stored in the map information DB. As shown in FIG. 6, in the map information DB, a feature ID, which is an identifier for specifying each feature, and an attribute of the feature are stored in association with each other.

As shown in FIG. 6, the attributes of the feature include attributes of a private nature such as a private home or a private store, and attributes of a highly public nature such as a commercial facility or a public facility.

In addition to the example shown in FIG. 6, the map information may also store utility poles and manholes as features. For example, there are many cases where utility poles are posted with signs that can identify addresses, such as block display boards, and if a utility pole is included in a photographed image, it is difficult to determine whether or not to include it in the image processing of this embodiment. judgment may be necessary. In addition, manholes often have characters or figures drawn on them that can identify the municipality where the manhole is installed. There may be cases where it is necessary to decide whether or not to include the target.

The mass storage device 43 includes a dictionary database (hereinafter referred to as dictionary DB) 43B. The dictionary DB 43B stores predetermined words that are referred to when the image processing device 10 performs processing regarding characters appearing in an image.

For example, the dictionary DB 43B stores a personal name dictionary containing a large number of personal names. For example, when the image processing device 10 performs a process of blurring the character part of the nameplate of a private residence that is reflected in the image, the person's name dictionary can be used to It is used to determine whether it is included in the

For example, the dictionary DB 43B stores a place name dictionary that includes many place names. For example, when the image processing device 10 performs a process of blurring a character portion indicating a place name such as a block sign board reflected in an image, the place name dictionary may It is used to determine whether it is included in the characters.

The control unit 45 includes a CPU (Central Processing Unit) 45A, a ROM (Read Only Memory) 45B, a RAM (Random Access Memory) 45C, and functions as a computer. The CPU 45A reads and executes various programs stored in the ROM 45B and the mass storage device 43, thereby realizing various functions.

The control unit 45 reads and executes an image processing program stored in the mass storage device 43 to identify a target area in the image to be subjected to image processing.

The control unit 45 reads and executes an image processing program stored in the mass storage device 43 to perform image processing on a target area in the image.

The control unit 45 reads and executes an image processing program stored in the mass storage device 43 to blur an area in the image in which characters that meet predetermined conditions that are highly likely to violate privacy are shown. Perform the processing to do. Characters that are likely to infringe on privacy are, for example, characters whose height from the ground is less than a predetermined height or whose size is less than a predetermined size, or characters that match words related to people's names, place names, etc. be.

The communication unit 47 is a network adapter such as a NIC (Network Interface Card) connected to a wireless device (not shown). The communication unit 47 performs communication between the image processing device 10 and the outside according to instructions from the control unit 45.

For example, the communication unit 47 communicates when the image processing device 10 acquires photographing information including an image photographed by the front camera 11 attached to the vehicle M, and the photographing position and photographing direction of the image from the terminal device 20. I do.

The functions of the control unit 45 of the image processing device 10 will be described in detail below with reference to FIGS. 7 to 11.

The control unit 45 functions as a photography information acquisition unit by acquiring photographic information including a photographed image photographed by the front camera 11 as an imaging device, a photographing position and a photographing direction of the photographed image from the terminal device 20.

[Identification of attributes]
The control unit 45 functions as an attribute specifying unit that specifies the attribute of the feature in the photographed image based on the map information indicating the position of the feature and the attribute of the feature, and the acquired photographic information.

For example, the control unit 45 as an attribute specifying unit generates a reproduced image that reproduces an image taken under conditions including the shooting position and shooting direction indicated by the shooting information, based on the map information stored in the map information DB 43A. By comparing the generated feature that is the object in the photographed image with the feature in the reproduced image, the attribute of the object that is the object in the photographed image is identified.

FIG. 7 is a top view schematically showing an example of the positional relationship between the vehicle M and features existing around the current position of the vehicle M. In FIG. 7, the features are represented by rectangular figures, and each feature is assigned a feature ID. Further, in the example shown in FIG. 7, a utility pole PP exists between the feature F1 and the feature F2.

FIG. 8 is a diagram schematically showing an example of a reproduced image generated when the control unit 45 identifies an attribute. In the example shown in FIG. 8, the image processing device 10 receives photographic information including an image photographed by the front camera 11 when the vehicle M is at the current position shown in FIG. A generated reproduction image IM is shown.

For example, when the control unit 45 receives photographing information from the terminal device 20, it refers to the map information DB 43A of the mass storage device 43 and determines the photographing position, photographing direction, and direction indicated by the photographing information, as shown in FIG. 4, for example. Generates a reproduced image by reproducing the size and angle of the object in the image when photographed under the viewing angle conditions. As described above, each feature is associated with an attribute in the map information DB 43A, so when a reproduced image is generated, the attribute of the feature is associated with the feature in the reproduced image.

For example, the control unit 45 generates a reproduction image using an image generation technique such as 3D rendering based on the three-dimensional map information stored in the map information DB 43A. Note that the control unit 45 may generate the reproduced image based on a two-dimensional map, for example.

In the example shown in Figure 8, the feature whose feature ID is F2 and whose attribute is a private residence and the feature whose feature ID is F5 and whose attribute is a commercial facility appear large in the foreground, and the feature in the background is A reproduced image is generated showing that the features with IDs F1 and F4 and whose attributes are private residences are shown in a small size.

The control unit 45 compares the feature that is the subject in the captured image included in the capturing information with the feature in the reproduced image as shown in FIG. The attributes of features can be identified.

[Identification of target area]
The control unit 45 also functions as a target area specifying unit that specifies, as a target area for image processing, an area in the captured image in which a feature whose attribute is a predetermined attribute specified by the attribute specifying unit is captured. do.

In specifying the target area, for example, if the purpose is to protect privacy, a target area for image processing to be blurred is specified. For example, if the purpose is to highlight characters, buildings, etc. that serve as landmarks when referring to an electronic map with an attached image, a target area for image processing to be emphasized is specified.

Below, we will mainly explain the case where the purpose is to protect privacy.

Examples of privacy protection targets include the privacy of the owner of the feature that is the subject of the photographed image, or the privacy of the user who uses the photographed image.

For example, if the privacy of the owner of a feature is to be protected, images showing personal information such as the exterior of a private residence or a person's name, including the last name or name displayed on the nameplate of a private residence, may be made public. I think it is necessary to prevent this. In this case, a private attribute such as a private residence or a privately run store is set as the attribute setting for specifying the target area based on the attribute of the feature.

In other words, the privacy of the owner of the feature can be protected by performing image processing on the target region using a region in which a feature with a preset private attribute is photographed.

Also, for example, if the privacy of the user of the photographed image is to be protected, it is possible to prevent the location of the photograph from being identified due to the publication of images that include public facilities, street sign boards, etc. I think it is necessary. In this case, highly public attributes such as commercial facilities and public facilities are set as attributes for specifying the target area based on the attributes of the features.

In other words, it is possible to protect the privacy of users who use captured images by applying image processing to an area in which a feature with a preset highly public attribute is captured as a target area. can.

Note that the attributes used to specify the target area may be set in advance in the image processing device 10, or may be set or changeable by the user via the touch panel display 15, for example. For example, the predetermined attribute may be stored in the mass storage device 43 together with information indicating the subject of privacy protection and information indicating the content of image processing.

FIG. 9 is a diagram illustrating an example of setting attributes used to specify a target area. In the example shown in FIG. 9, "privacy protection mode" indicates different attribute setting modes depending on the privacy protection target. In the example shown in FIG. 9, "mode 1" is used to protect the privacy of the owner of the feature, and "mode 2" is used to protect the privacy of the user who uses the photographed image.

Further, the "protection range level" shown in FIG. 9 indicates the breadth of the privacy protection range, in other words, the degree of protection. In the example shown in FIG. 9, the higher the protection range level, the more attributes are set as attributes used to specify the target area.

Note that the more attributes that are set, the stronger the privacy protection, but the area in the image that is identified as the target area is large and sagging, so if blurring is applied, the information in the image may be lost. The scope of what can be done becomes wider.

Note that, for example, in the setting example of FIG. 9, only one of mode 1 or mode 2 may be used, or the settings of mode 1 and the settings of mode 2 may be combined. For example, mode 1 may be set to level 3, and mode 2 may be set to level 1.

Furthermore, in addition to Mode 1 and Mode 2 illustrated in FIG. 9, a mode may be set that aims to make a landmark part in an image stand out. For example, image processing is performed to emphasize a target area specified as a landmark by the mode.

FIG. 10 is a diagram showing how a target area is specified in a photographed image FL1000 as an example of a photographed image. FIG. 10 shows an example in which a target area is specified when the privacy of the owner of a feature is to be protected.

Each feature shown in the photographed image FL1000 is associated with an attribute of the feature. Based on these attributes, a target area is specified in photographed image FL1000.

In FIG. 10, the area surrounded by the broken line indicates the image processing target area AR1. In the example shown in FIG. 10, the area in which the feature having the attribute "private residence" is photographed is specified as the target area AR1.

Furthermore, in FIG. 10, the area surrounded by the dashed line indicates an area AR2 that is not subject to image processing even if it includes a feature. In the example shown in FIG. 10, the area in which the feature having the attribute "commercial facility" is photographed is the non-target area AR2.

Furthermore, when specifying the target area, the control unit 45 excludes from the target area an area in which an object different from the feature with the predetermined attribute is photographed within the region in which the feature with the predetermined attribute is photographed. You may.

Examples of such different objects include telephone poles, which often have a display indicating an address such as a block sign board attached thereto, and are useful as images used for route guidance using, for example, live-action images. In this case, for example, if image processing is performed to make the area where the telephone pole is obscured, useful information will be lost. Therefore, by excluding the area in which the telephone pole is photographed from the target area, it is possible to prevent useful information from being lost from the captured image. In the example shown in FIG. 10, the area in which the telephone pole is photographed in the target area AR1 may be excluded from the target area.

For example, the map information stored in the map information DB 43A may include position information of a utility pole, and the control unit 45 uses the position information of the utility pole to identify features with predetermined attributes in the captured image. The utility pole within the photographed area may be identified, and the area where the utility pole is photographed may be excluded from the target area. For example, if the map information does not include the location information of a utility pole, if a utility pole is detected in an area where a feature with a predetermined attribute is captured through image recognition, the area where the utility pole is captured will be targeted. It may be excluded from the area.

[Aspects of image processing]
Further, the control unit 45 functions as an image processing unit that performs processing to make at least a part of the target area in the photographed image unclear or to emphasize the target area.

Examples of the process for making at least a portion of the target area unclear include image processing such as mosaic processing, blurring processing, and masking processing.

For example, the control unit 45 performs a process of blurring a portion of a nameplate or the like in the target area where characters are shown. For example, when a private residence is included in the photograph, the name, which is personal information of the owner of the private residence, can be prevented from being disclosed on the Internet, and the privacy of the owner can be protected.

Further, the control unit 45 may perform processing to make the entire target area unclear, for example. In this case, for example, the entire building, which is a predetermined feature in the photographed image, can be blurred, and for example, the exterior of a private residence or the like can be prevented from being made public on the Internet.

FIG. 11 is a diagram illustrating an example of a photographed image after image processing that makes the entire target region unclear. In the example shown in FIG. 11, the target area AR1 in which the building whose attribute is a private residence is shown is unclear.

On the other hand, examples of processing for emphasizing the target area include image processing such as increasing the contrast of the target area in the photographed image, increasing brightness or saturation, and the like.

[Character detection]
Further, the control unit 45 functions, for example, as a character detection unit that detects characters in a photographed image. The control unit 45 detects characters in the captured image using, for example, a known character recognition technique such as OCR (Optical Character Recognition) or AI-based character recognition (AI-OCR). For example, the control unit 45 serving as a character detection unit may detect characters within the target area.

[Image processing related to text]
The control unit 45 serving as an image processing unit may, for example, perform a process of blurring an area in the image in which characters that meet a predetermined condition are shown.

For example, if the actual vertical height of the character detected by the character detection unit is lower than a predetermined height, or if the actual size of the character is smaller than the predetermined size, the control unit 45 as an image processing unit In this case, processing is performed to make the character in question unclear.

The control unit 45 calculates the height of the character in the vertical direction based on the distance from the photographing position where the photographed image was taken to the character detected by the character detection unit.

For example, the control unit 45 acquires information indicating the shooting position of the shot image together with the shot image, and calculates the actual distance from the shooting position to the detected character based on the shooting position and map information. . The control unit 45 converts the vertical height of the character in the image from the ground to the actual height of the character based on the calculated actual distance.

In this embodiment, if the actual vertical height of a character detected in a photographed image is higher than a predetermined height, the character is a character on a store signboard or the like that is displayed to an unspecified person. If the actual height is less than or equal to the predetermined height, it is considered that the character is likely to be the character written on the nameplate.

For example, the height of the nameplate from the ground is often less than 2 meters. On the other hand, if the height of the letters above the ground exceeds 2 m, they are often letters on a signboard. Therefore, the control unit 45 performs control such that if the calculated height of the character is 2 m or less, the character is made unclear, and if it exceeds 2 m, the character is not made unclear. It's okay.

This prevents the names of individuals written on nameplates from being published on the Internet, and prevents the writing on signboards from becoming blurred, which prevents excessive loss of information from photographed images. You can prevent this from happening.

Furthermore, the control unit 45 calculates the size of the character based on the distance from the photographing position where the photographed image was taken to the character detected by the character detection unit.

For example, the control unit 45 acquires information indicating the photographing position of the photographed image together with the photographed image, and calculates the actual distance from the photographing position to the detected character based on the photographing position and map information. The control unit 45 converts the size of the character in the image into the actual size of the character based on the calculated actual distance.

In this example, if the actual size of the characters detected in the photographed image is larger than the predetermined size, there is a high possibility that the characters are characters that are intended to be shown to an unspecified person, such as on a store signboard. If the actual size of the character is less than or equal to a predetermined size, it is considered that there is a high possibility that the character is written on a nameplate.

For example, the characters on nameplates are often less than 15 cm wide in the height direction, and the characters on signboards are often larger than this. Therefore, regarding the calculated character size, if the width in the height direction is 15 cm or less, the control unit 45 performs processing to make the character unclear, and if the width in the height direction is larger than 15 cm, Control may be performed to prevent the characters from becoming unclear.

By doing this, for example, an individual's name written on a nameplate is prevented from being published on the Internet, and by not blurring the letters written on a signboard, etc., excessive information is not extracted from the photographed image. You can prevent it from being lost.

For example, either the vertical height of the detected character is lower than a predetermined height (height-based determination) or the actual size of the character is smaller than a predetermined size (size-based determination) If the above applies, the character may be made unclear, or the determination based on height and the determination based on size may be combined, and the text may be made unclear when both apply.

The control unit 45 serving as an image processing unit may perform a process of making the characters blurred when the characters detected in the photographed image match a predetermined word. The predetermined word is, for example, a word related to a person's name or a place name.

For example, when a character string consisting of one or more characters detected in a photographed image matches either a person's name or a place name stored in the dictionary DB 43B of the mass storage device 43, the control unit 45 Processing may be performed to make the characters unclear. With this, for example, characters that are not a person's name or place name, such as "Be careful not to jump out", can be left in the image.

For example, characters that match a given word may be part of a string of characters. For example, only the "Takahashi" part of "Takahashi Clinic" corresponds to a person's name and may be subject to blurring processing.

Further, the determination as to whether or not the word matches a predetermined word may be combined with the determination based on the height of the character or the determination based on the size of the character described above.

For example, if the signboard for "Takahashi Clinic" is written at a higher position than the nameplate or in larger letters, applying the above-mentioned determination based on the height of the characters or determination based on the size of the characters, the "Takahashi Clinic" written on the sign " characters are not subject to image processing and can be left in the photographed image without being blurred.

In this way, by subjecting characters that meet predetermined conditions to image processing to make them unclear, and excluding other characters, it is possible to prevent excessive loss of information in a photographed image.

For example, the control unit 45 may specify a target area for image processing as described above, and when characters in the target area match a predetermined word, perform processing to blur the characters.

For example, if the vertical height of the character detected within the target area is lower than a predetermined height, or if the actual size of the character is smaller than the predetermined size, the control unit 45 controls the character detected in the target area. matches a predetermined word, processing may be performed to blur the character.

A control routine executed by the control unit 45 of the image processing apparatus 10 in this embodiment will be described with reference to FIGS. 12 to 18.

FIG. 12 is a flowchart showing an attribute specifying routine RT1, which is an example of a routine executed by the control unit 45 when specifying the attribute of a feature in a photographed image. For example, when transmission of photographing information from the terminal device 20 is started, the control unit 45 starts the attribute specifying routine RT1.

When the control unit 45 starts the attribute specifying routine RT1, the control unit 45 newly receives and acquires photographing information including a photographed image photographed by the front camera 11, a photographing position and a photographing direction of the photographed image from the terminal device 20 (step S101). In step S101, photographing information including, for example, one frame image, the photographing position at which the frame image was acquired, and the photographing direction is acquired. For example, as shown in FIG. 4, the photographing information may include information indicating the angle of view at which the photographed image was photographed.

After executing step S101, the control unit 45 refers to map information stored in the map information DB 43A (step S102). In step S102, the control unit 45 reads map information of the area including the shooting position indicated by the shooting information acquired in step S101.

After executing step S102, the control unit 45 generates a reproduced image based on the map information read out in step S102, which reproduces the image taken under the conditions including the shooting position and shooting direction indicated by the shooting information acquired in step S101. is generated (step S103). In step S103, for example, a reproduced image as shown in FIG. 8 is generated. Further, in step 103, the attributes of the features in the reproduced image are specified based on the attributes associated with the features included in the map information.

After executing step S103, the control unit 45 compares the photographed image and the reproduced image (step S104).

After executing step S104, the control unit 45 specifies the attribute of the feature in the photographed image (step S105). In step S105, for example, the attribute of the feature in the captured image is identified by comparing the feature that is the subject in the captured image with the feature in the reproduced image.

After executing step S105, the control unit 45 stores the image with information indicating the attribute specified in step S104 (step S106).

After executing step S106, the control unit 45 ends the attribute specifying routine RT1 and starts a new attribute specifying routine RT1.

FIG. 13 is a flowchart showing a target region specifying routine RT2, which is an example of a routine executed by the control unit 45 when specifying a target region for image processing. For example, when the attribute specifying routine RT1 is started, the control unit 45 starts the target area specifying routine RT2.

When the control unit 45 starts the target area identification routine RT2, the control unit 45 newly acquires a photographed image to which the attribute of a terrestrial feature is attached (step S201). In step S201, the control unit 45 reads, for example, an image in which the attribute of the feature is attached to the feature in the photographed image and stored in step S106 of the attribute specifying routine RT1.

After executing step S201, the control unit 45 refers to the setting of attributes for specifying the target area (step S202). In step S202, the control unit 45 refers to attributes that are preset according to the privacy protection target or protection scope level, for example, as illustrated in FIG.

For example, when the object of privacy protection is the privacy of the owner of a feature, a private attribute such as a private residence is set as an attribute for specifying the target area.

For example, if the privacy protection target is the privacy of a user who uses a photographed image, a highly public attribute such as a commercial facility is set as an attribute for specifying the target area.

After executing step S202, the control unit 45 specifies, as a target area, an area in the acquired captured image in which a feature with an attribute corresponding to the settings referenced in step S202 is shown (step S203).

After executing step S203, the control unit 45 stores the image with target area information attached (step S204). In step S204, an image with information indicating the target area, such as that illustrated in FIG. 10, is stored in the mass storage device 43.

After executing step S204, the control unit 45 ends the target area specifying routine RT2 and starts a new target area specifying routine RT2.

FIG. 14 is a flowchart showing an image processing routine RT3, which is an example of a routine executed by the control unit 45 when performing image processing. For example, when the target area identification routine RT2 is started, the control unit 45 starts the image processing routine RT3.

When the control unit 45 starts the image processing routine RT3, the control unit 45 acquires a captured image with information on the target area attached (step S301). In step S301, for example, the control unit 45 reads the image stored in step S204 of the target area identification routine RT2.

After executing step S301, the control unit 45 refers to the image processing settings (step S302). In step S302, for example, with reference to the mass storage device 43, it is indicated whether the content of the image processing to be executed is image processing that blurs at least a part of the target area or processing that emphasizes the target area. Read information.

After executing step S302, the control unit 45 executes image processing on the target area (step S303). In step S303, for example, image processing corresponding to the content of the image processing referred to in step S302 is executed. In step S303, for example, a process of blurring the entire target area or a process of emphasizing the target area is executed.

After executing step S303, the control unit 45 stores the photographed image subjected to image processing (step S304).

After executing step S304, the control unit 45 ends the image processing routine RT3 and starts a new image processing routine RT3.

The image stored in step S304 of image processing routine RT3 is available to the user. For example, the user of the image processing system 100 can publish an image whose target area has been subjected to image processing on the Internet through SNS or an electronic map without infringing on privacy that is to be protected.

FIG. 15 is a flowchart showing an image processing routine RT4, which is an example of a routine executed by the control unit 45 when performing image processing to blur characters. For example, the control unit 45 starts the image processing routine RT4 when transmission of the captured image or captured information from the terminal device 20 is started.

When the control unit 45 starts the image processing routine RT4, it acquires a captured image captured by the front camera 11 (step S401). In step S401, for example, the control unit 45 receives the photographed image and the photographing position of the photographed image from the terminal device 20.

After executing step S401, the control unit 45 uses character recognition technology to detect characters in the photographed image, and determines whether or not characters exist in the photographed image (step S402). In step S402, for example, an area where characters are shown is specified as a character area LT.

FIG. 16 is a diagram illustrating an example in which a region in which characters appear in photographed image FL1000 is specified as character region LT. As shown in Figure 16, the area where the letters written on the nameplate of a private residence are shown, the area where the words ``〇〇 Clinic'' written on the signboard are shown, and the area where the words ``〇〇〇 store'' are written on the signboard are shown. The area in which the written characters are shown is specified as the character area LT.

In step S402, when the control unit 45 determines that there are no characters in the captured image (step S402: NO), it ends the image processing routine RT4 and starts a new image processing routine RT4.

In step S402, if the control unit 45 determines that a character exists in the photographed image (step S402: YES), the control unit 45 determines the vertical height of the detected character from the ground (hereinafter also referred to as the height position) and the height position of the detected character. The size of the characters is calculated (step S403). In step S403, the control unit 45 calculates the height and size of the detected character based on the distance from the photographing position where the image was photographed to the detected character, for example, as described above.

After executing step S403, the control unit 45 determines whether the height of the character calculated in step S403 is lower than a predetermined height (step S404). In step S404, for example, in view of the fact that the height of the nameplate from the ground is often 2 m or less, the predetermined height from the ground serving as a reference is determined to be, for example, 2 m.

In step S404, if it is determined that the calculated character height is lower than the predetermined height (step S404: YES), the control unit 45 determines whether the calculated character size is smaller than the predetermined size. (Step S405). In step S405, for example, in view of the fact that the size of characters written on a nameplate is often less than 15 cm in width in the height direction, the predetermined size serving as a reference is determined to be, for example, 15 cm in width in the height direction. .

In step S405, if it is determined that the calculated character size is smaller than the predetermined size (step S405: YES), the control unit 45 executes processing to blur the detected character in step S402 ( Step S406). In step S406, image processing such as mosaic processing, blurring processing, or masking processing is performed on the detected character portion.

FIG. 17 shows an example in which processing to make the characters unclear is executed on the character area LT in the photographed image FL1000. In the example shown in FIG. 17, the area of the character area LT where the characters written on the nameplate of a private residence are shown is unclear.

In the example shown in FIG. 17, for example, the actual vertical height of the characters written on the nameplate of a private residence is less than or equal to a predetermined height, and the actual size is less than or equal to a predetermined size. , and is subject to image processing to make it blurry. For example, the actual vertical height of the characters written as "〇〇Clinic" is less than the predetermined height, but the size is larger than the predetermined size, so the image processing to make it unclear is necessary. Not applicable.

In the example shown in FIG. 17, the actual vertical height of the characters written as "〇〇〇 store" on the signboard is higher than the predetermined height, and the actual size is larger than the predetermined size. Therefore, it is not subject to image processing to make it unclear.

After executing step S406, the control unit 45 stores the image, which has been subjected to image processing to make the characters unclear, in the mass storage device 43 (step S407). In step S407, for example, the image subjected to image processing may be saved in a dedicated folder that can be uploaded by the user.

If the control unit 45 determines in step S404 that the height position of the character is not lower than the predetermined height (step S404: NO), or determines that the size of the character is not smaller than the predetermined size in step S405. If so (step S405: NO), in step S407, an image that has not been subjected to image processing is stored according to this routine.

In step S407, for example, the image that has not been subjected to the image processing may be saved in a dedicated folder that can be uploaded by the user, similarly to the image that has been subjected to the image processing. An image that has not been subjected to the image processing in step S407 can be said to be an image that has been confirmed as an image that can be uploaded by the user according to the determination criteria of this routine through step S404 or step S405.

After executing step S407, the control unit 45 ends the image processing routine RT4 and starts a new image processing routine RT4.

Note that the image processing routine RT4 may be executed, for example, on a photographed image to which target area information has been attached via the target area specifying routine RT2. For example, in step S401, a photographed image with target area information is acquired, and in step S402, instead of determining whether or not characters exist in the photographed image, whether or not characters exist in the target area is determined. It may be determined whether or not.

By doing so, it is possible to efficiently and accurately specify the character area to be subjected to image processing. For example, in the example shown in FIG. 17, by determining the vertical height and character size of the characters as described above only for the target area AR1, the characters "〇〇Clinic" outside the target area can be excluded from image processing without specifying the character area.

According to this routine, areas in the photographed image that have a high probability of infringing on privacy, such as an area where a nameplate is shown, can be reliably subjected to image processing to blur (blurring processing). Furthermore, areas where there is no high probability of infringing on privacy, such as signs that are located higher than the nameplate or written in larger letters than the nameplate, are excluded from blurring processing. be able to. Therefore, privacy can be protected without unnecessarily damaging the scenery or information in the photographed image.

FIG. 18 is a flowchart showing an image processing routine RT5, which is an example of a routine executed by the control unit 45 when performing image processing to make characters unclear. For example, when the target area identification routine RT2 is started, the control unit 45 starts the image processing routine RT5.

When the control unit 45 starts the image processing routine RT5, the control unit 45 acquires an image with information on the target area attached, as in the case of the image processing routine RT3 (step S501).

After executing step S501, the control unit 45 uses character recognition technology to detect characters within the target area, and determines whether or not there are characters within the target area (step S502).

If it is determined in step S502 that no characters exist within the target area (step S502: NO), the control unit 45 ends the image processing routine RT5 and starts a new image processing routine RT5.

In step S502, if it is determined that a character exists in the target area (step S502: YES), the control unit 45 refers to the dictionary DB 43B of the mass storage device 43 (step S503), and selects the character detected in step S502. It is determined whether or not the word matches a word included in a dictionary in the dictionary DB 43B (step S504).

If it is determined in step S504 that the word matches the word included in the dictionary in the dictionary DB 43B (step S504: YES), the control unit 45 executes processing to blur the characters detected in step S502 (step S505). .

At step S505, for example, as illustrated in FIG. Image processing such as mosaic processing is performed so that only the "〇〇" part where is written becomes unclear.

After executing step S505, the control unit 45 stores the image, which has been subjected to image processing to make the characters unclear, in the mass storage device 43 (step S506). In step S506, for example, the image that has been subjected to image processing may be saved in a dedicated folder that can be uploaded by the user.

When the control unit 45 determines in step S504 that the word does not match the word included in the dictionary in the dictionary DB 43B (step S504: NO), in step S506, the control unit 45 selects an image that has not been subjected to image processing according to this routine. Remember.

In step S506, for example, the image that has not been subjected to the image processing may be saved in a dedicated folder that can be uploaded by the user, similarly to the image that has been subjected to the image processing.

After executing step S506, the control unit 45 ends the image processing routine RT5 and starts a new image processing routine RT5.

Note that the image processing routine RT5 may be executed, for example, on a photographed image to which information about the target area is not attached.

Further, the image processing routine RT5 may be executed in combination with the image processing routine RT4. For example, steps S503 and S504 regarding the determination of whether the word matches a word included in the dictionary in the image processing routine RT5 may be performed before or after the determination regarding the height or size of the character in the image processing routine RT4, that is, step It may be executed before step S403 or after step S405. By doing this, it is possible to more accurately target only the characters that have a high probability of infringing on privacy, and exclude other characters from the blurring process, so that the information in the original image is lost. It is possible to prevent this from happening.

Note that the image processing according to the routine explained using FIGS. 12 to 18 may be executed when the user intends to publish the image via SNS etc., and may be executed sequentially when the captured image is acquired. Alternatively, it may be executed periodically and saved in the upload folder.

Note that in this embodiment, image processing may be performed to emphasize at least a part of the target area or the photographed image in order to make landmarks, characters, etc. stand out.

Further, the image processing for enhancing may be performed in combination with the blurring processing as described above. For example, in order to protect the privacy of the owner of a feature in the photo, image processing is performed to make the area where a private residence or a person's name is written unclear, and the feature that serves as a landmark is Image processing may be performed to emphasize areas where characters are written, such as on block signboards or signboards.

FIG. 19 shows an image in which the photographed image FL1000 has been subjected to image processing to make the area of the nameplate blurred, while emphasizing the building of a commercial facility, which is an effective landmark for route guidance using, for example, a real image. An example of processing is shown.

As described above in detail, the image processing device 10 of the present embodiment includes a photographing information acquisition unit that acquires photographic information including a photographic image photographed by an imaging device, a photographing position and a photographing direction of the photographic image; an attribute identification unit that identifies attributes of features in a captured image based on map information including information indicating the location of the object and attributes of the feature, and imaging information; The image processing apparatus includes a target area specifying unit that specifies an area in which a feature whose specified attribute is a predetermined attribute is photographed as a target area for performing image processing.

With such a configuration, for example, an area that requires image processing to protect privacy can be specified based on the attributes of the feature. For example, for areas other than the target area, the appearance and information of the original image can be maintained even after image processing.

The image processing device 10 of the present embodiment also includes an image acquisition unit that acquires a captured image captured by an imaging device, a character detection unit that detects characters in the captured image, and an actual vertical alignment of the detected characters. and an image processing unit that performs processing to make the character unclear when the height in the direction is lower than a predetermined height or when the actual size of the character is smaller than a predetermined size.

With this configuration, for example, a process is executed to blur the text on a nameplate, etc., where a person's name is written, and a part that does not require blurring processing, such as text written on a signboard, is excluded from the processing. Can be excluded. Therefore, the appearance and information of the original image can be maintained in the excluded portion even after image processing.

Therefore, according to the image processing device 10 of the present embodiment, it is possible to provide an image processing device that can perform appropriate image processing on a subject in a photographed image. For example, according to the image processing device 10 of the present embodiment, information in a captured image is prevented from being excessively lost due to image processing for protecting the privacy related to the subject of the image. It is possible to provide an image processing device that enables this.

The configurations and routines in the embodiments described above are merely examples, and can be selected and changed as appropriate depending on the application and the like.

For example, the terminal device 20 may have the functions possessed by the image processing device 10 in the above embodiment and the information stored in the storage unit, and the terminal device 20 may specify attributes and target areas, and identify character parts. Image processing such as image processing to blur the image may be performed.

For example, the terminal device 20 may have a configuration in which a terminal device having the same configuration as the terminal device 20, the front camera 11, and the touch panel display 15 are integrated. Specifically, for example, the terminal device 20 may be a terminal device such as a smartphone, a tablet, or a PC with a camera equipped with an application that performs the same functions as the terminal device 20 described above. In this case, the terminal device 20 may be mounted on the dashboard DB, for example, with a cradle or the like, so that the built-in camera can photograph the front of the vehicle M through the windshield of the vehicle M.

Further, the terminal device 20 may be configured not to display a screen to be presented to the driver of the vehicle M. For example, the terminal device 20 may have a configuration like a drive recorder, and may be a device integrated with the front camera 11, for example. Specifically, the terminal device 20 is, for example, a device that has built-in hardware within the housing of the front camera 11 that performs the function of transmitting the captured image or captured information of the terminal device 20 to the image processing device 10. There may be. In this case, the terminal device 20 may not display the privacy protection mode selection input screen or the like as described above.

Although the example in which the front camera 11 as an imaging device and the terminal device 20 are mounted on a vehicle has been described, the imaging device and terminal device of the present invention may be moved together with other moving objects such as bicycles, motorcycles, and ships. It's okay. Further, the moving object may be a person. For example, the terminal device 20 may have a built-in camera such as a smartphone, and may be held by a person, and an image processing device may perform image processing on a photographed image taken while walking, for example.

Further, the imaging device of the present invention is not limited to one that moves together with a moving body, and may be a fixed-point camera fixed at a predetermined position along a road, for example.

10 Image processing device 20 Terminal device 23 Input section 25 Storage section 27 Control section 29 Communication section 31 Output section 43 Mass storage device 43A Map information DB
43B Dictionary DB
45 Control unit 47 Communication unit 100 Image processing system M Vehicle

Claims (14)

a photographic information acquisition unit that acquires photographic information including a photographic image photographed by an imaging device, a photographing position and a photographing direction of the photographic image;
an attribute identifying unit that identifies an attribute of a feature in the captured image based on map information including information indicating a location of the feature and an attribute of the feature, and the imaging information;
A target area specifying unit that specifies, in the photographed image, an area in which a feature having a predetermined attribute specified by the attribute specifying unit is shown as a target area for image processing. Image processing device. The attribute identification unit generates, based on the map information, a reproduced image that reproduces the image when the image is captured under conditions including the shooting position and shooting direction indicated by the shooting information, and generates a reproduced image that reproduces the image that is the subject in the shot image. 2. The image processing apparatus according to claim 1, wherein the attribute of the feature that is the subject in the photographed image is identified by comparing the feature with the feature in the reproduced image. 1 . The target area specifying unit is configured to exclude from the target area an area in which an object different from the terrestrial feature is photographed within the region in which the terrestrial feature having the predetermined attribute is photographed. Or the image processing device according to 2. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing is blurring processing that blurs at least a part of the target area. further comprising a character detection unit that detects characters within the target area,
The image according to any one of claims 1 to 4, wherein the image processing is a process of performing image processing on a character area that is an area in which a character detected by the character detection unit is reflected. Processing equipment. 6. The image processing apparatus according to claim 5, wherein the image processing unit executes processing to make the characters unclear when the characters in the character area match a predetermined word. 7. The image processing apparatus according to claim 6, wherein the predetermined word is a word related to a person's name or a place name. The predetermined attribute is an attribute according to the object of privacy protection by the image processing,
The target area specifying unit receives the designation of the protection target and identifies, as the target area, an area in which a feature having an attribute corresponding to the designated protection target is photographed. 8. The image processing device according to any one of 1 to 7. 9. The image processing apparatus according to claim 8, wherein the protection target includes the privacy of an owner of a feature that is a subject in the photographed image or the privacy of a user who uses the photographed image. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing is processing for emphasizing the target area. The image processing apparatus according to any one of claims 1 to 10, further comprising an image processing unit that performs image processing on the target area. An image processing method executed by an image processing device, the method comprising:
a photographing information acquisition step of acquiring photographic information including a photographed image photographed by an imaging device, a photographing position and a photographing direction of the photographed image;
an attribute specifying step of specifying an attribute of the feature in the captured image based on map information including information indicating the location of the feature and the attribute of the feature, and the imaging information;
a target area specifying step of specifying, in the photographed image, an area in which a feature whose attribute specified by the attribute specifying unit is a predetermined attribute is shown as a target area for image processing;
An image processing method characterized by comprising: An image processing program executed by an image processing device including a computer, the computer comprising:
a photographing information acquisition step of acquiring photographic information including a photographed image photographed by an imaging device, a photographing position and a photographing direction of the photographed image;
an attribute specifying step of specifying an attribute of the feature in the captured image based on map information including information indicating the location of the feature and the attribute of the feature, and the imaging information;
a target area specifying step of specifying, in the photographed image, an area in which a feature whose attribute specified by the attribute specifying unit is a predetermined attribute is shown as a target area for image processing;
An image processing program to run. An image processing device equipped with a computer,
a photographing information acquisition step of acquiring photographic information including a photographed image photographed by an imaging device, a photographing position and a photographing direction of the photographed image;
an attribute specifying step of specifying an attribute of the feature in the captured image based on map information including information indicating the location of the feature and the attribute of the feature, and the imaging information;
a target area specifying step of specifying, in the photographed image, an area in which a feature whose attribute specified by the attribute specifying unit is a predetermined attribute is shown as a target area for image processing;
A computer-readable storage medium that stores an image processing program that executes an image processing program.

Images