JP6699518B2 - Image recognition processing method, image recognition processing program, data providing method, data providing system, data providing program, recording medium, processor and electronic device - Google Patents

Description

  The present invention relates to an image recognition processing method, an image recognition processing program, a data providing method, a data providing system, a data providing program, a recording medium, a processor, and an electronic device, and more specifically, an image for performing image recognition processing in consideration of privacy. The present invention relates to a recognition processing method, an image recognition processing program, a data providing method, a data providing program, a recording medium, a processor, and an electronic device.

  In recent years, development of a sensor data distribution market is being considered in which data detected by various sensors is collected, new information is extracted by utilizing the collected data, and new value-added services are newly provided.

For example, in a camera system using a surveillance camera, a third party (data use destination) who receives image data from a system operating source (data providing source) provides a service by a new application using the provided image data. Consideration is also being made.
Images captured by surveillance cameras often include people, vehicles, etc., and include privacy information that can identify an individual. Therefore, when promoting the distribution of such image data, the privacy information It is necessary to build a data distribution system that considers protection.

  Conventionally, various techniques have been proposed for protecting the privacy of a specific object in an image captured by a camera. For example, in Patent Document 1 below, a protection area (face area) requiring privacy protection in a captured image is detected, and blurring processing and mosaic processing are performed on the protection area on the captured image (actual image). Alternatively, a system is disclosed that generates and outputs a protected image that has been subjected to image processing such as painting processing.

[Problems to be Solved by the Invention]
However, in the technique described in Patent Document 1, when the detection of the protection area fails, for example, when the protection target imaged on the mirror or the window glass cannot be detected, the protection areas for which the detection fails are detected. The captured image is output as it is without performing image processing such as blurring processing. Therefore, the output image is in a state where the area to be originally protected (the protection area that has failed to be detected) is not image-processed, and the image data in which the protection area is not image-processed is processed by a third party. However, there is a problem in that the privacy of the protection target cannot be properly protected.
In order to eliminate such a problem, a method of performing blurring processing on the entire captured image may be considered. However, when blurring processing is performed on the entire image, objects existing in the background (structures or exhibitions -Display information) is also removed, which makes it difficult to understand the relationship between the object to be protected and the object existing in the background, and there is a problem that the advantage of utilizing image data by a third party decreases.

JP, 2006-217161, A

Means for Solving the Problems and Their Effects

  The present invention has been made in view of the above problems, and even if the detection of an object whose privacy should be protected from the acquired image fails, the object that fails in the detection is not disclosed. An object of the present invention is to provide an image recognition processing method, an image recognition processing program, a data providing method, a data providing system, a data providing program, a recording medium, a processor, and an electronic device that can generate an image.

  In order to achieve the above object, an image recognition processing method (1) according to the present invention is an image recognition processing method in which an image captured by at least one imaging unit is processed by at least one computer. Using the position data of the target object including privacy information detected from the image captured by the image capturing unit and the background image read from the background image storage unit in which the background image not including the target object is stored, The method is characterized by including a step of generating a composite image in which the model of the target object in which the privacy information of the target object is removed is drawn on the background image.

According to the image recognition processing method (1), the position data of the target object detected from the captured image and the background image are used to synthesize a model of the target object on the background image. Generate an image. Therefore, since the target object whose position is detected from the captured image is drawn as the model on the background image, it is possible to generate a composite image that does not include privacy information of the detected target object. Further, since the model of the target object is drawn on the background image, the processing load on the computer when continuously generating the composite image is greater than the method of performing mosaic processing or the like for each captured image. Can be reduced. Further, even if the detection of the target object from the captured image fails, the background image is used, so the composite image does not include the target object that failed in the detection, and the detection fails. The privacy of the target object can also be appropriately protected, a complicated process for checking the privacy information of the target object is not required, and a simplified image recognition process can be realized.
Note that the model of the object is a display object imitating the object existing in the captured image in a form that does not include the privacy information, and the computer substitutes the object on the background image. It is a display object drawn as an object.

  An image recognition processing method (2) according to the present invention is an image recognition processing method in which an image captured by at least one image capturing unit is processed by at least one computer, and the image is captured by the at least one image capturing unit. From the background image storage unit that stores the position data of the detected target object and the background image that does not include the target object. Using the read background image, and generating a composite image in which the model of the target object in which the privacy information of the target object is removed is drawn in the background image.

  According to the image recognition processing method (2), the same effect as that of the image recognition processing method (1) can be obtained, and the step of detecting the position of the object, and the composite image generation. And the step of performing can be continuously performed, and the processing efficiency of the computer can be improved.

  An image recognition processing method (3) according to the present invention is characterized in that, in the image recognition processing method (1) or (2), the background image corresponds to an imaging range of the imaged image. I am trying.

  According to the image recognition processing method (3), an image corresponding to the image capturing range of the captured image can be generated as the composite image, and the reproducibility of the background can be improved.

  Further, an image recognition processing method (4) according to the present invention is the image recognition processing method (1) to (3), wherein the synthetic image is the background image and the model of the object is imaged. It is characterized in that it is drawn so that the position state of the object in the image is reproduced.

  According to the image recognition processing method (4), the composite image in which the model is drawn so as to reproduce the position state of the target object in the image can be generated, and the position and orientation of the target object, etc. The reproducibility of the state of can be improved.

  An image recognition processing method (5) according to the present invention is characterized in that, in any of the image recognition processing methods (1) to (4), the background image is generated from three-dimensional background data. I am trying.

  According to the image recognition processing method (5), since the background image is generated from the three-dimensional background data, the reproducibility of the background portion of the captured image can be improved and the background image can be captured. It is possible to generate the composite image with less discomfort even when compared with other images.

  An image recognition processing method (6) according to the present invention is the image recognition processing method according to any one of the image recognition processing methods (1) to (4), wherein the background image is obtained by averaging the plurality of captured images. It is characterized by being generated from the average image.

  According to the image recognition processing method (6), since the background image is generated from the average image, the background image of the captured image is faithfully reproduced, and the background image is faithfully reproduced. Can be generated.

  The image recognition processing method (7) according to the present invention is the image recognition processing method according to any one of (1) to (6), wherein the position data of the object is the object detected from the captured image. It is characterized in that it is two-dimensional position data on the imaged image restored from the three-dimensional position data of the object.

  According to the image recognition processing method (7), even when the position detection of one target object is performed three-dimensionally using the images captured by the plurality of imaging units, the three-dimensional position data is obtained. By using the two-dimensional position data restored from the above, it is possible to generate a composite image with high positional accuracy in which the model of the object is drawn on the background image, that is, the two-dimensional image.

  The image recognition processing method (8) according to the present invention is the image recognition processing method (1) or (2), wherein the position data of the object is 3 of the object detected from the captured image. Dimensional position data, the background image is three-dimensional background data, and the step of generating the composite image uses the three-dimensional position data of the target object, and the three-dimensional position data of the target object on the three-dimensional background data. It is characterized in that the model is drawn at a three-dimensional position, and an image obtained by cutting out the imaged image region including the model from the three-dimensional background data on which the model is drawn is generated as the composite image. ..

  According to the image recognition processing method (8), using the three-dimensional position data of the object, the model is drawn at the three-dimensional position of the object on the three-dimensional background data, and the model is drawn. Since the image obtained by cutting out the captured image region including the model from the three-dimensional background data is generated as the composite image, it is possible to generate, as the composite image, a bird's-eye view image with less discomfort in the drawing state of the model. ..

  An image recognition processing program (1) according to the present invention is an image recognition processing program for causing at least one computer to execute processing of an image captured by at least one imaging unit, and the at least one computer Reading the position data of the target object from a position data storage unit that stores position data of the target object including privacy information detected from an image captured by the at least one image capturing unit; The background image is read from a background image storage unit that stores a background image that is not stored, position data of the read target object, and the read background image are used, and the privacy of the target object is added to the background image. Generating a synthetic image in which the model of the object from which information has been drawn is drawn.

  According to the image recognition processing program (1), the same effect as that of the image recognition processing method (1) can be obtained by causing the at least one computer to execute the program.

  An image recognition processing program (2) according to the present invention is an image recognition processing program for causing at least one computer to execute processing of an image captured by at least one image capturing unit. A step of detecting a position of an object including privacy information from an image captured by the at least one image capturing section, a step of storing position data of the detected object in a position data storage section, A step of reading the position data of the target object from a position data storage part; a step of reading the background image from a background image storage part storing a background image that does not include the target object; and a position of the read target object Using the data and the read background image, a step of generating a composite image in which a model of the object without the privacy information of the object is drawn on the background image is executed. ..

  According to the image recognition processing program (2), the same effect as that of the image recognition processing method (2) can be obtained by causing the at least one computer to execute the program.

  An image recognition processing program (3) according to the present invention is an image recognition processing program for causing at least one computer to execute processing of an image captured by at least one imaging unit, and the image recognition processing method ( The method according to any one of 3) to (8) is executed by the at least one computer.

  According to the image recognition processing program (3), the same effect as any one of the image recognition processing methods (3) to (8) can be obtained by causing the at least one computer to execute the program.

  A computer-readable recording medium (1) according to the present invention is characterized in that it is a computer-readable medium recording any one of the image recognition processing programs (1) to (3).

  According to the computer-readable recording medium (1), by causing the computer to read and execute the image recognition processing program recorded on the recording medium, the image recognition processing programs (1) to (3) The same effect as any of the above can be realized. The recording medium may be a tangible recording medium, and includes various built-in or externally attached IC memories, hard disks, optical disks, and the like.

  A processor (1) according to the present invention is characterized by reading and executing any of the image recognition processing programs (1) to (3).

  According to the processor (1), by reading and executing any one of the image recognition processing programs (1) to (3), the processor recognizes any one of the image recognition processing programs (1) to (3). The same effect can be realized.

  An electronic device (1) according to the present invention is characterized in that the processor (1) is mounted.

  According to the electronic device (1), since the processor (1) is mounted, it is possible to achieve the same effect as that of any one of the image recognition processing programs (1) to (3).

  A data providing method (1) according to the present invention is a data providing method executed in a system including at least one computer capable of communicating via a network, wherein the at least one computer is the image recognition processing method. The method is characterized by including the step of transmitting the composite image generated in the step of generating the composite image in any one of (1) to (8) to another computer.

  According to the data providing method (1), the at least one computer separates the composite image generated in the step of generating the composite image in any of the image recognition processing methods (1) to (8). To the third computer so that the at least one computer can provide the composite image to the other computer, and by providing the third party with the privacy-protected composite image, It is possible to expand the possibility of providing new services using composite images.

  In the data providing method (2) according to the present invention, in the data providing method (1), the captured image is stored in the another computer when a predetermined condition that requires the captured image is satisfied. It is characterized by further including a step of transmitting.

  According to the data providing method (2), the captured image can be transmitted to the another computer when the predetermined condition is satisfied, and the image captured under the predetermined condition is also utilized. can do.

  A data providing method (3) according to the present invention is a data providing method executed in a system including at least one computer communicable via a network, wherein the at least one computer is the image recognition processing method. The method is characterized by including a step of displaying the composite image generated in the step of generating the composite image in any one of (1) to (8) on a display unit.

  According to the data providing method (3), the at least one computer displays the composite image generated in the step of generating the composite image in any of the image recognition processing methods (1) to (8). Since it is displayed on the display unit, it is possible to display the composite image in which the privacy is appropriately protected on the display unit and present it to a third party, thereby expanding the possibility of providing a new service using the composite image. You can

  Further, the data providing method (4) according to the present invention is characterized in that, in the data providing method (3), the method further includes a step of switching the composite image and the captured image and displaying the same on the display unit. I am trying.

  According to the data providing method (4), the composite image and the captured image can be switched and displayed on the display unit, and the composite image and the captured image can be switched depending on the situation. It is possible to provide a service that can provide the service, and increase the added value of the service.

  A data providing system (1) according to the present invention is a data providing system in which at least one computer provides data to another computer via a network, and the at least one computer is at least one imaging unit. A position data transmission unit that transmits position data of an object including privacy information detected from a captured image to the another computer, and a background that transmits a background image that does not include the object to the other computer. An image transmission unit is provided, and the model of the target object is drawn on the other computer by using the position data of the target object and the background image, and the privacy information of the target object is removed from the background image. The feature is that a synthetic image is generated.

  According to the data providing system (1), the background image and the position data of the object are transmitted from the at least one computer to the other computer, and the another computer generates the composite image. It is possible to provide the third party who uses the other computer with the synthesized image whose privacy is appropriately protected, and expand the possibility of providing a new service using the synthesized image. In addition, since it is possible to appropriately transmit only the position data of the object after transmitting the background image to the other computer, it is possible to transmit the captured image to the other computer in real time. Compared with the configuration, the data capacity (data transfer amount) transmitted to the other computer can be significantly reduced.

  A data providing program (1) according to the present invention is a data providing program to be executed by a system including at least one computer capable of communicating via a network, the at least one computer including the image recognition process. It is characterized in that the step of transmitting the composite image generated in the step of generating the composite image in any of programs (1) to (3) to another computer is executed.

  According to the data providing program (1), it is possible to cause the at least one computer to provide the composite image to the other computer, and provide the third party with the composite image in which privacy is appropriately protected. As a result, the possibility of providing a new service using the composite image can be expanded.

  A data providing program (2) according to the present invention is imaged in the data providing program (1), when a predetermined condition that requires the imaged image is satisfied in the at least one computer. It is characterized in that the step of transmitting the image to the another computer is further executed.

  According to the data providing program (2), the captured image can be transmitted to the another computer when the predetermined condition is satisfied, and the image is captured by the other computer under the predetermined condition. You can also make use of the captured images.

  A data providing program (3) according to the present invention is a data providing program to be executed by a system including at least one computer capable of communicating via a network, wherein the at least one computer is provided with the image recognition processing. It is characterized in that the step of displaying the composite image generated in the step of generating the composite image in any of the programs (1) to (3) on the display unit is executed.

  According to the data providing program (3), by causing the at least one computer to display the composite image generated in the step of generating the composite image on a display unit, the combination in which privacy is appropriately protected An image can be displayed on the display unit and presented to a third party, and the possibility of new service provision can be expanded.

  The data providing program (4) according to the present invention, in the data providing program (3), a step of switching the composite image and the captured image to the at least one computer and displaying the combined image on the display unit. It is characterized by further executing.

  According to the data providing program (4), the composite image and the captured image can be switched and displayed on the display unit, and the composite image and the captured image can be switched depending on the situation. It is possible to provide a service that can provide the service, and increase the added value of the service.

  A computer-readable recording medium (2) according to the present invention is characterized in that it is a computer-readable medium recording any of the data providing programs (1) to (4).

  According to the computer-readable recording medium (2), by causing the computer to read and execute the data providing program recorded on the recording medium, any one of the data providing programs (1) to (4) is obtained. The same effect can be realized. The recording medium may be a tangible recording medium, and includes various built-in type or external type IC memories, hard disks, optical disks, and the like.

  A processor (2) according to the present invention is characterized by reading and executing any of the data providing programs (1) to (4).

  According to the processor (2), by reading and executing any one of the image data providing programs (1) to (4), the same processor as any one of the data providing programs (1) to (4) is executed by the processor. The effect can be realized.

  An electronic device (2) according to the present invention is characterized in that the processor (2) is mounted.

  According to the electronic device (2), since the processor (2) is mounted, it is possible to achieve the same effect as that of any one of the image data providing programs (1) to (4).

It is a conceptual diagram for demonstrating the structure of the data provision system which concerns on Embodiment (1) of this invention. It is a block diagram which shows schematically the hardware constitutions of an image sensor apparatus. It is a block diagram which shows schematically the hardware constitutions of an image processing server and an image provision server. It is a functional block diagram for demonstrating the process which an image sensor apparatus, an image processing server, and an image provision server perform. 6 is a flowchart illustrating a registration process of various data performed by the image processing server. 7 is a flowchart illustrating a composite image generation process performed by the image processing server. 6 is a flowchart for explaining a processing operation executed in the data providing system according to the embodiment (1). It is a display example for explaining a composite image, (a) is a captured image, (b) is a background image generated from 3D map data, (c) is a composite image when the detection of the target object is successful, (D) is an image when the detection of the target object fails. It is a conceptual diagram for demonstrating the structure of the data provision system which concerns on Embodiment (2). It is a functional block diagram for explaining processing performed by an image sensor device and an image providing server. 9 is a flowchart for explaining a processing operation executed in the data providing system according to the embodiment (2). It is a conceptual diagram for demonstrating the structure of the data provision system which concerns on Embodiment (3). It is a functional block diagram for explaining the processing which an image sensor device, an image processing server, and a user terminal perform. 9 is a flowchart for explaining a processing operation executed in the data providing system according to the embodiment (3). It is a figure showing the example of a display of the synthetic picture displayed on the indicator of a user terminal.

  Embodiments of an image recognition processing method, an image recognition processing program, a data providing method, a data providing system, a data providing program, a recording medium, a processor, and an electronic device according to the present invention will be described below with reference to the drawings. FIG. 1 is a conceptual diagram for explaining the configuration of the data providing system according to the embodiment (1).

  The data providing system 1 is configured to include an image sensor device 10, a router 20, an image processing server 30, and an image providing server 40, and provides the user terminal 50 with an image in which the privacy of the object 2 including privacy information is protected. It is a possible system. In the embodiment (1), the image processing server 30 functions as an image processing device 300 (FIG. 4) including a computer that executes the image recognition processing method according to the present embodiment.

  In the present embodiment, the case where the object 2 for protecting privacy is a human will be described, but the object 2 is not limited to a human. For example, information indicating personally identifiable information and information such as cars and bicycles that are operated or possessed by humans can be set as the target. In the present specification, the privacy information is information that can identify the person when the object 2 is a human, and can identify the owner when the object 2 is a car or a bicycle.

  The image sensor device 10 functions as a camera device that captures an image. The image (actual image) captured by the image sensor device 10 may be a moving image, a still image, a color image, a gray image, or the like.

  The router 20 is a communication connection device provided between the image sensor device 10 and the image processing server 30. The router 20 includes a control unit, a storage unit, a network interface, a device connection interface, a power supply unit (not shown), and has a function of selecting a network path (destination) for transferring data according to an IP address. I have it. The connection between the image sensor device 10 and the router 20 may be a wired connection or a wireless connection. Note that the router 20 is not an indispensable configuration, and the image sensor device 10 may be directly connected to the network 60.

  The image processing server 30 is an electronic device equipped with a computer, and generates a composite image in which the privacy of the target object 2 is protected at a predetermined timing such as when there is a request from the image providing server 40 side or at fixed time intervals. However, it has a function of uploading (transferring) to the image providing server 40 via the network (communication network) 60. The network 60 may be the Internet, a VPN network, a dedicated line, a communication line combining these, or the like, and includes wired and wireless networks. The number of installed image processing servers 30 and image providing servers 40 is not particularly limited. The data providing system 1 may be equipped with at least one or more.

  The installation place and the number of the image sensor devices 10 are not particularly limited. The image sensor device 10 is installed in, for example, a store, a commercial facility, a shopping district, an underground mall, a business facility, a moving body such as an outdoor or a vehicle (hereinafter referred to as a facility), and functions as a surveillance camera of the facility, for example. To do. Further, the image processing server 30 can be provided as an internal (in-facility network) server for each facility or the like in which the image sensor device 10 is installed (which manages the image sensor device 10). It should be noted that in another embodiment, a system configuration in which the router 20 is equipped with the function of the image processing server 30 may be employed.

  The image providing server 40 is an electronic device including a computer, and, for example, provides image data uploaded from the image processing server 30 to the user terminal 50 via the network 60 based on a request from the user terminal 50. It has a function to do. The image sensor device 10, the image processing server 30, the image providing server 40, and the user terminal 50 are used for image recognition processing and data providing for processing and providing image data in cooperation with each other via the network 60. The program is built in. By constructing a service platform that can cooperate with each other, the system can be operated efficiently.

  The user terminal 50 is an electronic device including a computer, and is configured by an information terminal device such as a general-purpose personal computer (PC), smart phone, or tablet terminal. The user terminal 50 is used by a third party who is different from the operator such as a facility that manages the image sensor device 10. In the image providing server 40 and the user terminal 50, application software (external application) 51 (see FIG. 4) that cooperates with each other is incorporated by downloading or the like. The external application 51 is software for providing a service in which the third party can use the image data provided from the image providing server 40.

  Services that can be provided to a third party through the external application 51 include information obtained from data such as a composite image provided from the image providing server 40, for example, a position state such as the position and orientation of the target object 2 and a background. There is a service that uses information such as a positional relationship with an object (structure or arrangement). More specifically, it includes various security services that detect abnormalities and abnormalities, various forecasting services such as power forecasting and traffic forecasting, and various marketing services such as sales promotion, customer acquisition, product placement, and personnel placement. Further, a plurality of image providing servers 40 can be provided according to the type of external application 51 and the like.

  FIG. 2 is a block diagram schematically showing the hardware configuration of the image sensor device 10. The image sensor device 10 includes an imaging unit 11, an image processing unit 12, a storage unit 13, a control unit 14, an input/output interface (I/F) 15, an operation unit 16, a clock unit 17, and a power supply unit 18. ing. The image sensor device 10 is not limited to an integrated type in which each of these parts is provided in one housing, but a camera part including the image pickup part 11 and a controller part including the control part 14 and the like are separately configured. Also includes things.

  The image pickup unit 11 is configured to include a lens system, an image pickup device such as CCD or CMOS, and a drive circuit. The image processing unit 12 is configured to include various image processing circuits and image processing processors, and performs various image correction processes and image recognition processes. The image recognition process includes a feature extraction process of the target object 2, an identification process, a statistical calculation process, an estimation/judgment process of position/posture/attribute, and the like. When the target object 2 is a human, it includes a processing function of identifying a human, extracting a physical feature point such as a head, estimating a posture, an attribute (male or female, adult or child, etc.), and the like. Preferably.

  The control unit 14 includes a microcomputer including a CPU (processor), a RAM, a ROM, and the like, and stores a program for controlling the operation of each unit of the image sensor device 10. These programs may be stored in a computer-readable built-in recording medium or an external recording medium. The storage unit 13 is configured by a semiconductor memory or the like that can temporarily store captured image data and the like.

  The input/output I/F 15 is composed of various interfaces for connecting to an external device such as the router 20. The operation unit 16 is configured to include operation buttons and an operation panel for operating various functions of the image sensor device 10. The clock unit 17 is configured to include a clock circuit that records the time when the image is captured. The power supply unit 18 is configured to include a power supply circuit that supplies power to each unit of the image sensor device 10 and a battery.

  FIG. 3 is a block diagram schematically showing the hardware configurations of the image processing server 30 and the image providing server 40. The image processing server 30 includes an image processing unit 31, a storage unit 32, a communication processing unit 33, a control unit 34, an operation unit 35, a display unit 36, and a power supply unit 37, but is not limited to this configuration. Not something. The image processing server 30 may not include the display unit 36.

  The image processing unit 31 is configured to include an image processing circuit and an image processing processor that perform processing for combining images in which the privacy of the object 2 is protected. The storage unit 32 is configured to include a large-capacity storage device such as a hard disk drive. The storage unit 32 is made to correspond to 3D map data (background data) used to generate a background image not including the object 2, parameter information of the image sensor device 10, and an image region imaged by the image sensor device 10. Various data such as the background image generated by the above are stored in a database.

  The communication processing unit 33 is configured to include a communication interface and a communication control unit for making a communication connection with the image sensor device 10 and an external image providing server 40 side via the network 60. The control unit 34 includes a microcomputer including a CPU (processor), a RAM, a ROM, and the like, and stores a program for controlling the operation of each unit of the image processing server 30. These programs may be stored in a computer-readable built-in recording medium or an external recording medium.

  The operation unit 35 is configured to include an input device such as a keyboard for operating various functions of the image processing server 30 and performing data input/output/update instructions. The display unit 36 includes a display device capable of displaying data processed by the image processing server 30. The power supply unit 37 includes a power supply circuit that supplies power to each unit of the image processing server 30, a backup power supply, and the like.

The image providing server 40 includes a storage unit 41, a communication processing unit 42, a control unit 43, an operation unit 44, a display unit 45, and a power supply unit 46, but is not limited to this configuration.
The storage unit 41 is configured to include a large-capacity storage device such as a hard disk drive, and is configured to store the data of the composite image and the actual image transmitted from the image processing server 30 as a database. ..

  The communication processing unit 42 includes a communication interface and a communication control unit for communicating with the user terminal 50 side and the image processing server 30 side via the network 60. The control unit 43 is configured to include a microcomputer including a CPU (processor), RAM, ROM, and the like, and stores a program for controlling the operation of each unit of the image providing server 40. These programs may be stored in a computer-readable built-in recording medium or an external recording medium.

  The operation unit 44 is configured to include an input device such as a keyboard for operating various functions of the image providing server 40 and performing data input/output/update instructions. The display unit 45 includes a display device capable of displaying data processed by the image providing server 40. The power supply unit 46 includes a power supply circuit that supplies power to each unit of the image providing server 40, a backup power supply, and the like.

FIG. 4 is a functional block diagram for explaining processing executed by the image sensor device 10, the image processing server 30, and the image providing server 40.
[Image sensor device 10]
The image acquisition unit 12a performs a process of acquiring the image captured by the imaging unit 11, and the target detection unit 12b detects (recognizes) the target object (human) 2 including the privacy information from the acquired image (actual image). Perform processing to The acquired image and the data of the object detection (image recognition) result are transmitted to the image processing server 30 in a state of being associated with the data such as the ID (identifier) of the image sensor device 10 and the image acquisition date and time.

[Image processing server 30]
The image processing server 30 is equipped with the image processing device 300. The image processing device 300 is configured to include an image receiving unit 33a, an image transmitting unit 33b, a position detecting unit 31a, a background image generating unit 31b, a composite image generating unit 31c, and a 3D map registration unit 32b, and each of these units is an image. It is realized by the functions of the image processing unit 31, the storage unit 32, the communication processing unit 33, and the control unit 34 of the processing server 30.

  The image receiving unit 33a receives and acquires the image data transmitted from the image sensor device 10 (functions as an image acquiring unit), and stores the image data in the actual image database 32a, while the position detecting unit 31a A process of calculating the position (coordinates or area) of the target object (human) 2 on the captured image is performed based on the object detection data. The position data of the object 2 detected by the position detection unit 31a is stored in the RAM or the storage unit 32 in the control unit 34 in a state of being associated with data such as the ID (identifier) of the image sensor device 10 and the image acquisition date and time. Remembered. The RAM in the control unit 34 or the storage unit 32 functions as a position data storage unit that stores the position data of the object 2. The function of the target detection unit 12b may be provided in the image processing server 30.

A two-dimensional position calculation method, a three-dimensional position calculation method, or the like is applied to the position calculation method of the object 2 executed by the position detection unit 31a.
As a two-dimensional position calculation method,
(1) A method of extracting feature points (x, y coordinates) of the human head, which is the object 2, from the captured image, and calculating the head position (coordinates or area),
(2) The physical position (x, y coordinates) of the human head, torso, arms, legs, etc., which is the object 2, is extracted from the captured image, and the human position (coordinates or area) is extracted. How to calculate,
(3) Background difference processing for comparing a pre-acquired image (comparative background image) with the captured image to extract a foreground region (x, y coordinates or region) in which the human being the target 2 exists. A method of performing and calculating a person position can be applied.

As a three-dimensional position calculation method,
(1) Position (coordinates or area) data of the object 2 detected on the image acquired by the monocular image sensor device 10, position/orientation (internal/external parameter) information of the image sensor device 10, three-dimensional map data , And the image recognition data such as the attributes (male or female, adult or child, etc.) of the detection target (human) and posture (standing, sitting, crouching, or falling) Method to calculate three-dimensional position coordinates by
(2) Position (coordinates or area) data of the object 2 detected on the image acquired by the monocular image sensor device 10 and sensor information other than the image, for example, the object (human) 2 possesses A method of calculating three-dimensional position coordinates using radio field intensity (Bluetooth Low Energy (BLE), Wi-Fi) data (positioning data indoors and outdoors) emitted from a mobile terminal such as a smartphone (in this case, an image sensor) A BLE receiver (beacon) or a Wi-Fi radio wave receiver is connected to the device 10.),
(3) When the same region is imaged by a plurality of image sensor devices 10, an epipolar condition (a basic matrix based on the camera posture based on camera postures) at which a line of sight (epipolar line) from the plurality of image sensor devices 10 intersects one point in space Using the constraint line of the corresponding points to determine the geometrical relationship existing between the two images) and the three-dimensional map data, the intersection point of the epipolar lines is used as the three-dimensional position of the object 2. Method of calculation (especially when there are multiple objects, it is possible to perform more accurate position estimation by combining three-dimensional geometric constraints such as epipolar conditions and time-series constraints. ,),
(4) The image sensor device 10, three-dimensional map data, and various sensors (magnetic sensor, electronic compass, acceleration sensor, gyro sensor, position-positioning sensor, etc.) are combined, and the three-dimensional image of the object 2 is obtained based on these sensor information. A method of calculating position coordinates or the like is applied.

  Further, the 3D map registration unit 32b performs a process of registering 3D (3D) map data of a facility or the like where the image sensor device 10 is installed in the background database (DB) 32c. The 3D map data is used to generate a background image of an area corresponding to the imaging range of the image sensor device 10.

  The external parameter registration unit 32d registers the external parameter of the image sensor device 10 in the parameter database (DB) 32f, and the internal parameter registration unit 32e performs the process of registering the internal parameter of the image sensor device 10 in the parameter DB 32f. The internal parameters include the focal length of the image sensor device 10, the image center, the image (pixel) size, and the distortion aberration coefficient. The external parameters include rotation and parallel movement amounts indicating the position and orientation of the image sensor device 10, and are parameters necessary for converting three-dimensional coordinates in one coordinate system into coordinates in another coordinate system.

  The background image generation unit 31b uses the 3D map data registered in the background DB 32c, the parameters of the image sensor device 10 registered in the parameter DB 32f, and the like to correspond to the imaging range of the image sensor device 10 from the 3D map data. Performs processing to generate a background image. The generated background image is registered in the background image database (DB) 32g in association with the ID of the image sensor device 10, for example.

The composite image generation unit 31c uses the position (coordinates) data of the target object read from the RAM or the storage unit 32 in the control unit 34 and the corresponding background image read from the background image DB 32g, and uses the background image on the background image. A process of drawing a model in which the privacy information of the object 2 is removed (not included) at the detection position of the object 2, for example, a CG image (doll) representing the silhouette of the object 2 to generate a composite image. I do. Note that the model of the object 2 is a display object that imitates the object 2 included in the captured image in a form that does not include privacy information, and is drawn by a computer as a substitute for the object 2 on the background image. It is a displayed object.
The image transmission unit 33b performs a process of transmitting the composite image data associated with the ID of the image sensor device 10 and the data such as the image acquisition date and time to the image providing server 40.

When the position detection unit 31a calculates the two-dimensional coordinates on the image of the object 2, the synthetic image generation unit 31c performs a process of generating a synthetic image in which a model is two-dimensionally arranged on the background image.
Further, when the position detection unit 31a calculates the three-dimensional coordinates of the object 2 on the 3D map, the composite image generation unit 31c restores the two-dimensional coordinates on the image from the three-dimensional coordinates, and restores the restored two-dimensional coordinates. Based on this, processing for generating a composite image in which a model is two-dimensionally arranged on the background image is performed.

  The image transmitting unit 33b also performs a process of transmitting the actual image stored in the actual image DB 32a to the image providing server 40 when a predetermined condition is satisfied. Further, the image transmitting unit 33b causes the management terminal device (not shown) in which the application software (internal application) 38 for managing the image providing server 40 is incorporated into the real image stored in the real image DB 32a or the composite image. A process of transmitting the composite image generated by the image generation unit 31c is also performed.

[Image providing server 40]
The image providing server 40 is an electronic device including a computer, and executes a process of transmitting a composite image generated by the image processing server 30 or a captured real image to the user terminal 50.
The image receiving unit 42a receives the data of the composite image and the real image transmitted from the image processing server 30, and associates the data of the ID of the image sensor device 10 and the date and time of the image pickup with the composite image database 41a and the real image database 41b. The process of storing in each is performed.

  The image transmitting unit 42b performs a process of transmitting the composite image read from the composite image DB 41a (the composite image at the location corresponding to the request) to the user terminal 50 that has requested the composite image. In addition, a process of transmitting the real image read from the real image DB 41b (real image at the location corresponding to the request) to the user terminal 50 that has requested the real image under predetermined conditions is performed. Since the real image may include the privacy information of the target object 2, the real image is displayed by the image processing server 30 or the image providing server 40 in a special case such as an abnormal situation or an emergency situation regarding the target object 2. It is possible to transmit to the user terminal 50 only when it is determined that the condition is satisfied.

  FIG. 5 is a flowchart for explaining a registration process of various data performed by the image processing server 30. This process is a process required to generate a background image of a region corresponding to the captured image, and is executed when the image sensor device 10 is installed, for example.

  In step S1, a process of registering the 3D map data as the background data from which the background image is generated in the background DB 32c is performed. As the 3D map data, three-dimensional background data including an appropriate background area based on the number of image sensor devices 10 installed and the imaging range is adopted. The 3D map data may be a CG-based 3D map as well as a live-action-based 3D map. As the 3D map data, data that does not include the data of the target object 2 including privacy information (at least the data relating to privacy) is adopted. You may perform the process which acquires and registers 3D map data from an external database server, a recording medium (all are not shown), etc.

  In step S2, a process of registering internal parameters of the image sensor device 10 is performed. The internal parameters include the focal length of the image sensor device 10, the image center, the distortion aberration coefficient image, and the pixel size. The internal parameter acquisition method includes a method of acquiring the factory calibration information of the image sensor device 10, a method of acquiring the internal parameter using a tool (software) for acquiring internal parameters, and a calibration installed in the image processing server 30. A method of using a tool (software) to acquire an internal parameter using a captured image or the like can be adopted. The calibration of internal parameters can also be performed by acquiring a plurality of images including a calibration marker such as a chessboard and calculating the corresponding points of the chessboard on these images.

  In step S3, a process of registering external parameters of the image sensor device 10 is performed. The external parameter includes information regarding the position and orientation of the image sensor device 10. As a method of acquiring the external parameter, there is a method of acquiring the external parameter by using a calibration tool after the image sensor device 10 is installed. A general calibration method performs a process of associating a plurality of combinations of a point a whose three-dimensional position is known on a 3D map and a point a′ projected on the image, and a rotation that is an external parameter. The parameter R and the parallel displacement amount parameter T are calculated.

Formula 1 below represents a relational expression between the actual space (real space) and the display image in the image sensor device 10. X, Y, and Z are three-dimensional coordinates in the real space, R is a rotation parameter, T is a translation amount parameter, A is an internal parameter, and x and y are projection points on the image.

  In step S4, after the internal parameter and the external parameter are registered, adjustment processing of these parameters is performed. The parameters can be adjusted, for example, by displaying a background image generated using the 3D map data and an image (actual image) captured by the image sensor device 10 on the display unit 36. Depending on the accuracy of the 3D map data and the calibration accuracy of the external parameters, the background image and the actual image may be misaligned. In such a case, the parameters may be finely adjusted by a tool that executes a process of associating the actual image and the corresponding points on the 3D map on the image on the substantially same viewpoint.

  In step S5, the 3D map data, the internal parameters, and the external parameters that have been registered in advance are used to perform a process of generating a background image in the image capturing range corresponding to the image captured by the image sensor device 10, and then the process ends. The generated background image is an image including a structure existing in the background, and does not include the object 2 including privacy information. In addition, it is possible to generate several types of images as background images by changing the light source according to the image capturing time, reflecting the processing such as swaying the trees in the background image, and matching the scene obtained from the actual image. You can The generated background image is registered in the background image DB 32g in association with data such as the ID of the image sensor device 10 and the image acquisition date and time.

  FIG. 6 is a flowchart for explaining the composite image generation process performed by the image processing server 30. This processing operation is executed, for example, when there is a request to send a composite image from the image providing server 40 side. Alternatively, it may be executed at regular time intervals after the transmission request.

  First, in step S11, a process of acquiring an image from the image sensor device 10 is performed. The image acquired from the image sensor device 10 may be an image captured in real time, or an imaged image (saved image) stored in the storage unit 13 of the image sensor device 10.

  In step S12, a process of detecting the object 2 whose privacy is to be protected (disclosure is limited) is performed from the image acquired in step S11. In the present embodiment, the case where the target object 2 is set to a human has been described. However, the target object 2 is not limited to a human, and may be a target whose privacy should be protected. Further, the object 2 is not limited to one kind, and two or more kinds can be set as the object.

  The method of detecting the object (human) 2 on the image is, for example, a method of extracting and detecting a characteristic part of a human body (head, body, arms, legs, etc.), acquired continuous frame images. Various image recognition methods can be adopted, such as a method of detecting a person based on the difference between the two and a method of detecting a person based on the difference between the previously acquired reference background image and the acquired image.

  In step S13, a process of detecting the position of the target object 2 detected in step S12 is performed. The method of detecting the position of the target object 2 is to calculate the two-dimensional position (x, y coordinates or area) on the image described above, and calculate the three-dimensional position (x, y, z coordinates or area) on the 3D map. A method of calculating can be adopted.

In step S14, a composite image in which the model (doll) in which the privacy information of the object 2 is removed is projected and drawn on the background image generated in advance using the position data of the object 2 detected in step S13. Is performed, the generated combined image is stored in the storage unit 32, and then the process ends.
As a model synthesizing method, the above-described method, that is, a method of projecting and synthesizing a model on a background image generated in advance (two-dimensional synthesizing method), and three-dimensional position data on 3D map data are available. A method of arranging the models three-dimensionally based on the above, and cutting out an area corresponding to the imaging area of the image sensor device 10 as an image to generate a combined image (a three-dimensional combining method) can be adopted. When a three-dimensional synthesizing method is adopted, it is possible to generate a synthetic image with less discomfort in the state in which the model is drawn.

  FIG. 7 is a flowchart for explaining the processing operation executed in the data providing system 1 according to the embodiment (1). This processing operation is started, for example, when the external application 51 is activated on the user terminal 50.

  The user terminal 50 performs a process of starting the external application 51 in step S21. In step S22, when it is determined that the composite image request operation is performed, a process of transmitting a composite image request signal to the image providing server 40 is performed. The request signal for the combined image includes the ID (identifier) of the user terminal 50, information regarding the requested combined image, and the like. The information regarding the requested composite image includes information such as the target facility, place, information of the image sensor device 10 and date and time of image capturing. These pieces of information can be appropriately input on the external application 51 or may be set in advance.

  Next, the image providing server 40 performs a process of receiving a request signal for a composite image from the user terminal 50 in step S23. In the next step S24, based on the received request signal for the composite image, processing for transmitting the request signal for the composite image to the image processing server 30 is performed.

  Next, the image processing server 30 performs a process of receiving a request signal for a composite image from the image providing server 40 in step S25. In the next step S26, a process of acquiring data including a captured image (actual image) from the image sensor device 10 designated as the image acquisition target is performed based on the received request signal for the combined image. The acquired data includes actual image data, image capturing date/time data, the ID of the image sensor device 10, and the like, and these data are stored in the actual image DB 32a in a mutually associated format.

  Next, in step S27, a process of detecting (recognizing) the target object 2 from the acquired real image is performed. This process is a process corresponding to step S12 of FIG. 6 described above. In the next step S28, a process of detecting the position of the object 2 detected in the actual image is performed. This process is a process corresponding to step S13 of FIG. 6 described above. In subsequent step S29, the detected position data of the object 2 is stored in the RAM or the storage unit 32 in the control unit 34. The position data of the target object 2 is stored in a format associated with data related to the actual image such as the ID of the image sensor device 10 and the image pickup date/time data.

  Next, in step S30, the background image corresponding to the imaging range of the real image is read from the background image DB 32g, and in the next step S31, the corresponding real image is read from the RAM in the control unit 34 or the storage unit 32. A process of reading the detected position data of the object 2 is performed, and the process proceeds to step S32.

  In step S32, the position data of the object 2 read in step S31 and the background image read in step S30 are used to draw a model of the object 2 from which the privacy information of the object 2 is removed. Processing for generating the combined image. This process is a process corresponding to step S14 of FIG. 6 described above. In the subsequent step S33, the generated combined image is stored in the storage unit 32. The data of the generated composite image is stored in a format associated with the ID of the image sensor device 10, the image pickup date and time data, and the like. Next, in step S34, processing of transmitting the data of the composite image read from the storage unit 32 to the image providing server 40 is performed.

  In the image providing server 40, in step S35, the process of receiving the data of the composite image transmitted from the image processing server 30 is performed, and in the next step S36, the process of storing the received data of the composite image in the composite image DB 41a. I do. The composite image is stored in a form associated with data such as the ID of the image sensor device 10 and the image pickup date and time. In the next step S37, a process of reading the data of the composite image from the composite image DB 41a and transmitting it to the user terminal 50 is performed.

  In step S38, the user terminal 50 performs a process of receiving the composite image data transmitted from the image providing server 40, and in the next step S39, the received composite image data is stored in a storage unit (not shown). Is stored in step S40, and in step S40, a process of displaying the combined image on the display unit is performed. After that, in step S41, data processing using the composite image is executed by the external application 51. Then, in step S42, if an end operation of the external application 51 is detected, the end processing is performed, and then the processing ends.

  FIG. 8 is a display example for explaining a composite image generated by the image processing server 30, (a) is an image (actual image) captured by the image sensor device 10, and (b) is 3D map data. (C) is a composite image when an object is detected, and (d) is an image when the object fails to be detected.

  As shown in FIG. 8A, the object 2 is shown in the real image P1. As shown in FIG. 8B, the background image P2 generated based on the 3D map data does not include the target object 2. As shown in FIG. 8C, when the target object 2 is detected, the model 3 from which the privacy information is removed is combined with the background image P2. Since the model 3 of the target object 2 is displayed at the position on the background image P2 corresponding to the position of the target object 2 on the actual image P1, it is possible to protect the privacy of the target object 2.

In addition, since the model 3 is drawn so as to reproduce the position and posture (in this case, the standing state) of the target object 2, and the background image P2 corresponds to the imaging range of the actual image P1, The reproducibility of the area excluding the object 2 can be improved, and information such as the positional relationship with the object existing in the background can be accurately grasped.
The display form of the model 3 is not particularly limited as long as the privacy information is removed so that the individual who is the object 2 is not specified. For example, by using a model of the model 3 or a motion model corresponding to an attribute (male or female, adult or child, etc.) or a motion posture based on the image recognition result of the target object 2 by the image sensor device 10, It is possible to improve the reproducibility of the second position state. By increasing the reproducibility of the composite image, it is possible to enhance the utility value of the composite image, and it is possible to widely deploy the service using the composite image by a third party.

  Further, as shown in FIG. 8D, when the detection of the target object 2 fails, the background image P2 is presented instead of the actual image P1, so the target object 2 shown in the actual image P1 is presented. Without fail, the privacy of the object 2 is surely protected.

  Although the background image P2 corresponds to the imaging range of the real image P1 in the above-mentioned display example of the composite image, the present invention is not limited to this display form, and in another display example, the real image P1. It is possible to use a background image showing an area wider than the image pickup range of. For example, a region represented by a plurality of real images is cut out from the 3D map data as a background image, and a synthetic image in which the model 3 is drawn on the background image at the position of the object 2 detected in the plurality of real images is generated. You can also do it.

  According to the data providing system 1 according to the above-described embodiment (1), the privacy of the target object 2 is protected by the image processing server 30 that functions as an electronic device including a computer that executes the image recognition processing method according to the present embodiment. An appropriately protected composite image is generated, the image providing server 40 can provide the composite image to the user terminal 50, and the third party creates the composite image in which the privacy of the object 2 is appropriately protected. To provide a new service using the composite image through the external application 51.

  According to the image processing server 30 that executes the image recognition processing method according to the above embodiment (1), the position of the target object 2 is detected from the actual image acquired from the image sensor device 10, and the position data of the target object 2 is acquired. By using this, a synthetic image in which the model 3 is drawn at the corresponding position on the background image is generated. Therefore, the target object 2 whose position is detected from the actual image is drawn as the model 3 on the background image, so that a composite image that does not include the privacy information of the target object 2 can be generated. Further, since the model 3 of the object 2 is drawn on the background image, the image processing server 30 is used when continuously generating the composite image, as compared with a method of performing mosaic processing or the like for each captured image. The processing load can be reduced. Further, even if the detection of the target object 2 from the actual image fails, the background image is used for the image to be provided, and therefore the target object 2 that has failed to be detected is not presented to the user terminal 50, and the target object that has failed to be detected. The privacy of the object 2 can also be appropriately protected, a complicated process for checking that the privacy information of the object 2 is not leaked is unnecessary, and a simplified image recognition process can be realized.

FIG. 9 is a conceptual diagram for explaining the configuration of the data providing system according to the embodiment (2). However, the same components as those of the data providing system 1 shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.
In the data providing system 1 according to the embodiment (1), the image processing server 30 is configured to function as the image processing device 300 including a computer that executes the image recognition processing method according to the present embodiment. However, in the data providing system 1A according to the embodiment (2), the image sensor device 10A functions as the image processing device 100 (FIG. 10) including a computer that executes the image recognition processing method according to the present embodiment. Is configured.

  The data providing system 1A is configured to include an image sensor device 10A and an image providing server 40, and is a sensor system capable of providing an image in which the privacy of the object 2 including privacy information is appropriately protected to the user terminal 50. ing. The image sensor device 10A is connected to the image providing server 40 via the network 60. The image providing server 40 is connected to the user terminal 50 via the network 60. The number of installed image sensor devices 10A and image providing servers 40 is not particularly limited. It is sufficient that at least one or more data providing system 1A is equipped.

  The hardware configuration of the image sensor device 10A is substantially the same as the configuration shown in FIG. 2, but the image processing unit 12A, the control unit 14A, and the storage unit 13A have different functions. Further, the image sensor device 10A is equipped with the function of the router 20 (FIG. 1) for connecting to the network 60. The router 20 is not an indispensable component, and the image sensor device 10A may be directly connected to the network 60. The hardware configuration of the image providing server 40 is substantially the same as the configuration shown in FIG.

FIG. 10 is a functional block diagram for explaining processing executed by the image sensor device 10A and the image providing server 40.
[Image sensor device 10A]
The image processing apparatus 100 is installed in the image sensor device 10A. The image processing apparatus 100 includes an image acquisition unit 12c, a position detection unit 12d, a composite image generation unit 12e, a background image generation unit 12f, a background image registration unit 13b, an image switching unit 14a, and an image transmission unit 15a. These units are realized by the functions of the image processing unit 12A, the storage unit 13A, the control unit 14A, and the input/output I/F 15 of the image sensor device 10A.

The image acquisition unit 12c performs a process of acquiring the image captured by the image capturing unit 11, and the position detection unit 12d detects (recognizes) the target object (human) 2 including the privacy information from the acquired image and is captured. A process of detecting the position (coordinates or area) of the object (human) 2 on the image is performed. The above-described two-dimensional position calculation method and three-dimensional position calculation method are applied to the detection (image recognition) and position detection method of the object 2. The position data of the object 2 detected by the position detection unit 12d is stored in the RAM or the storage unit 13A in the control unit 14A in a state of being associated with data such as the ID (identifier) of the image sensor device 10A and the image acquisition date and time, for example. Remembered. The RAM in the control unit 14A or the storage unit 13A functions as a position data storage unit that stores the position data of the object 2.
A state in which the position (coordinates or area) data of the object 2 read from the RAM in the control unit 14A or the storage unit 13A is associated with data such as the ID (identifier) of the image sensor device 10A and the image acquisition date and time. Is sent to the composite image generation unit 12e.
The image acquired by the image acquisition unit 12c is stored in the actual image DB 13a in a state of being associated with data such as the ID (identifier) of the image sensor device 10A and the image acquisition date and time.

The background image generation unit 12f performs a process of generating a background image that does not include the target object 2. The background image registration unit 13b performs a process of registering the generated background image in the background image DB 13c.
Background image generation and registration processing according to the present embodiment includes (1) a method of registering an average image obtained by averaging a plurality of images captured by the image capturing unit 11 as a background image, and (2) an object 2 A method of registering an image captured under the condition that there is no image as a background image, and (3) a method of registering an image converted from a captured image into a painting-style image in which only the background is represented by image processing or an image converted into a CG image as a background image. Applied.

  The composite image generation unit 12e uses the position (coordinate) data of the target object 2 read from the RAM in the control unit 14A or the storage unit 13A and the background image read from the background image DB 13c, and the target object on the background image is displayed. At a detection position of 2, a model from which the privacy information of the object 2 is removed, for example, a CG image (doll) representing the silhouette of the object 2 is drawn to generate a composite image. The data of the generated composite image is stored in the storage unit 13A in a state of being associated with the ID of the image sensor device 10A and the data of the image acquisition date and time, and then sent to the image switching unit 14a.

When the position detection unit 12d calculates the two-dimensional coordinates on the image of the target object 2, the combined image generation unit 12e performs a process of generating a combined image in which a model is two-dimensionally arranged on the background image.
In addition, when the position detection unit 12d performs the position detection of one object 2 three-dimensionally using the plurality of image sensor devices 10A, the combined image generation unit 12e displays the image from the three-dimensional coordinates of the object 2. The two-dimensional coordinates are restored, and a process of generating a composite image in which the model is two-dimensionally arranged on the background image is performed based on the restored two-dimensional coordinates.

  The image switching unit 14a performs a process of switching between the image (actual image) acquired by the image acquisition unit 12c and the composite image generated by the composite image generation unit 12e as the image to be transmitted to the image providing server 40.

  The image transmitting unit 15a transmits the composite image or the actual image to the image providing server 40. Further, the image transmission unit 15a causes the management terminal device (not shown) in which the application software (internal application) 19 for managing the image sensor device 10A is installed to generate a composite image generated by the composite image generation unit 12e. It is also possible to send the image stored in the actual image DB 13a.

[Image providing server 40]
The image providing server 40 is an electronic device including a computer, and executes a process of transmitting a composite image generated by the image sensor device 10A or a captured real image to the user terminal 50.
The image receiving unit 42a performs a process of receiving the data of the composite image or the real image transmitted from the image sensor device 10A, and the received composite image or the data of the real image is stored in the composite image database 41a or the real image database 41b. Remembered.

The image transmitting unit 42b performs a process of transmitting the composite image read from the composite image DB 41a to the user terminal 50 that has requested the composite image. In addition, a process of transmitting the real image read from the real image DB 41b under a predetermined condition is performed to the user terminal 50 that has requested the real image.
Since the real image may include the privacy information of the target object 2, the real image may be displayed by the image sensor device 10A or the image providing server 40 in a special case such as an abnormal situation or an emergency situation regarding the target object 2. It is possible to transmit to the user terminal 50 only when it is determined that the condition is satisfied.

  FIG. 11 is a flowchart for explaining a processing operation executed in the data providing system 1A according to the embodiment (2). This processing operation is started, for example, when the external application 51 is activated on the user terminal 50.

  The user terminal 50 performs the process of starting the external application 51 in step S51. In step S52, when it is determined that the composite image request operation has been performed, a process of transmitting a composite image request signal to the image providing server 40 is performed. The request signal for the combined image includes the ID (identifier) of the user terminal 50 and information about the requested combined image. The information about the requested composite image includes information such as a target facility, a place, the information of the image sensor device 10A, and an imaging date/time. These pieces of information can be appropriately input by the external application 51, or may be set in advance.

  Next, the image providing server 40 performs the process of receiving the request signal of the composite image from the user terminal 50 in step S53. In the next step S54, processing for transmitting the request signal for the composite image to the image sensor device 10A is performed based on the received request signal for the composite image.

  Next, the image sensor device 10A performs a process of receiving a composite image request signal from the image providing server 40 in step S55. In the next step S56, a process of acquiring data including the captured image (actual image) specified as the acquisition target is performed based on the received request signal for the combined image. The acquired data includes actual image data, image capturing date/time data, the ID of the image sensor device 10A, and the like, and these data are stored in a form associated with each other in the actual image DB 13a.

  Next, in step S57, a process of detecting (recognizing) the target object 2 from the acquired real image is performed. This process is the same process as step S12 of FIG. 6 described above. In the next step S58, a process of detecting the position of the object 2 detected in the actual image is performed. This processing is the same processing as step S13 of FIG. 6 described above. In the following step S59, the detected position data of the object 2 is stored in the RAM in the control unit 14A or the storage unit 13A. The position data of the target object 2 is stored in a format associated with data relating to the actual image such as the ID of the image sensor device 10A and the image pickup date/time data.

  Next, in step S60, the background image corresponding to the imaging range of the real image is read from the background image DB 13c, and in the next step S61, the corresponding real image is read from the RAM in the control unit 14A or the storage unit 13A. A process of reading the detected position data of the object 2 is performed, and the process proceeds to step S62.

  In step S62, the position data of the object 2 read in step S61 and the background image read in step S60 are used to draw a model of the object 2 in which the privacy information of the object 2 is removed from the background image. A process for generating a combined image is performed. This processing is the same processing as step S14 of FIG. 6 described above. In a succeeding step S63, the generated combined image is stored in the storage unit 13A. The data of the generated composite image is stored in a format associated with the ID of the image sensor device 10A, the image capturing date and time data, and the like. Next, in step S64, a process of transmitting the data of the composite image read from the storage unit 13A to the image providing server 40 is performed.

  In the image providing server 40, in step S65, the process of receiving the data of the composite image transmitted from the image sensor device 10A is performed, and in the next step S66, the process of storing the received data of the composite image in the composite image DB 41a. I do. The composite image is stored in a format associated with data such as the ID of the image sensor device 10A and the date and time of image capturing. In the next step S67, a process of reading the data of the composite image from the composite image DB 41a and transmitting it to the user terminal 50 is performed.

  In the user terminal 50, in step S68, the process of receiving the data of the composite image transmitted from the image providing server 40 is performed, and in the next step S69, the received data of the composite image is stored in a storage unit (not shown). Is stored in step S70, and in step S70, the combined image is displayed on the display unit. Then, in step S71, data processing using the composite image is executed by the external application 51. After that, in step S72, when the ending operation of the external application 51 is detected, the ending processing is performed, and then the processing ends.

  According to the data providing system 1A according to the above-described embodiment (2), the privacy of the target object 2 is protected by the image sensor device 10A that functions as an electronic device including a computer that executes the image recognition processing method according to the present embodiment. An appropriately protected composite image is generated, the image providing server 40 can provide the composite image to the user terminal 50, and the privacy protection of the object 2 is appropriately protected to a third party. By providing, a new service using the composite image can be provided through the external application 51.

  According to the image sensor device 10A that executes the image recognition processing method according to the above embodiment (2), the position of the object 2 is detected from the image acquired from the image acquisition unit 12c, and the position data of the object 2 is used. As a result, a synthetic image in which the model 3 is drawn at the corresponding position on the background image is generated. Therefore, the target object 2 whose position is detected from the actual image is drawn as the model 3 on the background image, so that a composite image that does not include the privacy information of the target object 2 can be generated. Further, since the model 3 of the object 2 is drawn on the background image, the image sensor device 10A when continuously generating the composite image is compared with a method of performing mosaic processing or the like for each captured image. The processing load can be reduced. Further, even if the detection of the target object 2 from the actual image fails, the background image is used for the image provided to the user terminal 50, so that the target object 2 that has failed in detection is not presented to the user terminal 50. The privacy of the target object 2 that has failed to be detected can also be appropriately protected, and complicated processing for checking that the privacy information of the target object 2 is not leaked is not required, and a simplified image recognition process can be realized. ..

FIG. 12 is a conceptual diagram for explaining the configuration of the data providing system according to the embodiment (3). However, the same components as those of the data providing system 1 shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.
In the data providing system 1 according to the embodiment (1), the image providing server 40 provides the user terminal 50 with the composite image generated by the image processing server 30 (the image in which the model is combined with the background image). Was configured. On the other hand, in the data providing system 1B according to the embodiment (3), the background image generated by the image processing server 30A is transmitted to the user terminal 50A in advance, and then the data request from the user terminal 50A is made. Accordingly, the position data of the target object 2 is transmitted to the user terminal 50A, and the user terminal 50A uses the received position data of the target object 2 and the background image, and a model is drawn on the background image. It is configured to generate an image. That is, the user terminal 50A functions as a computer that executes the image recognition processing method according to the present embodiment.

The data providing system 1B includes an image sensor device 10, a router 20, an image processing server 30A, and a user terminal 50A, and is a sensor system capable of providing an image with appropriately protected privacy to the user terminal 50A. Is becoming The image processing server 30A is equipped with the image processing device 300A (FIG. 13).
The image sensor device 10 is connected to the router 20, and the router 20 is connected to the image processing server 30A via the network 60. The image processing server 30A can be connected to the user terminal 50A via the network 60. Note that the router 20 is not an indispensable component, and the image sensor device 10 may be connected to the network 60.

  The hardware configuration of the image sensor device 10 is substantially the same as the configuration shown in FIG. The hardware configuration of the image processing server 30A is substantially the same as the configuration of the image processing server 30 shown in FIG. 3, but the image processing unit 31A and the control unit 34A have different functions.

FIG. 13 is a functional block diagram for explaining processing executed by the image sensor device 10, the image processing server 30A, and the user terminal 50A. Components having the same functions as those in the functional block diagram shown in FIG. 4 are designated by the same reference numerals, and the description thereof will be omitted.
[Image sensor device 10]
The image acquisition unit 12a performs a process of acquiring an image captured by the imaging unit 11, and the target detection unit 12b performs a process of detecting (recognizing) a target object (human) 2 including privacy information from the acquired image. .. The acquired image and the data of the object detection (image recognition) result are transmitted to the image processing server 30A in a state of being associated with the data such as the ID (identifier) of the image sensor device 10 and the image acquisition date and time.

[Image processing server 30A]
The image processing server 30A is equipped with the image processing device 300A. The image processing device 300A includes an image receiving unit 33a, an image transmitting unit 33b, a position detecting unit 31a, a background image generating unit 31b, a position transmitting unit 31d, a background image transmitting unit 31e, and a 3D map registration unit 32b. These units are realized by the functions of the image processing unit 31A, the storage unit 32, the communication processing unit 33, and the control unit 34A of the image processing server 30A.

  The image receiving unit 33a performs a process of receiving the image transmitted from the image sensor device 10, and the received image (real image) is stored in the real image DB 32a. The position detection unit 31a performs a process of detecting the position of the detection target (human) 2 on the captured image. The above-described two-dimensional position calculation method or three-dimensional position calculation method is applied to the position detection method of the object 2. When the image processing server 30A receives the request signal from the user terminal 50A, the position transmitting unit 31d transmits the position data of the object 2 calculated by the position detecting unit 31a to the user terminal 50A. To do. Note that the function of the target detection unit 12b may be provided in the image processing server 30A.

  The background image generation unit 31b uses the 3D map data registered in the background DB 32c, the external parameters and internal parameters registered in the parameter DB 32f, and the like to generate a background corresponding to the imaging range of the image sensor device 10 from the 3D map data. Performs processing to generate an image. The generated background image is registered in the background image DB 32g in association with the ID of the image sensor device 10, for example. When the image processing server 30A receives the service (external application) use start signal from the user terminal 50A, the background image transmission unit 31e reads the background image generated by the background image generation unit 31b from the background image DB 32g, The process of transmitting to the user terminal 50A together with the ID of the image sensor device 10 associated with the image is performed.

[User terminal 50A]
The user terminal 50A is an electronic device equipped with a computer, and is configured by an information terminal device such as a general-purpose personal computer (PC), smartphone, or tablet terminal. The user terminal 50A includes a communication processing unit 52, a storage unit 53, a control unit 54, a display unit 55, and an operation unit 56.

  The communication processing unit 52 has the functions of the background image receiving unit 52a, the position receiving unit 52b, and the actual image receiving unit 52c, and includes a communication interface and a communication control circuit for transmitting and receiving various data via the network 60. It is configured. The storage unit 53 is configured by a semiconductor memory or the like that can store various data and the like received by the communication processing unit 52. The display unit 55 is configured to include various display panels such as a liquid crystal panel. The operation unit 56 is configured by an input device such as a touch panel, button switch, or keyboard. The control unit 54 has a function as a composite image generation unit 54a in addition to the function of controlling these units, and is configured to include a microcomputer (CPU (processor), RAM, ROM). The control unit 54 stores programs for executing the processes of these units. These programs may be stored in a computer-readable built-in recording medium or an external recording medium, for example, a removable semiconductor memory.

The background image receiving unit 52a performs a process of receiving the background image transmitted from the image processing server 30A. The background image received at the start of service use or the like is stored in the storage unit 53.
The position receiving unit 52b performs a process of receiving the position data of the target object 2 transmitted from the image processing server 30A. The received position data of the object 2 is stored in the RAM in the control unit 54, the storage unit 53, or the like in a state of being associated with data such as the ID (identifier) of the image sensor device 10 and the image acquisition date and time. The RAM in the control unit 54 or the storage unit 53 functions as a position data storage unit.
The composite image generation unit 54a uses the position data of the target object 2 read from the RAM or the storage unit 53 in the control unit 54 and the corresponding background image read from the storage unit 53, and the target object 2 on the background image. The model in which the privacy information of the target object 2 is removed is drawn at the detection position of 1, and a process of generating a combined image is performed, and the generated combined image can be displayed on the display unit 55.

  Further, the actual image receiving unit 52c performs a process of receiving the actual image transmitted from the image processing server 30A when the predetermined condition is satisfied, and the received actual image can be displayed on the display unit 55. .. Since the real image may include the privacy information of the object 2, the image processing server 30A determines that the special condition such as an abnormal situation or an emergency situation regarding the object 2 is satisfied in the real image. Only when it is done, it can be provided to the user terminal 50A.

  FIG. 14 is a flowchart for explaining the processing operation executed in the data providing system 1B according to the embodiment (3). This processing operation is started, for example, when the external application 51 is activated on the user terminal 50A.

  50 A of user terminals perform the process which starts the external application 51 in step S81. When it is determined in step S82 that the operation for requesting the composite image has been performed, processing for transmitting a request signal for the composite image to the image processing server 30A is performed. The request signal for the composite image includes the ID (identifier) of the user terminal 50A and the information about the requested composite image. The information regarding the requested composite image includes information such as the target facility, place, information of the image sensor device 10 and date and time of image capturing. These pieces of information can be appropriately input by the external application 51, or may be set in advance.

  Next, the image processing server 30A performs a process of receiving a request signal for a composite image from the user terminal 50A in step S83. In the next step S84, a process of transmitting the request signal for the composite image to the image sensor device 10 is performed based on the received request signal for the composite image, and the process proceeds to step S85. In step S85, the background image data corresponding to the imaging range of the image sensor device 10 is read from the background image DB 32g and transmitted to the user terminal 50A.

  In step S86, the user terminal 50A performs the process of receiving the background image data transmitted from the image processing server 30A, and in the next step S87, the process of storing the received background image data in the storage unit 53. I do.

  On the other hand, in the image sensor device 10, in step S88, the process of receiving the request signal for the composite image from the image processing server 30A is performed. In the next step S89, a process of acquiring data including the captured image (actual image) designated as the acquisition target is performed based on the received request signal for the combined image. The acquired data includes actual image data, image capturing date/time data, the ID of the image sensor device 10, and the like, and these data are stored in the storage unit 13 in a mutually associated format.

  In subsequent step S90, a process of detecting (recognizing) the target object 2 from the acquired real image is performed, and the recognition data of the detected target object 2 is stored in the storage unit 13. This process is the same process as step S12 of FIG. 6 described above. In the next step S91, the data including the actual image and the detection (recognition) data of the object 2 are read from the storage unit 13 and are transmitted to the image processing server 30A.

  In step S92, the image processing server 30A performs a process of receiving the data transmitted from the image sensor device 10, and in the next step S93, the data including the received real image is stored in the real image DB 32a and the object 2 is stored. A process of storing the detection data in the storage unit 32 is performed. In a succeeding step S94, a process of detecting the position of the object 2 detected (recognized) in the actual image is performed. This processing is the same processing as step S13 of FIG. 6 described above.

  In the following step S95, the detected position data of the object 2 is stored in the RAM or the storage unit 32 in the control unit 34A. The position data of the target object 2 is stored in a format associated with data related to the actual image such as the ID of the image sensor device 10 and the image pickup date/time data. In the next step S96, the position data of the object 2 corresponding to the actual image is read from the RAM in the control unit 34A or the storage unit 32 and is transmitted to the user terminal 50A.

  In step S97, the user terminal 50A receives the position data of the target object 2 transmitted from the image processing server 30A, stores the position data in the RAM in the control unit 54 or the storage unit 53, and proceeds to step S98. .. In step S98, the data of the corresponding background image is read from the storage unit 53, and in the next step S99, the process of reading the position data of the object 2 from the RAM in the control unit 54 or the storage unit 53 is performed.

  In the subsequent step S100, the position data of the read target 2 and the read background image are used to generate a composite image in which the model of the target 2 from which the privacy information of the target 2 is removed is drawn. Perform processing to This processing is the same processing as step S14 of FIG. 6 described above. In the subsequent step S101, the generated combined image is stored in the storage unit 53. The data of the generated composite image is stored in a format associated with the ID of the image sensor device 10, the image capturing date/time data, and the like.

  In a succeeding step S102, processing for displaying the combined image read from the storage unit 53 on the display unit is performed. Thereafter, in step S103, data processing using the composite image is executed by the external application 51. Then, in step S104, when an end operation of the external application 51 is detected, the end processing is performed, and then the processing ends.

FIG. 15 shows a display example of the composite image displayed on the display unit 55 of the user terminal 50A.
The display unit 55 displays a composite image in which the model (doll) 3 is composited with the background image P2. The composite image is displayed when the CG display button 56a is operated. When the CG display button 56a is operated, the user terminal 50A requests the image processing server 30A for the position data of the object 2, and the position data of the object 2 received in response to the request and the background image acquired in advance. Is used to generate a composite image and display it on the display unit 55.

  The actual image display button 56b does not function normally, but when the image processing server 30A detects an abnormal situation or an emergency situation relating to the object 2, for example, when the model 3 has fallen down for a certain period of time. In addition, the user can be warned by blinking the real image display button 56b, and the operation of the real image display button 56b can be accepted. When the actual image display button 56b is operated, the actual image received by the actual image receiving section 52c is displayed.

  According to the data providing system 1B according to the embodiment (3), the background image and the position data of the object 2 are separately transmitted from the image processing server 30A to the user terminal 50A, and the user terminal 50A synthesizes them. Since the image is generated, it is possible to provide the third party with the synthesized image in which the privacy is appropriately protected and expand the possibility of providing a new service using the synthesized image. Further, the background image is transmitted to the user terminal 50A in advance and stored therein, and only the information necessary for drawing the model 3 such as the detected position data of the object 2 is transmitted to the user terminal 50A in real time. Since the user terminal 50A generates an image in which the background image and the model 3 are combined, the user terminal 50A is compared with a configuration in which the captured image (real image) is transmitted to the user terminal in real time. It is possible to construct a system capable of significantly reducing the amount of data transferred (data transfer amount).

  It is needless to say that the present invention is not limited to the above-described embodiments, and various modifications can be made, which are also included in the scope of the present invention.

(Appendix 1)
An image recognition processing method for processing an image captured by at least one imaging unit by at least one processor,
By said at least one processor,
The position data of the object including privacy information detected from the image captured by the at least one imaging unit, and the background image read from the background image storage unit that stores the background image that does not include the object. And the step of generating a composite image in which the model of the object in which the privacy information of the object is removed is drawn on the background image.

(Appendix 2)
An image recognition processing method for processing an image captured by at least one imaging unit by at least one processor,
By said at least one processor,
Performing a process of detecting a position of an object including privacy information from an image captured by the at least one image capturing unit,
Using the detected position data of the target object and the background image read from the background image storage unit that stores the background image that does not include the target object, the background image contains privacy information of the target object. And a step of generating a composite image in which the model of the removed object is drawn.

(Appendix 3)
A computer-readable recording medium recording an image recognition processing program for causing at least one processor to execute processing of an image captured by at least one imaging unit,
The at least one processor,
Reading position data of the object from a position data storage unit that stores position data of the object including privacy information detected from an image captured by the at least one image capturing unit;
Reading the background image from a background image storage unit that stores a background image that does not include the object;
Using the read position data of the target object and the read background image, generating a composite image in which the model of the target object in which the privacy information of the target object is removed is drawn in the background image. A computer-readable recording medium recording an image recognition processing program to be executed.

(Appendix 4)
A computer-readable recording medium recording an image recognition processing program for causing at least one processor to execute processing of an image captured by at least one imaging unit,
The at least one processor,
Reading position data of the object from a position data storage unit that stores position data of the object including privacy information detected from an image captured by the at least one image capturing unit;
Reading the background image from a background image storage unit that stores a background image that does not include the object;
Using the read position data of the target object and the read background image, generating a composite image in which the model of the target object in which the privacy information of the target object is removed is drawn in the background image. A computer-readable recording medium recording an image recognition processing program to be executed.

(Appendix 5)
A data providing system in which at least one computer provides data to another computer via a network,
The at least one computer has a storage unit and at least one processor connected to the storage unit,
The storage unit is
A position data storage unit that stores position data of an object including privacy information detected from an image captured by at least one image capturing unit;
A background image storage unit that stores a background image that does not include the object,
The at least one processor is
Sending the position data of the object read from the position data storage unit to the another computer,
The background image read from the background image storage unit is configured to be transmitted to the another computer,
The other computer has at least one processor,
The at least one processor is
Using the position data of the target object and the background image received from the at least one computer, a composite image is created in which the model of the target object without the privacy information of the target object is drawn on the background image. A data providing system configured as follows.

1, 1A, 1B Data providing system 10, 10A Image sensor device 100 Image processing device 12a, 12c Image acquisition unit 12b Target detection unit 12d Position detection unit 12e Composite image generation unit 13b Background image registration unit 14a Image switching unit 15a Image transmission unit 20 router 30, 30A image processing server 300, 300A image processing device 31a position detection unit 31b background image generation unit 31c composite image generation unit 32b 3D map registration unit 40 image provision server 42a image reception unit 42b image transmission unit 50, 50A user Terminal 60 network

Claims (24)

An image recognition processing method for processing an image captured by at least one image capturing unit by at least one computer,
A background image in which position data of an object including privacy information detected from an image captured by the at least one image capturing unit and a background image generated from three-dimensional background data not including the object are stored. Using the background image read from the storage unit, and generating a composite image in which a model of the object without the privacy information of the object is drawn in the background image. Image recognition processing method. An image recognition processing method for processing an image captured by at least one image capturing unit by at least one computer,
Performing a process of detecting a position of an object including privacy information from an image captured by the at least one image capturing unit,
Using the detected position data of the target object and the background image read from the background image storage unit that stores the background image generated from the three-dimensional background data that does not include the target object, the background image And a step of generating a composite image in which the model of the object without the privacy information of the object is drawn.   The image recognition processing method according to claim 1, wherein the background image corresponds to an image capturing range of the captured image.   The synthetic image is a model in which the object model is drawn on the background image so that the position state of the object in the captured image is reproduced. The image recognition processing method according to any one of 1. The position data of the object is two-dimensional position data on the imaged image restored from the three-dimensional position data of the object detected from the imaged image. The image recognition processing method according to any one of 1 to 4 . The position data of the object is three-dimensional position data of the object detected from the captured image,
The background image is a the three-dimensional background data,
The step of generating the composite image comprises
The three-dimensional position data of the object is used to draw the model at the three-dimensional position of the object on the three-dimensional background data, and the imaging including the model from the three-dimensional background data on which the model is drawn. The image recognition processing method according to claim 1 or 2, wherein an image obtained by cutting out the generated image area is generated as the composite image. An image recognition processing program for causing at least one computer to execute processing of an image captured by at least one imaging unit,
The at least one computer,
Reading position data of the object from a position data storage unit that stores position data of the object including privacy information detected from an image captured by the at least one image capturing unit;
Reading the background image from a background image storage unit that stores a background image generated from three-dimensional background data that does not include the object;
Using the read position data of the target object and the read background image, generating a composite image in which the model of the target object in which the privacy information of the target object is removed is drawn in the background image. An image recognition processing program characterized by being executed. An image recognition processing program for causing at least one computer to execute processing of an image captured by at least one imaging unit,
The at least one computer,
Performing a process of detecting a position of an object including privacy information from an image captured by the at least one image capturing unit,
Storing the detected position data of the object in a position data storage unit;
Reading the position data of the object from the position data storage unit;
Reading the background image from a background image storage unit that stores a background image generated from three-dimensional background data that does not include the object;
Using the read position data of the target object and the read background image, generating a composite image in which the model of the target object in which the privacy information of the target object is removed is drawn in the background image. An image recognition processing program characterized by being executed. An image recognition processing program for causing at least one computer to execute processing of an image captured by at least one imaging unit,
The image recognition processing method according to any one of claims 3-6, the image recognition processing program characterized by executing at least one computer. A computer-readable recording medium in which the image recognition processing program according to any one of claims 7 to 9 is recorded. A processor which reads and executes the image recognition processing program according to any one of claims 7 to 9 . An electronic device comprising the processor according to claim 11 . A method for providing data executed in a system including at least one computer capable of communicating via a network, comprising:
The at least one computer is
Data, characterized in that it includes the step of transmitting the synthesized image generated by the step of generating the composite image in the image recognition processing method according to any one of claims 1 to 6 to another computer How to provide. 14. The data providing method according to claim 13 , further comprising the step of transmitting the captured image to the another computer when a predetermined condition that requires the captured image is satisfied. A method for providing data executed in a system including at least one computer capable of communicating via a network, comprising:
The at least one computer is
Data provided which is characterized in that it includes the step of displaying the synthesized image generated in the step of generating the composite image in the image recognition processing method according to any one of claims 1 to 6 on the display unit Method. 16. The data providing method according to claim 15 , further comprising a step of switching the composite image and the captured image to display the image on the display unit. A data providing system in which at least one computer provides data to another computer via a network,
The at least one computer is
A position data transmission unit for transmitting position data of an object including privacy information detected from an image picked up by at least one image pickup unit to the another computer;
A background image transmission unit that transmits a background image generated from three-dimensional background data that does not include the object to the another computer,
To the other computer,
A data providing system characterized by using the position data of the object and the background image to generate a composite image in which the model of the object without the privacy information of the object is drawn on the background image. .. A data providing program to be executed by a system including at least one computer capable of communicating via a network,
The at least one computer,
10. Data provision, characterized in that the step of transmitting the composite image generated in the step of generating the composite image in the image recognition processing program according to any one of claims 7 to 9 to another computer is executed. program. The at least one computer,
The data providing program according to claim 18 , further comprising transmitting the captured image to the another computer when a predetermined condition that requires the captured image is satisfied. A data providing program to be executed by a system including at least one computer capable of communicating via a network,
The at least one computer,
A data providing program, characterized by causing a step of displaying the composite image generated in the step of generating the composite image in the image recognition processing program according to any one of claims 7 to 9 on a display unit. .. The at least one computer,
21. The data providing program according to claim 20 , further comprising: executing the step of switching the composite image and the captured image to display on the display unit. A computer-readable recording medium in which the data providing program according to any one of claims 18 to 21 is recorded. A processor for reading and executing the data providing program according to any one of claims 18 to 21 . An electronic device comprising the processor according to claim 23 .

Images