JP2019068316A - Image processing system and image processing apparatus - Google Patents

Abstract

To limit utilization of image, in a system for synthesizing a captured image and a frame image.SOLUTION: An image processing system has imaging means for imaging a subject, storage means for storing multiple frame images having a transmission region, selection means for making a user select a frame image for use in synthesis, from the multiple frame images stored in the storage means, synthesis means for creating a composite image by synthesizing the captured image in the transmission region of the frame image, and output means for outputting the composite image by one or more output methods. The storage means stores the frame image in association with the limit information associated with use restriction of the frame image, the selection means determines selection propriety of the corresponding frame image on the basis of the limit information, or the output means limits the output method of the composite image created by using a corresponding frame image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to techniques for processing images.

There is known an apparatus and software for combining an image of a person, a character, and the like with an image captured by a camera and generating a combined image. For example, software for portable terminals such as stationary print seal machines and smartphones has become widespread (see Patent Document 1). In particular, by using a portable terminal such as a smartphone possessed by a user, it is possible to easily combine images.
An image such as a person or a character to be combined with the image is also called a frame. The business operator can appeal the product to the user by supplying a new frame or a highly rare frame.

Unexamined-Japanese-Patent No. 2013-197803

  On the other hand, when images are synthesized on a terminal owned by the user, there arises a problem that the obtained image data can be used without restriction. For example, if it is possible to save an image after composition, it is assumed that the image will be posted to a social network or the like, but the rights holder of the frame can not track where the image was uploaded. Further, since the image after combination is electronic data, the risk of alteration or the like is greater than that of a print, a seal or the like.

  The present invention has been made in consideration of the above problems, and an object of the present invention is to limit the use of images in a system for combining a captured image and a frame image.

  An image processing system according to the present invention is an image processing system that combines a captured image obtained by capturing an object and a frame image. The captured image is an image captured by the user. The frame image is an image to be combined with the captured image, and is an image including, for example, a person (entertainer), a character, and the like.

  The image processing system according to the present invention may be used for synthesis from an imaging unit for acquiring the captured image, a storage unit for storing a plurality of frame images having a transmission area, and a plurality of frame images stored in the storage unit. Selecting means for causing the user to select a frame image to be selected, combining means for combining the captured image in the transmission region of the frame image to generate a combined image, and outputting the combined image by one or more output methods And output means, wherein the storage means stores limit information which is information associated with the use limit of the frame image in association with the frame image, and the selection means stores the limit information based on the limit information. Determining whether or not to select a corresponding frame image, or the output means limits an output method of a composite image generated using the corresponding frame image And wherein the door.

The imaging means is a means for imaging a subject. The imaging means may be a stationary imaging device or a portable camera-equipped computer (such as a smartphone) possessed by the user.
The storage means is a means for storing a plurality of frame images. The frame image has a transmission area, and the synthesizing unit synthesizes the image by synthesizing the captured image in the transmission area. That is, the frame image is the foreground (front side layer), and the captured image is the background (back side layer).

The selection means is a means for causing the user to select a frame image to be used for composition.
Note that “selection” in the present invention indicates that a frame image is selected by the user and the frame image becomes actually usable. For example, when the storage means is connected via a network, the user's selection may deliver the target frame image to the local device. In addition, when the storage means exists in the local device, the target frame image may be available by user selection.

The generated composite image is output by one or more output methods.
The output method is, for example, provision to a user (storage on a local device), upload to a server (storage on a remote device), and the like. It is also possible to select an output method that does not leave the data of the composite image at hand.

The image processing system according to the present invention is characterized by storing restriction information in association with a frame image, determining whether or not to select a frame based on the related restriction information, or restricting a method of outputting a composite image. Do.
The restriction information is information for restricting the use of a frame image (including a composite image).
For example, usage conditions for a frame image such as “permit selection only in a predetermined geographical area” or “permit selection only in a predetermined period” are represented by restriction information, and the selection means Whether to select a frame image is determined according to the restriction information. Here, when there is a frame image determined to be unselectable, the frame image can not be selected (ie, used).

  In addition, for example, the use condition for the composite image is represented by the restriction information such as “do not allow storage on the local device”, “do not allow transmission via the network”, “permit only transmission to a predetermined server device” The output unit restricts the method of outputting the composite image in accordance with the restriction information.

According to this configuration, it is possible to control how the frame image is used according to the contents of the restriction information associated with the frame image.
For example, if the idle included in the frame image is subject to a strong restriction on the use of portraits, do not allow image storage on the local device, but allow transmission only to servers that provide dedicated print services. It becomes possible to cope.

  Further, the image processing system according to the present invention further includes position information acquisition means for acquiring position information of the imaging means, and the restriction information is information representing a geographical place where the corresponding frame image can be selected. The selection unit may select a corresponding frame image when it is determined that the position information conforms to the restriction information.

The position information acquisition means may not necessarily be built in the imaging means. Also, the position information may not necessarily be accurate position information of the imaging means. For example, position information of a portable terminal having an imaging unit may be used.
By providing the restriction information with information on a geographical place (for example, a region) where the frame image can be used, it is possible to set a frame image which can be selected only in a predetermined place.
According to this configuration, it is possible to improve the ability to attract customers at commercial facilities, concert halls and the like.

  Further, the restriction information further includes information on a period in which the corresponding frame image can be selected, and the selection means can select the corresponding frame image when it is determined that the current date and time conform to the restriction information. It may be characterized by

  By using date and time information in addition to position information, it is possible to select a frame image only at a predetermined place and predetermined date and time. That is, it is possible to use a frame image limited to a specific event.

  Further, the one or more output methods include provision by data of the composite image, the restriction information includes a permission flag for provision by data of the composite image, and the output unit is based on the restriction information. The method may be characterized in that the output method is limited.

  As described above, by making it possible to set whether to provide the composite image data to the user, it is possible to limit the use of the composite image data. If the provision of composite image data is not permitted, for example, the composite image may be output to a device for printing an image.

  Further, the output means indicates that the restriction information does not permit provision of the composite image as data, and a screen shot image is generated for the composite image generated using the corresponding frame image. It may be characterized in that when it is detected that it has been acquired, the saved screen shot image is deleted.

  As described above, when a screen shot is acquired under a situation where storage of composite image data is not permitted, the stored image may be automatically deleted. According to this configuration, it is possible to prohibit the storage of the composite image by data at all.

Furthermore, the image processing system according to the present invention further includes acquisition means for acquiring subject information from the second subject imaged through the imaging means, and the restriction information is information associated with the subject information. The selection means may make it possible to select a frame image associated with the subject information.
Further, the subject information may be information encoded by a two-dimensional barcode attached to the second subject.

For example, a related frame image can be selected by reading subject information (for example, a character string attached to a product or a character string encoded by a barcode) from a second object (for example, an arbitrary product) It will be. This makes it possible to add value to the product.
The subject information may be a character string or encoded data, but may be data representing the nature of the subject itself. For example, the result of classifying the attribute of the subject may be used as the subject information.

  In the image processing apparatus according to the present invention, an imaging unit for imaging a subject and acquiring a captured image, and a plurality of frame images having a transmission area stored in the storage unit are used for combining. Selecting means for the user to select, combining means for combining the captured image in the transmission region of the frame image to generate a combined image, and output means for outputting the combined image by one or more output methods; , And the frame image is stored in association with restriction information which is information associated with the restriction of use of the frame image, and based on the restriction information, the selection means determines whether or not the corresponding frame image can be selected. It is characterized in that the determination or the output means limits the output method of the synthesized image generated using the corresponding frame image.

  The present invention can be specified as an image processing system or an image processing apparatus including at least a part of the above means, or can be specified as an image processing method performed by the above image processing system or image processing apparatus. The above-mentioned processes and means can be freely combined and implemented as long as no technical contradiction arises.

  According to the present invention, it is possible to limit the use of images in a system for combining a captured image and a frame image.

It is a block diagram of the image processing system concerning a first embodiment. It is an example of a frame image. It is a figure explaining composition of an image pick-up picture and a frame picture. 5 is an example of frame data stored in a storage unit 203. It is a flowchart figure of the processing which personal digital assistant 100 performs. It is a block diagram of the image processing system which concerns on 2nd embodiment. It is a figure explaining the process of step S11 in 2nd embodiment. It is a figure explaining the process of step S11 in 3rd embodiment.

(First embodiment)
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. The image processing system according to the first embodiment is a system that combines an image (hereinafter referred to as a frame image) for composition with an image captured by a portable terminal to generate a composite image.
Specifically, it comprises a portable terminal 100 which is a terminal possessed by a user, a server apparatus 200 for providing a service, and a print server 300 for providing a print service of an image.

FIG. 1 is a system configuration diagram of an image processing system according to the first embodiment.
The portable terminal 100, the server device 200, and the print server 300 are connected via a network. A plurality of frame images are stored in the server device 200, and a predetermined frame image is transferred to the portable terminal 100 in response to a request from the portable terminal 100. The portable terminal 100 also has a camera, and has a function of combining a selected frame image with a captured image to generate a combined image.

The portable terminal 100 is a small computer that has a processor and a memory and can process data. The portable terminal 100 is typically a smartphone, a tablet computer, or the like, but may be other than this.
The portable terminal 100 includes an image acquisition unit 101, a communication unit 102, a control unit 103, and a storage unit 104.

The image acquisition unit 101 is a means for capturing an image through a built-in camera. The image captured by the image acquisition unit 101 is transmitted to the control unit 103 described later, and is used for image composition.
Hereinafter, the image acquired by the image acquisition unit 101 is referred to as a captured image.

The communication unit 102 is communication means for connecting the portable terminal 100 to the server apparatus 200 via a network (for example, the Internet or VPN). The communication unit 102 may be a wired network interface or a wireless network interface. It may also be means for accessing a mobile telephone network or a public wireless network.

  The control unit 103 is a unit that controls the entire portable terminal 100. Specifically, it manages connection between the mobile terminal 100 and the server device 200, selection and acquisition of frame images, transmission and reception of information, and the like. Specific processing content will be described later.

  The storage unit 104 is means for temporarily or permanently storing data used by the mobile terminal 100. It is preferable to use a large-capacity storage medium capable of reading and writing at high speed as the storage unit 104. For example, a flash memory can be suitably used. The storage unit 104 stores, for example, a captured image acquired by the image acquisition unit 101, a frame image distributed from the server apparatus 200, a composite image after combination, and the like. The specific content of the data will be described later.

  Control of each means which portable terminal 100 has is realized when processing devices, such as CPU, run control programs. Further, the control may be realized by a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like, or may be realized by a combination thereof. Also, it may be realized by dedicated hardware.

Next, the server device 200 will be described.
The server apparatus 200 includes a communication unit 201, a control unit 202, and a storage unit 203.
The server device 200 includes an arithmetic processor such as a CPU, a main storage device such as a RAM, an auxiliary storage device such as an HDD, an SSD, or a DVD-ROM, a wired or wireless communication device, an input device such as a keyboard or a mouse, and a display It can be configured as a computer including the device. The server apparatus 200 does not necessarily have to be configured of one computer, and the functions described below may be realized by cooperation of a plurality of computers.

  Similarly to the communication unit 102, the communication unit 201 is means for performing communication with the portable terminal 100 by accessing the network through the communication line.

  The control unit 202 is a unit that controls the entire server apparatus 200. Specifically, connection of the portable terminal 100 and the server device 200, management and provision of frame images, transmission and reception of data, and the like are managed. Specific processing content will be described later.

The storage unit 203 is means for storing a plurality of frame images provided to the mobile terminal 100.
Here, the frame image stored in the storage unit 203 will be described. FIG. 2 is an example of a frame image provided to the portable terminal 100. In the drawing, a hatched portion means a region to which a transparent color is assigned (hereinafter, a transparent region).

  Here, an example in which the captured image illustrated in FIG. 3A is combined with the frame image illustrated in FIG. 2 will be described. In the present embodiment, the transmission area of the frame image is an area for combining the captured image. That is, the frame image is a foreground (front side layer), and the captured image is a rear view (back side layer), so that the synthesis is performed. When the frame image shown in FIG. 2 and the captured image shown in FIG. 3A are combined, a composite image as shown in FIG. 3B is obtained.

The storage unit 203 stores information incidental to the frame image as well as the frame image as illustrated. FIG. 4 shows data actually stored in the storage unit 203 in the form of a table. In the present embodiment, the storage unit 203 stores a frame identifier (ID), image data (binary data), and an output restriction flag in association with each other. These data, that is, the information associated with the frame image are collectively referred to as frame data hereinafter.

  The output restriction flag is an identifier for restricting the use of the image after composition. The output restriction flag indicates, for example, a condition such as “permit / not to save the composite image in the portable terminal 100” or “permit or not to transmit the composite image to the external service”. This will be described later.

The print server 300 is an apparatus that receives an order for print service in response to a request from a user who holds the portable terminal 100. The print server 300 may be disposed at a store that prints on demand. The print server 300 may be integrated with a printer. In the case of printing on demand, the composite image can be printed at the place where the order is placed and handed over to the user.
The print server 300 is not an essential component of the present invention. For example, the print server 300 may be shared with the server apparatus 200, or may be omitted when the print service is not performed.

  Next, the process in which the mobile terminal 100 according to the present embodiment performs image composition will be described with reference to the flowchart in FIG. The process illustrated in FIG. 5 is started by a user operation as a trigger and executed by the control unit 103.

  First, in step S11, frame data corresponding to a frame image to be used is acquired from the server apparatus 200. When a plurality of frame images are stored in the server apparatus, the user may select one to be used. In the first embodiment, no limitation is imposed on the selection of frame images. The frame data transmitted from the server device 200 is temporarily stored in the storage unit 104 while holding the format shown in FIG.

  Next, in step S12, a captured image is acquired via the image acquisition unit 101. At this time, the preview screen and the frame image may be overlaid and displayed on a display unit (not shown) to allow the user to confirm how the image is captured.

  Next, in step S13, the obtained captured image and frame image are combined to generate a combined image. As described above, the composition is performed by fitting the captured image into the transmission region of the frame image.

Next, in step S14, the generated composite image is output. Here, it is assumed that the following options can be selected as an output method: (1) save the composite image as image data in the storage unit 104; (2) transmit the composite image to the print server 300 and order a print The output restriction flag in one embodiment indicates whether or not to prohibit (1). That is, when the output restriction flag is set to 1, storage of the composite image in the local device is prohibited, and only ordering on a paper medium is permitted. When the output restriction flag is set to 0, both the operations (1) and (2) are possible.

  As described above, according to the image processing system according to the first embodiment, the method of using the image obtained by combining can be restricted by attaching information to the frame image. Thus, for example, when a talent with a strong restriction on the use of portraits is a frame image, it is not permitted to save the image on the local device, but is permitted to transmit only to a server that provides a dedicated print service. Such correspondence becomes possible.

In the first embodiment, “save image to portable terminal 100” and “order for print service” are exemplified as the output method, but the portable terminal 100 can determine the output method by the output restriction flag. For example, there may be other output methods.
For example, a flag relating to pay sale of a composite image or a frame image may be provided, so that use or output can be made only for those whose payment has been completed. In addition, it may have a value regarding the number of times of downloading or the number of times of combining.

  Further, in the present example, an example in which a frame image is acquired each time from the server apparatus 200 has been described, but if there is a frame image acquired in the past and stored in the storage unit 104, the frame image may be reused. Good.

Further, the output restriction flag may have information indicating whether the screen shot can be acquired. For example, when the information to the effect that the capture of the screen shot is not permitted is set in the frame image, and the operation for capturing the screen shot on the composite image is detected, the operation is invalidated or a warning is issued. An operation of displaying a message may be performed.
Furthermore, when the control unit 103 has the access right to the storage where the screen shot image is stored, the control unit 103 may display a warning message and perform an operation of automatically deleting the stored image. . According to such a configuration, it is possible to perform control so as not to leave any composite image as data at hand of the user.

Second Embodiment
In the first embodiment, the aspect of limiting the output method for each frame image has been described. On the other hand, the second embodiment is an embodiment in which the use of a frame image is restricted based on the position information of the portable terminal 100.

In the second embodiment, the mobile terminal 100 further includes a position information acquisition unit 105.
The position information acquisition unit 105 is means for acquiring position information (latitude and longitude) of the own terminal using a GPS device (not shown) provided in the terminal. The acquired position information is transmitted to the server device 200 via the communication unit 102.

  In the second embodiment, as shown in FIG. 4B, the “permitted position” is added to the frame data stored in the storage unit 104. The permission position is information that describes a geographical location that permits the use of the corresponding frame image. The permitted position may be, for example, one representing a range (for example, latitude and longitude at a plurality of points) or may represent a specific point. Also, it may be a combination of a specific point and information on the range (for example, the shape of the area).

FIG. 7 shows the process performed in step S11 in more detail in the second embodiment.
First, in step S111, the position information acquisition unit 105 acquires position information (for example, a combination of latitude and longitude) of the own terminal. The acquired position information is transmitted to the server device 200 in step S112.

In step S113, the server apparatus 200 (control unit 202) extracts only the one corresponding to the received position information from the stored frame data from the storage unit 203, and presents it to the user via the portable terminal 100. For example, when a valid frame image is stored only within a predetermined range (for example, within a specific facility), is it possible to compare the received position information with the permitted position associated with the frame image and to permit the use? Decide whether or not. Here, the frame image determined not to permit use is not presented to the user. That is, it becomes impossible to select.
Thus, in step S113, based on the position information of the portable terminal 100, the frame image selectable by the user is filtered.
The steps after the process of selecting the frame image to be used by the user are the same as in the first embodiment, and thus detailed description will be omitted.

  As described above, according to the image processing system according to the second embodiment, the delivery control of the frame image can be performed based on the position information by attaching the information to the frame image. As a result, for example, it becomes possible to cope with distributing a frame image only at a specific event site (such as a concert), which can lead to a customer attraction.

  In the second embodiment, the availability (selectability) of the frame image is determined based on only the position information, but additional conditions may be added. For example, in addition to the position information, a period for permitting use of the frame image may be set, and use (selection) of the frame image may be permitted only during a predetermined period (for example, during an event holding period).

Further, in the second embodiment, the limitation is imposed only on the delivery of the frame image, but the same limitation can be imposed on the reuse of the frame image delivered in the past. For example, it is assumed that a frame image acquired in the past is stored in the portable terminal 100. In this case, in step S112, processing is performed to compare the position information of the own terminal with the permitted position associated with the stored frame image. Then, only the frame image satisfying the condition may be selected in step S13.
In either case, it is common to the point that the corresponding frame image can not be used (selected) if the condition is not satisfied.

Third Embodiment
In the second embodiment, the frame image permitted to be used is determined based on the place and the date and time. On the other hand, the third embodiment is an embodiment in which the portable terminal 100 transmits information included in a captured image to the server device 200, and the server device 200 distributes a corresponding frame image.
The configurations of the portable terminal 100 and the server device 200 according to the third embodiment are the same as those of the first embodiment, and therefore the description thereof is omitted.

In the third embodiment, as shown in FIG. 4C, “supplementary information” is added to the frame data stored in the storage unit 104. The incidental information corresponds to a code for the portable terminal 100 to obtain a corresponding frame image.
The incidental information is attached to a specific product or a specific place (e.g., an event hall, etc.), for example, in a form encoded in a two-dimensional barcode. The portable terminal 100 reads this, decodes the information, and transmits it to the server device 200, whereby the corresponding frame image is distributed.

FIG. 8 shows the process performed in step S11 in more detail in the third embodiment.
First, in step S114, the control unit 103 acquires an image via the image acquisition unit 101. Here, when a two-dimensional barcode is detected in the image, the contained information is decoded and transmitted to the server apparatus 200 in step S115.

In step S116, the control unit 202 of the server apparatus 200 extracts from the storage unit 203 a frame image (frame data) whose received information matches the incidental information, and delivers the frame image to the portable terminal 100. Thus, in step S116, based on the information transmitted from the portable terminal 100, the frame image to be distributed is determined.
Note that the distributed frame image may be available only on the spot, or may be stored and available at an arbitrary timing.

  As described above, according to the image processing system according to the third embodiment, the frame image can be distributed by encoding the incidental information and attaching it to an arbitrary product or place. As a result, for example, it is possible to deliver a frame image only to a user who has purchased a specific product or a user who has visited a specific event site (such as a concert), which can lead to a customer attraction.

(Modification)
The above embodiment is merely an example, and the present invention can be implemented with appropriate modifications without departing from the scope of the invention.

  For example, in the description of the embodiment, the form in which the portable terminal 100 and the server device 200 are separated is exemplified, but both may be the same device. Moreover, the portable terminal 100 may not necessarily be a terminal owned by the user. For example, it may be a terminal lent to a user at a store.

Further, in the description of the embodiment, a mode is presented in which only the available frame images are presented to the user for selection among the plurality of frame images stored in the server device 200, but other modes may be used. Frame images and unavailable frame images may be selected.
For example, the portable terminal 100 collectively acquires and stores a plurality of frame image data stored in the server device 200, extracts only available frame images under current conditions, and allows the user to present and select It is also good. In this case, unavailable frame images may be hidden. Further, in this case, the portable terminal 100 may perform the processes of steps S112 to S113.

  Further, in the description of the embodiment, a mode in which the mobile terminal 100 performs image combining has been described, but the server device 200 may perform image combining. In this case, the captured image may be transmitted from the portable terminal 100 to the server device 200, and the composite image generated by the server device 200 may be transmitted to the portable terminal 100.

100 ... portable terminal 101 ... image acquisition unit 102, 201 ... communication unit 103, 202 ... control unit 104, 203 ... storage unit 200 ... server device

Claims (8)

An image processing system that combines a captured image obtained by capturing an object with a frame image,
Imaging means for acquiring the captured image;
Storage means for storing a plurality of the frame images having a transmission area;
Selection means for causing a user to select a frame image to be used for composition from a plurality of frame images stored in the storage means;
Combining means for combining the captured image with the transmission area of the frame image to generate a combined image;
Outputting means for outputting the composite image by one or more output methods;
Have
The storage means stores limit information, which is information associated with the use limit of the frame image, in association with the frame image.
The selection means determines whether or not the corresponding frame image can be selected based on the restriction information, or the output means restricts an output method of a composite image generated using the corresponding frame image. And an image processing system. It further comprises position information acquisition means for acquiring position information of the imaging means,
The restriction information is information representing a geographical location from which the corresponding frame image can be selected,
The image processing system according to claim 1, wherein the selection means enables selection of a corresponding frame image when it is determined that the position information conforms to the restriction information. The restriction information further includes information on a period in which the corresponding frame image can be selected,
The image processing system according to claim 2, wherein the selection means enables selection of a corresponding frame image when it is determined that the current date and time conforms to the restriction information. The one or more output methods include providing the data of the composite image,
The restriction information includes a permission flag for provision of the composite image by data;
The image processing system according to any one of claims 1 to 3, wherein the output unit restricts the output method based on the restriction information. The output means indicates that the restriction information does not permit provision of data of the composite image, and a screen shot image is obtained for the composite image generated using the corresponding frame image. 5. The image processing system according to claim 4, wherein the stored screen shot image is deleted when an event is detected. The image processing apparatus further comprises acquisition means for acquiring subject information from the second subject imaged through the imaging means,
The restriction information is information associated with the subject information,
The image processing system according to any one of claims 1 to 5, wherein the selection means enables selection of a frame image associated with the subject information. The image processing system according to claim 6, wherein the subject information is information encoded by a two-dimensional barcode attached to the second subject. Imaging means for imaging a subject and acquiring a captured image;
Selection means for causing a user to select a frame image to be used for composition among a plurality of frame images having a transmission area stored in the storage means;
Combining means for combining the captured image with the transmission area of the frame image to generate a combined image;
Outputting means for outputting the composite image by one or more output methods;
Have
The frame image is stored in association with restriction information which is information associated with the use restriction of the frame image.
The selection means determines whether or not the corresponding frame image can be selected based on the restriction information, or the output means restricts an output method of a composite image generated using the corresponding frame image. And an image processing device.

Images