JP7468097B2 - Image sales system and program - Google Patents

Description

The present invention relates to an image sales system and program.

There is a known service that sells photos taken by photographers at events such as marathons, sporting events, and school events. The image data taken by the photographer is uploaded to a photo sales system via a network. Customers can view the uploaded images, select their favorite photos, and purchase them. Purchased images can then be made available for download, or printed copies can be delivered to the customer.

Customers had to search through the large number of images posted on the photo sales system to find images that included themselves or their family members, making ordering a time-consuming process.

One known method for extracting a desired image from a large number of images is the face search function. With this face search function, for example, a user uploads an image in which they appear to a photo sales system. The photo sales system detects faces in the images uploaded by the user, and searches the images for sale for images that contain faces similar to the detected face. This allows users to easily find images in which they appear.

There is a demand for a service that uses this face search function to extract scenes in which a person appears from video footage shot by a cameraman and allow the user to purchase the images (stills).

JP 2001-325082 A JP 2014-10788 A

The present invention has been made in consideration of the above-mentioned conventional situation, and aims to provide an image sales system and program that sells images extracted from videos by face search.

The image sales system according to the present invention includes a storage unit that stores video data including a plurality of images, an image receiving unit that receives search images including a search target person from a user terminal, a setting unit that accepts search condition settings including a range of face orientations and a threshold of face similarity from the user terminal, an analysis unit that extracts images that satisfy the search conditions from the plurality of images included in the video data, including faces whose similarity to the face image of the search target person in the search images is equal to or greater than the threshold and whose face orientation is within the range, and a sales processing unit that accepts purchases of the images extracted by the analysis unit from the user terminal.

The program according to the present invention causes a computer to execute the steps of accepting settings of search conditions including a range of face orientations and a threshold value of face similarity, and extracting images that satisfy the search conditions from a plurality of images included in the video data, including faces whose similarity to the face image of the person to be searched for in the search image is equal to or greater than the threshold value and whose face orientation is within the range.

According to the present invention, images extracted from videos using face search can be sold.

1 is a schematic diagram of an image sales system according to an embodiment of the present invention. FIG. 2 is a schematic configuration diagram of a server according to the embodiment. FIG. 2 is a functional block diagram realized by executing an image processing program. 1 is a flowchart illustrating an image sales method. FIG. 13 is a diagram illustrating the direction of a face. FIG. 13 is a diagram showing an example of image extraction.

The following describes an embodiment of the present invention with reference to the drawings.

Figure 1 is a schematic diagram of an image sales system according to an embodiment of the present invention. The image sales system includes a server 1 as an image management device that provides a website for image sales, and an administrator terminal 2 used by a system administrator. The images for sale are, for example, images (still images) cut out from videos taken by a photographer of events such as sporting events or school events. Users who purchase images are participants in the event or the parents of the participants.

A user accesses the image sales site using a user terminal 4 such as a smartphone or PC, and uploads and registers an image (search image) that shows the user or a person to be searched for, such as a family member. The user can extract frames (images) in which the user appears from videos by performing a face search using the registered image. The user selects the image to purchase from among the extracted images. Various search conditions can be set for the face search. The search conditions will be described later.

When a user selects an image to purchase and completes the payment process, the user can download the image data and place an order for a print. When the server 1 accepts an order for a print, it transmits the data of the purchased image to the factory 5. The print 6 produced at the factory 5 is delivered to the user. The print 6 is, for example, an L or 2L size photograph, an A4 size poster, etc.

As shown in FIG. 2, the server 1 is a computer having a CPU 10 (central processing unit), a memory unit 11, a communication unit 12, and the like. The communication unit 12 performs communication processing with the administrator terminal 2 and the user terminal 4, and transmits and receives data. The communication unit 12 receives video data from the administrator terminal 2 and stores it in the memory unit 11.

The communication unit 12 transmits page information of the image sales site to the user terminal 4. The communication unit 12 also accepts the selection of images to be purchased from the user terminal 4 and receives payment information.

The storage unit 11 is a combination of flash memory, ROM (Read-only Memory), RAM (Random Access Memory), a hard disk, etc., and stores video data, image processing programs, search images received from the user terminal 4 for face search, member information, etc.

The member information is information about the member who uses the image sales site provided by the server 1, and includes the ID and password for logging in to the image sales site, name, address, etc.

When the CPU 10 executes the image processing program, the functions of an authentication processing unit 100, an image receiving unit 101, a setting unit 102, an analysis unit 103, a display processing unit 104, and a sales processing unit 105 are realized, as shown in FIG. 3. The processing of each unit will be described later.

Next, the image sales method using this image sales system will be explained with reference to the flowchart shown in Figure 4.

When a user accesses the image sales site using the user terminal 4, the user is prompted to enter an ID and password. When the user enters the ID and password, the authentication processing unit 100 refers to the member information stored in the memory unit 11 and performs authentication processing (step S1).

After authentication, the server 1 requests the user terminal 4 to register an image for face search. The user uploads and registers an image (search image) from the user terminal 4 that shows the person to be searched, such as the user or a family member (step S2). The image receiving unit 101 of the server 1 stores the search image received from the user terminal 4 in the memory unit 11.

The setting unit 102 accepts the setting of search conditions (step S3). The setting unit 102 accepts the setting of the range of face orientations to be extracted and the threshold value of face similarity (%) as search conditions. The face orientation is expressed by the rotation angles of three axes, Yaw, Pitch, and Roll, as shown in FIG. 5, for example, and accepts the setting of the range of each rotation angle.

The setting unit 102 may accept condition settings regarding the quality of the photo, such as brightness and sharpness.

In this embodiment, images are extracted from video data, so there is a high possibility that the profile or back of the person being searched for will be captured in the time period before or after the image that satisfies the search criteria. Therefore, the setting unit 102 may accept the setting of the time range for image extraction based on the time of the image that satisfies the criteria. For example, the setting of time ranges T1 and T2 may be accepted, such as "Extract images from T1 seconds (first time) before to T2 seconds (second time) after the image that satisfies the search criteria."

After the search conditions are set, the analysis unit 103 performs image analysis on multiple images (frames) included in the video data, and extracts images that satisfy the search conditions as images for sale (step S4). For example, the analysis unit 103 detects a face in each image and calculates the orientation of the detected face. The analysis unit 103 also calculates the similarity between the detected face and the search target face image in the search image using a known method. The analysis unit 103 extracts images where the calculated similarity is equal to or greater than a predetermined threshold and the orientation of the face is within a set range. The analysis unit 103 also extracts images from the time period before and after the image that satisfies the search conditions as images for sale.

For example, as shown in FIG. 6, the analysis unit 103 extracts images F2 to F4 that satisfy the search conditions, image F1 that is in the range from T1 seconds before the time of image F2, and images F5 and F6 that are in the range from T2 seconds after the time of image F4 as images for sale.

The analysis unit 103 may analyze all images in the video data, or images at a fixed interval. For example, if the video data is 30 fps (frames per second), it contains 30 images per second. All 30 images may be analyzed, or 10 of the 30 images may be analyzed at 0.1 second intervals. The longer the time interval between images to be analyzed, the shorter the time required for analysis processing.

Images in the video data may be analyzed at a first time interval, and if an image that satisfies the search target is detected, images within a specified time range based on the time of the detected image may be analyzed at a second time interval that is shorter than the first time interval.

The display processing unit 104 generates a viewing page including a list screen of the extracted images, and transmits it to the user terminal 4 for display (step S5).

The user views the extracted images on the user terminal 4 and selects the image they wish to purchase (step S6). Images can be purchased as data (data for download), photographs, posters, etc. If a photograph or poster is selected, the user also inputs the number of images to be purchased. The sales processing unit 105 accepts the purchase selection of the image for sale from the user terminal 4.

After the image to be purchased is selected, the sales processing unit 105 performs a payment process (step S7). After the payment process, the server 1 makes the purchased image data available for download and transfers the print order details to the factory 5.

In this way, according to this embodiment, images of good scenes in which the user himself/herself is in the video and his/her face is oriented in a suitable way can be cut out and sold.

Of the images extracted in step S4 of the flowchart in FIG. 4, images that do not satisfy the search conditions, i.e., images taken before or after the images that satisfy the search conditions (e.g., images F1, F5, and F6 in FIG. 6), may not include the person being searched for, so certain restrictions may be placed on the purchase of the images. For example, only printed matter such as photographs and posters may be available for purchase, and image data may not be purchased (downloaded). This is because digital content poses a high risk of being misused, such as being spread on social media. In addition, an upper limit may be placed on the number of printed matter such as photographs and posters that can be purchased. The upper limit on the number of purchases may differ depending on the type (photograph/poster).

For images (scenes) that were not extracted in step S4, only video viewing may be permitted, and the images may not be available for purchase.

In the above embodiment, an example was described in which the server 1 performs image analysis, but the analysis process may also be performed by an external analysis terminal (not shown).

By installing and executing an image processing application (application program) on the user terminal 4, the user terminal 4 may have the functions of the setting unit 102, analysis unit 103, and display processing unit 104. In this case, the user terminal 4 accepts search condition settings from the user and extracts images that satisfy the search conditions from the video data in the user terminal. The extracted images are stored in the user terminal 4 as image data (still image data).

The present invention is not limited to the above-described embodiments as they are, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be deleted from all of the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

Reference Signs List 1 Server 2 Administrator terminal 4 User terminal 10 CPU
REFERENCE SIGNS LIST 11 Storage unit 12 Communication unit 100 Authentication processing unit 101 Image receiving unit 102 Setting unit 103 Analysis unit 104 Display processing unit 105 Sales processing unit

Claims (2)

A storage unit that stores video data including a plurality of images;
an image receiving unit that receives a search image including a person to be searched from a user terminal;
A setting unit that receives settings of search conditions including a range of face orientations and a threshold value of face similarity from the user terminal;
an analysis unit that extracts, from the plurality of images included in the video data, images that satisfy the search conditions, including a face whose similarity to a face image of the person to be searched for in the search image is equal to or greater than the threshold value and whose facial orientation is within the range;
a sales processing unit that accepts purchases of the images extracted by the analysis unit from the user terminal;
Equipped with
the analysis unit further extracts, from the plurality of images included in the video data, images from a first hour before the time of the image satisfying the search condition and images from a second hour after the time of the image satisfying the search condition;
An image sales system characterized in that, among the images extracted by the analysis unit, images that satisfy the search conditions can be purchased in the form of prints and data downloads, and images that do not satisfy the search conditions can be purchased in the form of prints . receiving a search image including a person to be searched from a user terminal;
receiving , from the user terminal, a setting of search conditions including a range of face orientations and a threshold value of face similarity;
extracting, from a plurality of images included in the video data, images that satisfy the search conditions, including a face whose similarity to a face image of a search target person in the search image is equal to or greater than the threshold value and whose facial orientation is within the range, as well as images from a first hour before the time of the image that satisfies the search conditions and images from a second hour after the time of the image that satisfies the search conditions ;
accepting, from the user terminal, purchases of the extracted images, with the purchase form of the images that satisfy the search conditions being prints and data downloads, and the purchase form of the images that do not satisfy the search conditions being prints;
A program that causes a computer to execute the following.

Images