JP2024025156A - image processing system - Google Patents

Abstract

To reduce the time and effort for a user to generate stamp data conforming to guidelines.SOLUTION: An image processing system has: generation means that generates stamp data available in a messenger application by performing image processing on photographed image data; and determination means that determines whether the stamp data generated by the generation means conforms to guidelines.SELECTED DRAWING: Figure 15

Description

The present disclosure relates to an image processing system, a server device, an imaging device, a processing method for an image processing system, a processing method for a server device, a processing method for an imaging device, and a program.

In recent years, with the spread of information processing terminals such as smartphones that are capable of capturing still images and moving images and equipped with communication functions, various types of communication using images in addition to voice and text have become commonplace.

Messenger apps are communication application software that runs on information processing terminals. In addition to text messages, Messenger apps use image data called stamps and stickers, which are illustrations of phrases, emotions, and reactions commonly used in daily conversation, to expand the range of communication and allow users to have fun with each other. used. The stamp data is a default data set included in the above-mentioned messenger app, and is provided as an environment in which several patterns of stamps prepared in advance can be used.

In addition, by accessing a predetermined online store linked to the messenger app and purchasing stamp data, users can add and use various stamps that suit their tastes. Furthermore, individual users can sell their own original stamps on an online store by passing a predetermined screening process determined by the messenger app operating company.

On the other hand, in order for an individual user to create and sell original stamps, it may be necessary to separately purchase drawing software, pen tablets, and other specialized equipment. In addition, in order to sell on an online store, the number of sets of designs required is large, numbering in the dozens, and the products must pass a review to ensure that they comply with guidelines, making it necessary to go through multiple processes.

Therefore, it takes a long time to prepare all the designs, and depending on the content of the design, the examination result may be rejected and sent back, and you may have to reapply for examination after making corrections. Because the game requires redoing, there is a high barrier to entry for beginner users.

Patent Document 1 discloses that by photographing the subject's face and the background around the face, and trimming the background around the face of the photographed data to a pre-specified size, one photograph can be used to print various applications and cards. A method for forming a facial image that matches the size of a facial photograph according to predetermined specifications is disclosed.

Japanese Patent Application Publication No. 2001-8018

By applying the method of Patent Document 1, it is possible to save the user the trouble of creating an illustration from scratch and to obtain image data for predetermined use that conforms to the guidelines.

However, Patent Document 1 only has means for automatically correcting the angle of view to be a predetermined angle of view. Patent Document 1 does not include the conversion process to the above-mentioned illustration data for stamps and the configuration that adapts the data content to a format that complies with the guidelines, resulting in streamlining the examination flow up to sales. do not have. Therefore, the method of Patent Document 1 still has the problem that it takes a lot of effort for an individual user to create and sell an original stamp.

An object of the present disclosure is to enable a reduction in a user's effort to generate stamp data that conforms to guidelines.

The image processing system includes a generation unit that generates stamp data that can be used in the messenger app by processing captured image data, and a process that determines whether the stamp data generated by the generation unit conforms to guidelines. and determining means for determining.

According to the present disclosure, it is possible to reduce the user's effort to generate stamp data that conforms to guidelines.

1 is a diagram illustrating a configuration example of an image processing system. 1 is a diagram showing an example of the configuration of an imaging device. It is a diagram showing an example of the configuration of a server device. 3 is a flowchart illustrating a processing method of the imaging device. It is a flowchart which shows the processing method of a server apparatus. FIG. 7 is a diagram illustrating an example of a pre-approval menu screen of the imaging device. FIG. 3 is a diagram showing an example of a stamp illustration style selection screen. FIG. 6 is a diagram illustrating an example of displaying the number of remaining images on the imaging device. FIG. 3 is a diagram showing an example of voice guided photography by an imaging device. FIG. 3 is a diagram illustrating an example of live view display guide shooting of the imaging device. FIG. 7 is a diagram illustrating an example of stamp preview operation of the imaging device. FIG. 7 is a diagram illustrating an example of an examination NG notification menu for an imaging device. FIG. 7 is a diagram showing an example of a network model relearning operation using examination results. 3 is a flowchart illustrating a processing method of the imaging device. It is a flowchart which shows the processing method of a server apparatus.

(First embodiment)
The first embodiment will be described in detail below based on the drawings. Note that the following embodiments do not limit the scope of the claims. Although a plurality of features are described in the embodiment, not all of these features are essential, and the plurality of features may be arbitrarily combined. Furthermore, in the drawings, the same or similar components are designated by the same reference numerals, and redundant description will be omitted.

<System configuration>
FIG. 1 is a diagram showing an example configuration of an image processing system 100 according to the first embodiment. The image processing system 100 includes an imaging device 200, a server device 300, a database 400, a server device 500, a mobile terminal 600, and a network 700. Network 700 is, for example, the Internet. These devices are interconnected via a network 700, and control signals and data are transmitted and received between the devices, and depending on the confidentiality and confidentiality of the data being handled or the requirements for acceptable transmission delay, a dedicated line may be used. , or they may be connected using physical cables.

In addition to the shooting function, the imaging device 200 has a display function that displays and outputs various setting values at the time of shooting and image data after shooting based on user operations, and a display function that outputs audio to the photographer and the main subject using a built-in speaker or the like. It has a voice guidance function that provides shooting guidance. Furthermore, the imaging device 200 has a function of connecting to a network 700 using a wireless communication method such as a public wireless communication network or WiFi, and transmitting and receiving data to and from the server devices 300 and 500. The imaging device 200 is typically a digital camera, a video camcorder, or the like, but is not limited to any device as long as it has similar functions.

The server device 300 is a server device for checking stamp guideline conformity. After receiving the stamp data created by performing image processing on the image data photographed by the imaging device 200, the server device 300 analyzes and checks the stamp data based on predetermined inspection rules, and requests the server device 500 for stamp sales review. Determine whether or not it is possible.

Here, the above inspection rules apply the requirements that must be met when selling stamps as set by the business operator who manages and operates the server device 500 as basic rules, and are further determined based on the examination results of stamp data submitted for sale in the past. Also check.

The server device 300 stores the stamp data examination results in a large-capacity database 400 connected to the server device 300. Then, the server device 300 transmits the determination result to the imaging device 200 for stamp data that cannot be sold based on the determination result, and prompts re-photography and stamp data creation.

On the other hand, if the determination result indicates that the stamps can be sold and the generation of a predetermined number of stamps has been completed, the server device 300 transmits the stamp data together with the stamp creation user's account information to the server device 500 and handles the sales application on behalf of the server device 500. Perform processing.

The server device 500 is a server device for distributing a social networking service (hereinafter referred to as SNS). The server device 500 is a server device that realizes a community-type membership service that promotes and supports connections between people on the Web, or a website that provides the service. It is assumed that the server device 500 is operated as a server that performs sales screening of SNS applications designated by a business operator, or as a server that also includes a website function.

For example, the SNS is LINE (registered trademark), FaceBook Messagener (registered trademark), Snapchat (registered trademark), WhatsApp (registered trademark), KakaoTalk (registered trademark), WeChat (registered trademark), or the like. The SNS is not limited as long as it is a service that realizes communication between users, sales, and screening functions using the stamps described above.

Further, the server device 500 can send and receive data, including stamp data sent from the server device 300, to and from the mobile terminal 600 connected to the network 700 in order to realize the above-mentioned service.

The mobile terminal 600 is a device owned by the same user as the imaging device 200, and, like the imaging device 200, has a function of connecting to the network 700 using a wireless communication method and an application program (hereinafter referred to as Equipped with a SNS app). The mobile terminal 600 allows many users to communicate with each other via the server device 500 using stamp data generated from image data captured by the imaging device 200 via the SNS application. Furthermore, the mobile terminal 600 receives the examination result of the stamp sales examination request via the server device 300. The mobile terminal 600 is, for example, a smartphone or a tablet terminal, but is not limited thereto as long as it has similar functions.

The above is the configuration of the image processing system 100. Next, the configuration and processing contents of the imaging device 200 and the server device 300 will be explained.

<Configuration of imaging device 200>
FIG. 2 is a diagram illustrating an example of the hardware configuration of the imaging device 200 in FIG. 1. The imaging device 200 includes a CPU 201, a primary storage section 202, an imaging section 203, an image processing section 204, a communication section 205, a recording/playback section 206, a display section 207, an audio processing section 208, and an operation section 209. has. Furthermore, the imaging device 200 has a CPU bus 210, a work memory 211, and a memory bus 212.

The CPU 201 is a Central Processing Unit. The CPU 201 performs functions such as starting and stopping the imaging unit 203, image processing unit 204, communication unit 205, recording/playback unit 206, display unit 207, audio processing unit 208, and operation unit 209, and notifying interrupts via the CPU bus 210. Perform system control. Further, the CPU 201 controls various operations of the imaging device 200.

The primary storage unit 202 is a storage area used as a work area of the CPU 201, etc. The primary storage unit 202 is realized by, for example, a DRAM (Dynamic Random Access Memory), an SRAM (Static Random Access Memory), a nonvolatile flash memory, or the like. For example, the CPU 201 implements various functions provided by the imaging device 200 by developing and executing a control program stored in the primary storage unit 202.

The imaging unit 203 includes a camera unit such as a lens and a CCD, and an optical unit, converts light captured from the lens into an electrical signal using a sensor, and then converts digital RAW image data into Bayer format via a memory bus 212. The data is then output to a work memory 211 such as a large-capacity DRAM.

The image processing unit 204 performs debayer processing (demosaic processing) on the RAW image data output by the imaging unit 203, converts it into a signal consisting of luminance and color difference, and then removes noise and optical distortion contained in each signal. A so-called development process is performed to correct and optimize the image. Thereafter, the image processing unit 204 performs image compression encoding on the developed image data based on standardized standards such as JPEG and HEVC, and outputs the compressed image data to the work memory 211 . Further, the image processing unit 204 reads the compressed data from the recording/reproducing unit 206, develops it in the work memory 211, and then performs decompression processing again to restore it to image data.

Further, the image processing unit 204 has a function of converting photographed image data after development processing into illustration data for stamps. The image processing unit 204 converts the captured natural image into a two-dimensional illustration. The conversion method includes color reduction processing using dithering and binarization, gradation correction, various filter processing such as edge filters and smoothing filters, and even multiple combinations of these to create manga style, anime style, oil painting, and pixel art style. Method etc. Further, the conversion method may be a method such as style conversion using content images and style images using DNN (Deep Neural Network), which has been attracting attention in recent years, and is a method that achieves the same image conversion function. If so, it is not limited.

The above image conversion function is realized by dedicated hardware such as a DSP (Digital Signal Processor) or wiring logic, and can perform high-speed real-time processing. Note that the above-mentioned image conversion function may be realized by software processing of the CPU 201 to achieve equivalent function and performance, and is not limited thereto.

The communication unit 205 includes a wireless communication interface that complies with various communication standards of public wireless communication such as LTE and 5G, and wireless LAN (Local Area Network) such as WiFi. The communication unit 205 connects to external devices via the network 700 and sends and receives image data and other various information. Furthermore, the communication unit 205 includes a short-range communication interface such as infrared communication or Bluetooth, and transmits and receives audio data and control signals to and from wireless earphones, remote controllers, and the like.

The recording/reproducing unit 206 is realized by a USB memory, a memory card, a hard disk drive, a nonvolatile memory represented by a flash memory, etc., and writes and reads various information and data. For example, the recording/reproducing unit 206 is mainly used as a secondary storage unit for storing or reproducing image data temporarily stored in the work memory 211.

The display unit 207 includes a display device such as an EVF (Electric View Finder) or an LCD (Liquid Crystal Display). The display unit 207 displays graphics data such as characters and menu icons created by the CPU 201, image data and decoded image data developed by the image processing unit 204, and stamp illustration data on the display device.

The audio processing unit 208 uses a device such as a speaker built into the imaging apparatus 200 to perform digital-to-analog conversion on audio data stored in the primary storage unit 202 or the work memory 211 and outputs the converted data.

The operation unit 209 has operation means including one or more operation members for the user to input various operation instructions. For example, the operation unit 209 detects an instruction to switch the shooting mode using a button or a switch mechanism, or detects a contact position using a touch panel integrated with the LCD of the display unit 207, and sends the detected information to the CPU 201. Notice.

The CPU bus 210 is a control bus that connects the CPU 201 and each of the processing blocks described above, and may be a standardized bus similar to the memory bus 212, or if there is sufficient processing margin, a low-speed I2C or other bus. A serial method may also be used, and there is no limitation.

The work memory 211, like the primary storage unit 202, is realized by DRAM or SRAM. The work memory 211 stores data as a temporary work area. The work memory 211 outputs RAW image data output by the imaging unit 203, image data input/output by the image processing unit 204, data received by the communication unit 205 from an external device, and superimposed on image data by the display unit 207. Stores menus, character data, etc. Further, the work memory 211 stores audio data and the like to be processed by the audio processing unit 208.

A memory bus 212 connects an imaging unit 203, an image processing unit 204, a communication unit 205, a recording/playback unit 206, a display unit 207, an audio processing unit 208, and an operation unit 209 to the work memory 211, and stores image data and various parameters. A data bus for transferring data at high speed. The bus transfer method of the memory bus 212 is not limited, and may be a standard bus standard such as ISA, PCI-Express, or AXI, or a proprietary bus method.

The above is an example of the hardware configuration of the imaging device 200. Next, the server device 300 will be explained.

<Configuration of server device 300>
FIG. 3 is a diagram showing an example of the hardware configuration of the server device 300 in FIG. 1. The server device 300 includes a CPU 301 , a primary storage section 302 , an image processing section 303 , a communication section 304 , a recording/playback section 305 , a display section 306 , an operation section 307 , and a machine learning and inference processing section 308 . Further, the server device 300 includes a CPU bus 309, a work memory 310, and a memory bus 311.

The CPU 301 performs system control such as starting and stopping the image processing unit 303, communication unit 304, recording/playback unit 305, display unit 306, operation unit 307, machine learning and inference processing unit 308, and notifying interrupts via the CPU bus 309. and controls various operations of the server device 300. For example, the CPU 301 implements various functions provided by the server device 300 by developing and executing a control program stored in the primary storage unit 302, work memory 310, or recording/reproducing unit 305.

The primary storage unit 302 is a storage area used as a work area of the CPU 301, etc. The primary storage unit 302 is realized by, for example, DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), or the like.

The image processing unit 303 has a function of decoding and analyzing stamp data received from the imaging device 200 via the communication unit 304. In the decryption process, the image processing unit 303 checks whether the file format (file extension) is in accordance with the stamp guideline defined by the operator of the server device 500 and whether it can be decrypted. The image processing unit 303 acquires basic image format information such as the image size, number of horizontal pixels, number of vertical pixels, number of colors, and pixel bit depth of the stamp data as analysis processing.

Furthermore, the image processing unit 303 separates and extracts text and non-text parts in the stamp data using, for example, an OCR method. The image processing unit 303 then processes the correctness of the text content, the ratio of the text part in the stamp data to the illustration part generated from objects other than text, the presence or absence of the illustration part, the degree of overlap and area between the illustration part and the text part, etc. Calculate. Then, the image processing unit 303 calculates a histogram or dynamic range of the number of colors and pixel values present in the data over the entire illustration portion and text portion including the background.

The functions performed by the image processing unit 303 are realized by dedicated hardware such as a DSP (Digital Signal Processor) or wiring logic, and can perform high-speed real-time processing. Note that the functions performed by the image processing unit 303 are not limited, and equivalent functions and performance may be realized by software processing of the CPU 301.

The communication unit 304 is an interface for connecting the server device 300 and an external device. The server device 300 uses the communication unit 304 to send and receive various information and data to and from external devices via the network 700.

The network 700 is, for example, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), a public line, or the like. Note that the network 700 is not limited as long as it can establish transmission and reception of information to and from the imaging device 200, the server device 500, and the mobile terminal 600. Further, the network 700 may be a wireless network or a wired network, and may include the plurality of different types of networks described above.

The recording/reproducing unit 305 is realized by a nonvolatile memory such as a hard disk drive or an SSD (Solid State Drive), and writes and reads various information, data, or programs. In addition, as another example, the recording and reproducing unit 305 writes and writes information, data, and programs to an external recording medium using an optical disk such as a DVD-RW, CD-ROM, CD-R, and DVD-RAM. It may be realized by a device that performs reading.

Furthermore, the recording/reproducing unit 305 is connected to the database 400 of FIG. 1, and records the stamp examination results in the database 400 or reads the stamp examination results from the database 400.

The display unit 306 is an output interface that displays display information such as images on a screen and presents various information to the user, and is realized by a display device such as an LCD. Note that the server device 300 does not necessarily need to include the display section 306, and in the case of a configuration that does not include the display section 306, it does not need to have a display control function for controlling the operation of the display section 306.

The operation unit 307 is an input interface that accepts instructions from the user, and is realized by, for example, a keyboard, a pointing device, a touch panel, or the like. Note that the server device 300 does not necessarily need to include the operation unit 307, and in the case of a configuration that does not include the operation unit 307, it does not need to have an operation unit control function for controlling the operation of the operation unit 307. .

The machine learning and inference processing unit 308 has a function of learning and inferring a model configured by a neural network. The machine learning and inference processing unit 308 re-learns the model and infers the content appearing in the stamp data based on the stamp data requested for review and the stamp review results, and outputs the stamp sales guideline conformity determination as a pre-check result. do.

The CPU bus 309 is realized by the same configuration as the CPU bus 210 in FIG. 2, and has similar functions.

Work memory 310, like primary storage unit 302, is realized by DRAM or SRAM. The work memory 310 stores, as a temporary work area, image data input and output by the image processing unit 303, data received by the communication unit 304 from an external device, data output by the display unit 306, and the like.

Memory bus 311 is realized by the same configuration as memory bus 212 in FIG. 2, and has similar functions.

<Operation flow of imaging device 200>
FIG. 4 is a flowchart showing a processing method of the imaging device 200. The flowchart in FIG. 4 is mainly executed by the CPU 201 of the imaging device 200 executing a program.

First, in step S401, the CPU 201 uses a mode setting from the operation unit 209 having an interface such as a button, a cross key, or a touch panel as a trigger, and determines whether the operation mode of the imaging device 200 is the shooting mode. If the operating mode is not the shooting mode (NO in step S401), the CPU 201 ends the process of the flowchart of FIG. 4, and if the operating mode is the shooting mode (YES in step S401), the process proceeds to step S402.

In step S402, the CPU 201 determines whether the stamp creation mode is selected by the user's operation on the operation unit 209. If the stamp creation mode is not selected (NO in step S402), the CPU 201 returns to step S401, and if the stamp creation mode is selected (YES in step S402), the process advances to step S403.

In step S403, the CPU 201 activates the stamp creation function of the imaging device 200 and displays the pre-approval menu screens shown in FIGS. 6(a) to 6(c) on the display unit 207. Specifically, the CPU 201 automatically generates stamp data from a series of photographed images and displays on the display unit 207 the service details and precautions regarding the procedure of acting on behalf of the customer for requesting sales examination.

<Example of pre-approval menu screen>
FIGS. 6A to 6C are diagrams showing examples of service function pre-approval menu screens displayed in step S403. FIG. 6(a) is a diagram showing the first page of the pre-approval menu screen, FIG. 6(b) is a diagram showing the second page of the pre-approval menu screen, and FIG. 6(c) is a diagram showing the pre-approval menu screen. It is a figure which shows the 3rd page of.

The pre-approval menu screens shown in FIGS. 6(a) to 6(c) show the contents for which the user is required to consent in advance in order to use the service. The final page of FIG. 6(c) is a screen for confirming whether or not you agree with the written content.

The screen in FIG. 6(a) is a screen that states that the preliminary check on the server device 300 is only a function to make it easier to pass the main examination. The screen shown in FIG. 6(b) is a screen that states that the stamp data applied for sale and the examination results will be used for machine learning and improving the identification accuracy of the inference model. Further, the screen shown in FIG. 6(b) is a screen that states that a portion of the proceeds from the stamps sold by the user after passing the sales examination will be returned to the provider who provides a series of services. Note that other incidental terms and restrictions may be added to the pre-approval menu screen, and the wording and expression thereof are not limited.

Returning to FIG. 4, in step S404, the CPU 201 determines whether the user has agreed to the service content on the screen of FIG. 6(c) through the user's operation on the operation unit 209. Specifically, the CPU 201 determines whether or not the user has consented to the service content by inputting from the operation unit 209 such as a button corresponding to the consent icon on the screen in FIG. 6(c), a cross key, or a touch panel. Determine. If the user does not agree with the service content (NO in step S404), the CPU 201 returns to step S401, and if the user agrees with the service content (YES in step S404), the process proceeds to step S405.

In step S405, the CPU 201 causes the image processing unit 204 to display the illustration style selection screen shown in FIG. 7 for selecting what style of stamp illustration to create based on the photographed real image data. 207.

<Example of illustration style selection screen>
FIG. 7 is a diagram showing an example of the stamp illustration style selection screen displayed in step S405. The screen in FIG. 7 is a screen for displaying stamp illustration styles and accepting selection instructions from the user. For example, the screen in FIG. 7 displays an image of the completed stamp image and a simple explanation of the illustration style for four stamp illustration styles. Further, the screen shown in FIG. 7 includes a user interface in which the user selects and moves which illustration style to use using the left and right icons at the bottom of the screen, and decides which illustration style to use using the "determination icon". The CPU 201 determines the illustration style of the stamp selected by the user through the user's operation on the operation unit 209 .

The example in FIG. 7 shows an example in which four illustration styles can be selected, but more illustration styles may be used as long as they can be realized as a program operation of the image processing unit 204 or the CPU 201. It is not limited to. Note that the screen in FIG. 7 may be configured to allow selection of a plurality of SNS applications for sale, and may be a screen for selecting a stamp style for each SNS application.

Further, in the example of FIG. 7, an image of the completed stamp illustration is displayed, but in addition, the contents of the characters to be superimposed on the generated stamp illustration may also be displayed. For example, it is preferable to display a plurality of message groups and a list of them so that the user can view them. For example, the first message group is a message group related to "greetings" such as "good morning, good night, and thank you." The second message group is a message group related to "reactions" such as "Wow, Hmm, Yes!, No". The third message group is a message group related to "calls" such as "Attention!, Please!, Are you okay now?". The fourth message group is a message group related to "emotions and states" such as "happy, painful, sleepy, tired."

Returning to FIG. 4, in step S405, the CPU 201 determines the illustration style of the stamp. Thereafter, the CPU 201 repeatedly executes the processing from step S406 onwards until it generates stamp data for a predetermined number of stamps determined as a stamp guideline.

In step S406, the CPU 201 determines whether stamp data for the required number of sheets has been created. If the CPU 201 has created the required number of stamp data (YES in step S406), the process proceeds to step S417, and if the required number of stamp data has not been created (NO in step S406), the process proceeds to step S407.

In step S407, the CPU 201 displays on the display unit 207 the screen of FIG. 8 showing how many stamps remain to be created and a pose guide for the next stamp to be created.

<Example of remaining number of sheets screen>
FIG. 8 is a diagram showing an example of the screen displayed in step S407. The screen in FIG. 8 is a screen for displaying the remaining number of stamps that need to be created in order to sell them and an example of a pose for turning them into stamps. In the example of FIG. 8, text is displayed on the screen to indicate the number of remaining stamps that need to be created and to urge the user to take a photo according to the pose of the next stamp to be created.

On the left side of the screen in Figure 8, the layout of the stamp that will be created in the subsequent steps is displayed as an illustration image of a character making a gesture expressing "OK", and the speech bubble section shows how the subject should be framed. Guiding text is also displayed.

On the right side of the screen in FIG. 8, an example of a stamp image created by applying the illustration style selected in step S405 is displayed in a layout with text information superimposed as an image of the completed stamp.

The screen in FIG. 8 notifies the user of the number of remaining images and the pose and guide in the visualized image in advance before starting the actual photographing of the subject in the next step. This allows the user to smoothly photograph subjects with "detailed expressions, rich facial expressions, and nuances," which are requirements for easy-to-use stamps.

Returning to FIG. 4, in step S407 described above, the CPU 201 displays on the display unit 207 the number of remaining stamps, the layout for the next stamp creation, and the photographing pose of the subject. In step S408, the CPU 201 executes photography with guidance for the user (subject or photographer) in accordance with the user's operation on the operation unit 209 so that photography is performed according to the pose, and the imaging unit 203 performs image data generate. There are two specific guide shooting methods: one using voice guidance as shown in Figures 9(a) and (b), and the other using screen display such as live view as shown in Figures 10(a) and (b). ,explain.

<Example of audio guide shooting>
FIG. 9A is a diagram illustrating a method of photographing using a guide in which the built-in speaker of the imaging device 200 directly outputs sound. A tripod 901 supports the imaging device 200. A subject 902 is a subject photographed by the imaging device 200. The imaging device 200 outputs audio through a built-in speaker to guide the position and shooting pose of the subject 902.

FIG. 9B is a diagram illustrating a method in which the imaging device 200 photographs using a guide that outputs audio through the wireless earphone 903. A tripod 901 supports the imaging device 200. A subject 902 is a subject photographed by the imaging device 200. The wireless earphone 903 is attached to the ear of the subject 902. The imaging device 200 wirelessly transmits audio data to the wireless earphones 903 in order to guide the position and shooting pose of the subject 902 . A subject 902 is photographed by the imaging device 200 while listening to a guide using wireless earphones 903 attached to the subject's 902 ears.

In FIGS. 9A and 9B, the subject 902 and the photographer operating the imaging device 200 are the same person, and the imaging device 200 is in an environment where it is fixedly installed using a tripod 901 or the like. Depending on the user's preference, such as the surrounding environment sound at the time of shooting and the distance between the imaging device 200 and the subject 902, the guide shooting method shown in FIG. 9(a) or FIG. 9(b) may be used.

The imaging device 200 operates in a shooting standby state and constantly detects the position and pose of the subject 902 so that the subject is framed within an angle of view suitable for stamp generation shown in step S407 and takes a predetermined pose. conduct. Then, the imaging apparatus 200 guides the subject 902 by instructing the subject 902 to move up, down, left and right, and forward and backward.

Note that detection of the subject 902, region identification of the subject 902, and pose detection of the subject 902 may be executed in the image processing unit 204 using a known algorithm such as pattern matching, or may be executed as program processing of the CPU 201. You may. Alternatively, it may be realized by a method applying an inference model of deep learning technology.

Furthermore, when the subject 902 falls within a predetermined framing and pose, the imaging device 200 may perform either automatic release shooting or shooting after notifying the user of the timer shooting countdown again.

<Live view display guide shooting example>
FIGS. 10A and 10B are diagrams illustrating an example of live view display guided photography of the imaging device 200. In step S407, the imaging device 200 displays an image on the live view panel or the like of the display unit 207, similar to the audio guide in FIG. 9, so as to have the stamp layout and pose shown in the screen in FIG. guides the photographing of the subject 902.

FIG. 10A is a diagram illustrating an example of live view display guided shooting when the photographer 1001 and the subject 902 are different people. The imaging apparatus 200 displays guide information on the live view panel of the display unit 207 and photographs the subject 902 according to the photographing operation instructions from the photographer 1001. Based on the guide information displayed on the live view panel of the display unit 207, the photographer 1001 verbally instructs the subject 902 to move and express so that the layout and pose are suitable for stamp creation. do. Then, after confirming that the subject 902 is in a predetermined framing and pose, the photographer 1001 performs a release operation in the same manner as a normal photographing operation, and executes photographing with the imaging apparatus 200.

FIG. 10B is a diagram showing an example of live view display guided shooting when the photographer and the subject 902 are the same person. A tripod 901 supports the imaging device 200. The imaging apparatus 200 displays guide information on the live view panel of the display unit 207 and photographs the subject 902 in accordance with the photographing operation instruction from the subject 902 using the remote controller 1002 . A subject 902 takes a self-photograph using the imaging device 200. A display device such as a live view panel included in the imaging device 200 is movable. The display surface of the live view panel is set in the same direction as the lens and the image sensor, and is rotated so that the subject 902 can visually recognize the display contents of the live view panel. The subject 902 visually checks the content displayed on the live view panel, confirms that the subject 902 is in a predetermined framing and pose, and then performs a release operation using the remote controller 1002 held at hand. Photographing by the imaging device 200 is executed.

In addition to the method using the movable live view panel described above, a method may also be used in which an external monitor or the like is connected as an attachment to the imaging device 200 and stamp creation guide information is similarly displayed on the external monitor or the like.

Returning to FIG. 4, in step S409, the CPU 201 uses the image processing unit 204 to create stamp illustration data in the illustration style selected in step S405, based on the image data generated in step S408. For example, the image processing unit 204 performs conversion processing from a live-action image to an illustration, and further performs conversion in accordance with an image format specified by the server device 500 and the SNS application operator prior to pre-check processing in the server device 300. conduct. For example, when generating a stamp to be used on LINE, the image processing unit 204 creates a PNG file with a size of 370 pixels vertically and 320 pixels horizontally, a transparent background, and a margin of about 10 pixels on the top, bottom, left, and right. Form into. Furthermore, when generating a stamp to be used in another SNS application, the image processing unit 204 converts the stamp into image data that complies with the stamp guidelines.

In step S410, the CPU 201 previews the stamp illustration data generated in step S409 on the display unit 207, as shown in FIG. We will make an inquiry to confirm whether or not the request is accepted.

<Stamp preview display example>
FIG. 11 is a diagram showing an example of a preview display of stamp illustration data performed in step S410. The example in FIG. 11 is an example in which image data taken in the subject pose in FIG. 8 in step S407 is converted into stamp illustration data.

In the center of the screen in FIG. 11, an image of the stamp illustration data generated in step S409 is displayed. At the bottom of the screen in FIG. 11, there is a user interface with icons that allows the user to select whether to use the image as it is as a sales stamp image and proceed to the check process of the server device 300, or to retake the image until the illustration is to the user's liking. Display and inquire. The user selects "proceed" or "reshoot" through a user operation on the operation unit 209.

Returning to FIG. 4, in step S411, the CPU 201 determines whether "proceed" or "reshoot" has been selected on the screen of FIG. 11. If “proceed” is selected (YES in step S411), the CPU 201 proceeds to step S412, and if “retake” is selected (NO in step S411), the process returns to step S407 to take the same framing and pose. Execute reshoot.

In step S412, the CPU 201 transmits the created stamp illustration data to the server device 300 prior to the sales review (hereinafter referred to as the main review) conducted by the SNS application service provider. The server device 300 performs a compatibility check process on the received stamp illustration data based on the flowchart of FIG. The compatibility check process is a process of checking whether the stamp illustration data complies with the guidelines set by the SNS application operator.

In step S413, the CPU 201 determines whether or not the preliminary check result corresponding to the transmission in step S412 has been received from the server device 300. If the CPU 201 has not received the preliminary check result (NO in step S413), the process returns to step S413, and if the preliminary check result has been received (YES in step S413), the process proceeds to step S414.

Note that in the case of NO in step S414, the CPU 201 may make an inquiry to receive the preliminary check result from the server device 300 after a predetermined period of time has elapsed. In addition, the CPU 201 sets a timeout value until reception, the number of inquiry retries, etc. in advance, and when a case occurs in which the preliminary check result cannot be received from the server device 300 for some reason, the CPU 201 forcibly terminates the flowchart of FIG. 4 as an exception. You can also handle errors.

In step S414, the CPU 201 determines whether the pre-check result of the stamp illustration data transmitted in step S412 is OK or NG based on the pre-check result received in step S413. If the pre-check result is OK (YES in step S414), the CPU 201 determines that the stamp illustration data has a suitability quality that allows it to proceed to the main examination as is, and proceeds to step S415. Further, if the pre-check result is NG (NO in step S414), the CPU 201 proceeds to step S416 because there is a high possibility that the stamp illustration data cannot be sold as is.

In step S415, the CPU 201 subtracts the remaining number of stamps required for sale, and if the remaining number is 1 or more, switches to template data for the next illustration pose, and returns to step S406.

In step S416, the CPU 201 displays on the display unit 207 that the pre-check result is NG, the reason thereof, and advice, as shown in FIG. 12, for example, and returns to step S407.

<Check result display example>
FIG. 12 is a diagram showing an example of the preliminary check result displayed in step S416. Figure 12 shows a case where the characters of the stamp illustration data generated from the subject in the photographed image do not all fit within the frame, and there is a large overlap between the text and the characters, resulting in poor visibility. .

The CPU 201 displays, on the display unit 207, the stamp illustration data as well as the reason why the pre-check result is NG: "No subject is included." Then, the CPU 201 displays a "reshoot" icon at the bottom of the screen in FIG. 12 for the user to confirm the reason for the NG, move to step S407, and confirm by inquiring that the process proceeds to the reshoot step. The CPU 201 returns to step S407 at the timing when the user selects the above-mentioned "Reshoot" icon by key selection or touch operation, and performs guide shooting and stamp generation again.

Returning to FIG. 4, step S417 is a process performed when the preliminary check result is OK and the generation of stamp illustration data for the number of stamps required for sale is completed. In step S417, the CPU 201 determines whether or not to request the server device 500 or the SNS application company for sales review of the stamp illustration data in response to the user's operation on the operation unit 209. Step S417 is a final confirmation step, in which the CPU 201 confirms with the user whether or not he/she really wants to request a sales review for the stamp illustration data image set created above. If the CPU 201 requests a sales review (YES in step S417), the process proceeds to step S418; if the CPU 201 does not request a sales review (NO in step S417), the process proceeds to step S419.

In step S418, the CPU 201 transmits to the server device 300 a proxy request for sales review of stamp illustration data, and ends the process of the flowchart of FIG. 4. The process performed when the server device 300 receives the sales review request proxy request will be described later using FIG. 5(b).

In step S419, the CPU 201 executes cancel processing to discard the stamp illustration data created above, and ends the processing of the flowchart in FIG. 4.

<Operation flow of server device 300>
FIGS. 5(a) to 5(c) are flowcharts showing the processing method of the server device 300. The flowcharts in FIGS. 5A to 5C are mainly executed by the CPU 301 of the server device 300 executing a program. FIG. 5A is a flowchart showing a check process performed when the server device 300 receives stamp illustration data from the imaging device 200. FIG. 5B is a flowchart illustrating a proxy sales review request process performed when the server device 300 receives a proxy request for sales review of stamp illustration data from the imaging device 200. FIG. 5(c) is a flowchart showing the process executed by the server device 300 when receiving the sales examination result from the server device 500.

<Operation flow when receiving stamp illustration data>
FIG. 5A is a flowchart showing a processing method when the server device 300 receives stamp illustration data.

In step S501, the CPU 301 determines whether the stamp illustration data transmitted in step S412 of FIG. 4 has been received from the imaging device 200. If the CPU 301 receives the illustration data for stamps (YES in step S501), the process proceeds to step S502, and if the illustration data for stamps has not been received (NO in step S501), the process returns to step S501.

In step S502, the CPU 301 determines whether the image format of the received stamp illustration data is an image format that complies with the guidelines used in the SNS application. As a specific determination method, the CPU 301 uses the image processing unit 303 to determine whether the received stamp illustration data is in a predetermined file format and can be decoded. If the image format is compliant (YES in step S502), the CPU 301 proceeds to step S503; if the image format is not compliant (NO in step S502), the CPU 301 proceeds to step Proceed to S510.

In step S503, the CPU 301 uses the image processing unit 303 to determine whether the image size of the received stamp illustration data is an image size that complies with the guidelines used in the SNS application. If the image size is compliant (YES in step S503), the CPU 301 proceeds to step S504; if the image size is not compliant (NO in step S503), the CPU 301 proceeds to step Proceed to S510.

In step S504, the CPU 301 determines whether the stamp illustration data includes a character that is an illustration of the subject. This determination is realized by the CPU 301 using the image processing unit 303 to separate the text portion from the background portion other than the text portion, and perform image analysis of the portion other than the text portion. If the stamp illustration data does not contain characters and consists only of characters (NO in step S504), the CPU 301 proceeds to step S510 because the guideline is not compliant. Further, if a character is included (YES in step S504), the CPU 301 proceeds to step S505.

In step S505, the CPU 301 determines whether the stamp illustration data is too monotonous (too light-colored) as a pattern. For example, the CPU 301 uses the image processing unit 303 to calculate a histogram or dynamic range of the number of colors and pixel values present in the stamp illustration data. Then, the CPU 301 can determine that the image is too monotonous if the number of colors used is less than a predetermined number or if the dynamic range is localized in a narrow range on the white level side.

If the CPU 301 is too monotonous (NO in step S505), there is a high possibility that the product cannot be sold, so the process proceeds to step S510, and if it is not too monotonous (YES in step S505), the process proceeds to step S506.

In step S506, the CPU 301 determines whether the stamp illustration data has good visibility. Although the number of colors, color scheme, and gradation range referred to in step S505 are also indicators related to visibility, in step S506, the CPU 301 determines whether visibility is good from a broader perspective.

For example, if the illustration data for a stamp contains multiple illustration characters, contains complex and detailed depictions, or has too much text, visibility may be impaired. As such, there is a high possibility that the product will be rejected in the sales examination.

As a specific example of the visibility determination method, the CPU 301 uses the image processing unit 303 to detect outlines and edges of the stamp illustration data, and if a predetermined number or more of edges or outlines are detected, It can be determined that visibility is not good. The visibility determination method is not limited to any other algorithm as long as it obtains the same effect.

If the visibility of the stamp illustration data is not good (NO in step S506), the CPU 301 proceeds to step S510 because it does not comply with the guidelines. Further, if the visibility of the stamp illustration data is good (YES in step S506), the CPU 301 proceeds to step S507.

In step S507, the CPU 301 determines whether the content of the stamp illustration data does not violate morals and public order and morals. Specifically, content that suggests underage drinking and smoking, sexual content, violent content, content that incites nationalism, content that slanders specific individuals or groups, etc. are against morals and public order. The content will be strictly examined according to the guidelines.

It is generally difficult to realize the above-mentioned content determination as program processing in the CPU 301 or wiring logic in the image processing unit 303 using a known rule-based algorithm. Therefore, in this embodiment, the CPU 301 stores in advance a huge amount of image data on the network and the examination results in a database, and uses a network model machine-learned using training data and teacher data in the database to make judgments. I do. This network model is processed at high speed by the machine learning and inference processing unit 308 described above under a dedicated software or hardware configuration.

If the CPU 301 does not violate morals or public order and morals (YES in step S507), the process proceeds to step S508, and if it violates morals or public order and morals (NO in step S507), the process proceeds to step S510 because there is a risk of violating the guidelines. move on.

In step S508, the CPU 301 determines whether the contents of the stamp illustration data violate copyrights and portrait rights. The illustration data for stamps is generated using a predetermined illustration style, but if there are people or characters who happen to be included in the image, or if the subject is intentionally disguised, there may be copyright issues. There are cases where illustrations violate portrait rights.

As a method for implementing the determination, the CPU 301 can use a network model trained to determine whether the content violates copyright or portrait rights, similar to step S507. Further, the CPU 301 can crawl the Internet via the communication unit 304 and determine whether there are any characters, celebrities, or public figures similar to the stamp illustration data.

If the CPU 301 does not violate copyright or portrait rights (YES in step S508), the process proceeds to step S509, and if the copyright or portrait rights violate (NO in step S508), the process proceeds to step S510.

In step S509, since all the check determinations in steps S502 to S508 have passed without any problems, the CPU 301 transmits to the imaging apparatus 200 that the pre-check results are OK, and ends the process of the flowchart in FIG. 5(a).

In step S510, since a NO determination was made in any of steps S502 to S508, the CPU 301 transmits the fact that the preliminary check result is NG together with the reason to the imaging apparatus 200, and sends the result of the pre-check to the imaging apparatus 200. Finish the process.

In step S413 in FIG. 4, the imaging device 200 receives the content transmitted in step S509 or S510.

As described above, the CPU 301 determines whether the stamp illustration data conforms to the items defined as guidelines on the messenger application. For example, the guidelines include image size, data format, image resolution, color mode, data size, background transparency, margins, content with sexual expressions, violent expressions, content for which the ownership of rights is unclear, and public order and morals. All or any of the following:

<Operation flow when receiving a sales review request agency request>
FIG. 5B is a flowchart showing a processing method of the server device 300 when receiving a proxy request for sales review of stamp illustration data.

In step S511, the CPU 301 determines whether or not it has received the proxy request for sales review of stamp illustration data transmitted from the imaging device 200 in step S418 of FIG. In step S418, the imaging device 200 sends to the server device 300 stamp information such as illustration data for stamps, account information, remittance information, copyright, and sales area necessary for the sales review request along with a request for sales review request. Send. If the CPU 301 receives a request for sales review request (YES in step S511), the process proceeds to step S512, and if it does not receive a request for sales review request (NO in step S511), the process returns to step S511.

In step S512, the CPU 301 sends the stamp information such as the stamp illustration data, account information, transfer information, copyright, and sales area sent from the imaging device 200 to the server device 500 in order to request a sales review. do. After that, the CPU 301 ends the process of the flowchart in FIG. 5(b). Server device 500 receives them, performs a sales review, and transmits the sales review results to server device 300.

<Operation flow when receiving sales screening results>
FIG. 5C is a flowchart showing a processing method when the server device 300 receives the sales examination results.

In step S513, CPU 301 determines whether the sales review result of the sales review request transmitted in step S512 has been received from server device 500. If the CPU 301 has received the sales examination result (YES in step S513), the process proceeds to step S514, and if the sales examination result has not been received (NO in step S513), the process returns to step S513.

In step S514, the CPU 301 saves the stamp illustration data for which the sales examination was requested together with the sales examination results in the database 400 connected to the server device 300.

In step S515, the CPU 301 uses the sales examination results stored in the database 400 in step S514 to relearn the network model for compatibility check in the machine learning and inference processing unit 308.

<Save sales review results and model relearning>
FIG. 13 is a diagram showing the relearning of the network model performed in step S515. Stamp illustration data (stamp data) 1301 and sales examination results (examination results) 1302 are stored in the database 400. Stamp illustration data 1301 is stamp illustration data for which a sales review request has been made. The sales examination result 1302 is OK or NG. The CPU 301 inputs the stamp illustration data 1301 to the network model 1304 as training data 1303. The network model 1304 is a network model used by the machine learning and inference processing unit 308 to determine stamp guideline suitability. The CPU 301 uses the sales examination result 1302 as the teacher data 1306 to learn the network model 1304 so that the training result 1305 of the network model 1304 matches the teacher data 1306 which is the correct label.

As described above, the CPU 301 executes step S515, and the training data 1303 in the database 400 is accumulated as various users repeatedly request sales review of stamp illustration data. As a result, the effect of improving the determination accuracy of the network model 1304 for compatibility check can be obtained.

Returning to FIG. 5(c), in step S516, the CPU 301 transmits the sales examination results described above to the mobile terminal 600 owned by the user who created the stamp illustration data, and Finish the process.

As described above, the image processing system 100 includes the imaging device 200, the server device 300, the database 400, the server device 500, and the mobile terminal 600. In step S409 of FIG. 4, the CPU 201 of the imaging device 200 functions as a generation unit and generates stamp data that can be used in the messenger application by processing the captured image data. The stamp data is the above-mentioned stamp illustration data.

In steps S502 to S508 in FIG. 5A, the CPU 301 of the server device 300 functions as a determination unit and determines whether the stamp data generated in step S409 conforms to the guidelines. For example, the CPU 301 checks the stamp data for image format, image size, the presence of a character corresponding to the subject, the fact that it is not too monotonous, the visibility is good, the stamp data does not violate morals and public order, or the copyright and determine whether the requirements of not infringing portrait rights are met.

In steps S507 and S508, the CPU 301 can make the determination using the network model 1304 in FIG. In step S515 in FIG. 5C, the CPU 301 functions as a learning unit and performs learning of the network model 1304 based on the stamp data 1301 and the review result 1302 of the sales review request.

In step S416 of FIG. 4, the CPU 201 functions as a display control unit and controls the display of the determination results of steps S502 to S508 as shown in FIG. Further, as shown in FIG. 12, when it is determined in steps S502 to S508 that the stamp data does not conform to the guidelines, the CPU 201 controls to display the reason why the stamp data does not conform. Further, as shown in FIG. 12, if it is determined in steps S502 to S508 that the stamp data does not conform to the guidelines, the CPU 201 controls to display an inquiry as to whether to retake the image. .

In step S403, the CPU 201 performs control to display the contents of the contract and to display an inquiry as to whether or not the user agrees to the contents of the contract, as shown in FIGS. 6(a) to 6(c). In step S404, if there is an instruction to agree to the contents of the contract, the CPU 201 proceeds to step S405.

In step S405, the CPU 201 functions as a selection unit, and selects one of the plurality of stamp data styles according to the user's operation, as shown in FIG.

In step S407, the CPU 201 controls to display an example pose for guiding the subject's pose, as shown in FIG. 8, before photographing in step S408. Further, as shown in FIG. 8, the CPU 201 controls to display an example of stamp data that is generated when a photograph is taken using an example of a guiding pose.

In step S408, the CPU 201 functions as a photographing unit and generates image data by photographing the subject. The CPU 201 detects the position of the subject within the angle of view to be photographed, guides the subject to the position, and when the position of the subject reaches the position to be guided, prompts the user to press the release button or performs an automatic release operation. I do. For example, as shown in FIG. 9(a) or (b), the CPU 201 uses audio to guide the position of the subject. Further, as shown in FIG. 10A or 10B, the CPU 201 guides the position of the subject by displaying graphics superimposed on the live view.

In step S409, the CPU 201 generates stamp data by performing image processing on the image data photographed in step S408 according to the stamp data style selected in step S405.

In step S410, the CPU 201 controls to display the stamp data generated in step S409, as shown in FIG. 11, and to display an inquiry as to whether or not to proceed with processing the stamp data. In step S411, if the CPU 201 receives an instruction to proceed with the processing of the stamp data, the process proceeds to step S412.

The loop processing of steps S407 to S415 is performed multiple times. In step S406, if the server device 300 determines that the predetermined number of stamp data conform to the guidelines, the CPU 201 proceeds to step S417. In step S418, the CPU 201 transmits to the server device 300 a proxy request for a sales review request for a predetermined number of stamp data to the messenger application management website. In step S512 of FIG. 5B, upon receiving the proxy request, the CPU 301 functions as a requesting unit and requests the messenger application operating website for a sales review of a predetermined number of stamp data.

In step 407, as shown in FIG. 8, the CPU 201 controls to display the number of remaining stamp data required to reach a determination that a predetermined number of stamp data conform to the guidelines.

Next, a processing method of the imaging device 200 in FIG. 4 will be explained. In step S412, the CPU 201 functions as a transmitter and transmits stamp data that is generated based on the photographed image data and can be used in the messenger application to the server device 300. In step S413, the CPU 201 receives from the server device 300 the determination result as to whether the stamp data that is generated based on the photographed image data and that can be used in the messenger application conforms to the guidelines. In step S416, the CPU 201 controls to display the determination result.

Next, a processing method of the server device 300 in FIG. 5(a) will be explained. In step S501, the CPU 301 functions as a receiving unit and receives stamp data that is generated based on photographed image data and that can be used in the messenger application. In steps S502 to S508, the CPU 301 determines whether the stamp data that can be used in the messenger application and that is generated based on the photographed image data conforms to the guidelines.

As described above, according to the present embodiment, it is not necessary for an individual user to create illustration stamp data from scratch. Users can easily create original stamp data based on photographed image data, register it on a sales site, and sell it. The image processing system 100 can reduce the user's effort to generate stamp data that conforms to guidelines.

(Second embodiment)
In the first embodiment, a method has been described in which a live image photographed by the imaging device 200 in the photographing mode according to a guide is converted into stamp illustration data. The second embodiment shows a method in which the imaging device 200, in playback mode, transmits already photographed live image data to the server device 300, and the server device 300 converts the received image data into stamp illustration data. .

The second embodiment will be described below with reference to FIGS. 14 and 15. Note that in the description of the second embodiment, descriptions of parts common to the above-described first embodiment will be omitted as appropriate.

In addition to decoding and analyzing the stamp illustration data described in the first embodiment, the image processing unit 303 in FIG. Equipped with a function to convert to illustration data.

<Operation flow of imaging device 200>
FIG. 14 is a flowchart showing a processing method of the imaging device 200 according to the second embodiment.

In step S1401, the CPU 201 uses a mode setting from the operation unit 209 having an interface such as a button, a cross key, or a touch panel as a trigger, and determines whether the operation mode of the imaging device 200 is the playback mode. If the operation mode is not the reproduction mode (NO in step S1401), the CPU 201 ends the process of the flowchart of FIG. 14, and if the operation mode is the reproduction mode (YES in step S1401), the process proceeds to step S1402.

In step S1402, the CPU 201 performs the same process as step S402 in FIG. If the stamp creation mode is not selected (NO in step S1402), the CPU 201 returns to step S1401, and if the stamp creation mode is selected (YES in step S1402), the process advances to step S1403.

In step S1403, the CPU 201 performs the same process as step S403 in FIG.

In step S1404, the CPU 201 performs the same process as step S404 in FIG. If the user does not agree with the service content (NO in step S1404), the CPU 201 returns to step S1401, and if the user agrees with the service content (YES in step S1404), the process proceeds to step S1405.

In step S1405, the CPU 201 downloads an illustration style from the server device 300 and updates the repertoire of selectable styles in order for the server device 300 to create stamp illustration data. After that, the CPU 201 performs the same process as step S405 in FIG.

In step S1406, the CPU 201 performs the same process as step S406 in FIG. If the CPU 201 has created the required number of stamp data (YES in step S1406), the process proceeds to step S1415, and if the required number of stamp data has not been created (NO in step S1406), the process proceeds to step S1407.

In step S1407, the CPU 201 performs the same process as step S407 in FIG.

In step S1408, in response to the user's operation, the CPU 201 searches the photographed image data stored by the user in the recording/playback unit 206, and captures the layout and subject pose suitable for stamp creation as instructed in step S1407. Select live image data. The photographed image data is original image data.

In step S1409, the CPU 201 transmits the photographed image data selected in step S1408 and the illustration style selected in step S1405 to the server device 300 in order to convert them into stamp illustration data.

In step S1410, the CPU 201 determines whether or not the stamp illustration data and the pre-check result of the stamp illustration data corresponding to the photographed image data transmitted in step S1409 have been received from the server device 300. If the CPU 201 receives the stamp illustration data and the preliminary check result (YES in step S1410), the process proceeds to step S1411. Further, if the CPU 201 has not received the stamp illustration data and the preliminary check result (NO in step S1410), the process returns to step S1410. If NO in step S1410, the CPU 201 can inquire about receiving the stamp illustration data and preliminary check results from the server device 300 after a predetermined period of time has passed.

In step S1411, the CPU 201 displays a preview of the stamp illustration data created by the server device 300 and the pre-check results received from the server device 300 on the display unit 207. For example, the CPU 201 displays on the display unit 207 a screen that is a combination of the stamp illustration data preview shown in FIG. 11 and the pre-check result display shown in FIG.

In step S1412, the CPU 201 performs the same process as step S414 in FIG. If the preliminary check result is OK (YES in step S1412), the CPU 201 proceeds to step S1413, and if the preliminary check result is NG (NO in step S1412), the process proceeds to step S1414.

In step S1413, the CPU 201 performs the same process as step S415 in FIG. 4, and returns to step S1406.

In step S1414, since the pre-check result is NG, the CPU 201 automatically searches the photographed image data stored in the recording/playback unit 206, and displays image candidates similar to the image selected in step S1408 on the display unit 207. Display and return to step S1407.

As an example of a method for determining similar images, the CPU 201 can determine whether the images are similar based on a known image similarity determination algorithm such as pattern matching or meta information such as shooting date and time and shooting location. The method for determining similar images is not limited.

In step S1415, the CPU 201 performs the same process as step S417 in FIG. If the CPU 201 makes a request for sales review (YES in step S1415), the process proceeds to step S1416, and if the request for sales review is not made (NO in step S1415), the process proceeds to step S1417.

In step S1416, the CPU 201 performs the same process as step S418 in FIG. 4, and ends the process in the flowchart in FIG.

In step S1417, the CPU 201 performs the same process as step S419 in FIG. 4, and ends the process in the flowchart in FIG.

<Operation flow of server device 300>
FIG. 15 is a flowchart showing a processing method of the server device 300 according to the second embodiment.

In step S1501, the CPU 301 determines whether the real image data transmitted in step S1409 in FIG. 14 and the illustration style selected in step S1405 in FIG. 14 have been received from the imaging apparatus 200. If the CPU 301 receives the real image data and illustration style (YES in step S1501), the CPU 301 advances to step S1502 to convert the real image data into stamp illustration data. Further, if the CPU 301 has not received the photographed image data and the illustration style (NO in step S1501), the process returns to step S1501.

In step S1502, the CPU 301 performs the same process as step S409 in FIG. Based on the photographed image data received in step S1501, the CPU 301 uses the image processing unit 204 to create stamp illustration data in the illustration style received in step S1501.

In step S1503, the CPU 301 performs the same process as step S502 in FIG. 5A on the stamp illustration data created in step S1502. If the image format is compliant (YES in step S1503), the CPU 301 proceeds to step S1504, and if the image format is not compliant (NO in step S1503), the process proceeds to step S1511.

In step S1504, the CPU 301 performs the same process as step S503 in FIG. 5A on the stamp illustration data created in step S1502. If the image size is compliant (YES in step S1504), the CPU 301 proceeds to step S1505, and if the image size is not compliant (NO in step S1504), the process proceeds to step S1511.

In step S1505, the CPU 301 performs the same process as step S504 in FIG. 5A on the stamp illustration data created in step S1502. If a character is not included (NO in step S1505), the CPU 301 proceeds to step S1511, and if a character is included (YES in step S1505), the process proceeds to step S1506.

In step S1506, the CPU 301 performs the same process as step S505 in FIG. 5A on the stamp illustration data created in step S1502. If the CPU 301 is too monotonous (NO in step S1506), the process proceeds to step S1511, and if it is not too monotonous (YES in step S1506), the process proceeds to step S1507.

In step S1507, the CPU 301 performs the same process as step S506 in FIG. 5A on the stamp illustration data created in step S1502. If the visibility of the stamp illustration data is poor (NO in step S1507), the CPU 301 proceeds to step S1511, and if the visibility of the stamp illustration data is good (YES in step S1507), the process proceeds to step S1508.

In step S1508, the CPU 301 performs the same process as step S507 in FIG. 5A on the stamp illustration data created in step S1502. If the CPU 301 does not violate morals or public order and morals (YES in step S1508), the process proceeds to step S1509, and if it violates morals or public order and morals (NO in step S1508), the process proceeds to step S1511.

In step S1509, the CPU 301 performs the same process as step S508 in FIG. 5A on the stamp illustration data created in step S1502. If the CPU 301 does not violate copyright or portrait rights (YES in step S1509), the process proceeds to step S1510, and if the copyright or portrait rights violate (NO in step S1509), the process proceeds to step S1511.

In step S1510, similarly to step S509 in FIG. 5A, the CPU 301 transmits to the imaging device 200 that the stamp illustration data and the preliminary check result are OK, and ends the process of the flowchart in FIG. 15.

In step S1511, similarly to step S510 in FIG. 5A, the CPU 301 transmits the stamp illustration data and the fact that the pre-check result is NG, together with the reason, to the imaging device 200, and then Finish the process.

As described above, in step S1502 of FIG. 15, the CPU 301 of the server device 300 functions as a generation unit and generates stamp data that can be used in the messenger application by processing the captured image data. The stamp data is the above stamp illustration data. In steps S1503 to S1509 in FIG. 15, the CPU 301 of the server device 300 functions as a determination unit and determines whether the stamp data generated in step S1502 conforms to the guidelines.

In step S1408 in FIG. 14, the CPU 201 functions as a selection unit and selects one of the plurality of photographed image data according to the user's operation. In step S1502 of FIG. 15, the CPU 301 generates stamp data by performing image processing on the photographed image data selected in step S1408.

In step S1412 of FIG. 14, if the server device 300 determines that the stamp data does not conform to the guidelines, the CPU 201 proceeds to step S1414. In step S1414, the CPU 201 searches for image data similar to the image data from which the stamp data was generated from among the plurality of photographed image data, and performs control to display the similar image data. Thereafter, in step S1408, the CPU 201 selects one of the similar image data according to the user's operation.

Next, a processing method of the imaging device 200 in FIG. 14 will be explained. In step S1409, the CPU 201 transmits the photographed image data to the server device 300. In step S1410, the CPU 201 receives from the server device 300 the determination result as to whether the stamp data that is generated based on the photographed image data and that can be used in the messenger application conforms to the guidelines. In step S1411, the CPU 201 controls to display the determination result.

Next, a processing method of the server device 300 in FIG. 15 will be explained. In step S1501, the CPU 301 receives photographed image data. In step S1502, the CPU 301 generates stamp data that can be used in the messenger application based on the photographed image data. In steps S1503 to S1509, the CPU 301 determines whether the generated stamp data conforms to the guidelines.

As described above, according to this embodiment, the server device 300 can generate stamp data based on photographed image data.

(Third embodiment)
In the first and second embodiments described above, a method has been described in which the server device 300 performs a preliminary check on stamp illustration data generated from image data taken by a user. In the third embodiment, the server device 500, not the server device 300, executes the pre-check process for stamp illustration data.

In the first and second embodiments, the server device 300 and the server device 500 are described as different devices. In this embodiment, the server device 300 and the server device 500 are configured as one server device. The server device 500 has a configuration similar to the hardware configuration of FIG. 2, and has both the functions of the server device 300 and the functions of the server device 500 of the first or second embodiment. As a result, network communication data traffic between the server device 300 and the server device 500 can be reduced, and as a result, for users who want to sell stamps, the turnaround time from taking photos to receiving sales review results can be reduced. Obtainable.

(Fourth embodiment)
In the first to third embodiments described above, the user uses the imaging device 200 to perform a shooting operation in order to create illustration data for a stamp, and the final sales review result is transferred to a mobile terminal owned by the user. Receive at 600. A specific product form of the mobile terminal 600 is a smartphone, a tablet, or the like. In recent years, the mobile terminal 600 has come to be equipped with an image sensor having performance equivalent to that of a digital camera or a video camera.

Therefore, in the fourth embodiment, the mobile terminal 600 may be integrated with the imaging device 200 as having the same or similar functional block configuration as the imaging device 200. The mobile terminal 600 can perform the same processing as the imaging device 200 described in the above-described first or second embodiment. As a result, the mobile terminal 600 has the same processing functions as the imaging device 200 described above, and can handle a series of guide shooting, automatic creation of stamp illustration data, and pre-check of stamp illustration data to reception of sales examination results. Processing can be completed within one device.

(Other embodiments)
The present disclosure provides a system or device with a program that implements one or more functions of the above-described embodiments via a network or a storage medium, and one or more processors in a computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Note that the above-described embodiments are merely specific examples for carrying out the present disclosure, and the technical scope of the present disclosure should not be construed as limited by these embodiments. That is, the present disclosure can be implemented in various forms without departing from its technical idea or main features.

The disclosure of this embodiment includes the following configuration, method, and program.
(Configuration 1)
a generation means for generating stamp data that can be used in a messenger application by processing the photographed image data;
and determining means for determining whether the stamp data generated by the generating means conforms to a guideline.
(Configuration 2)
The image processing system includes an imaging device and a server device,
The imaging device includes the generating means,
The image processing system according to configuration 1, wherein the server device includes the determination means.
(Configuration 3)
The image processing system includes a server device,
The image processing system according to configuration 1, wherein the server device includes the generating means and the determining means.
(Configuration 4)
4. The image processing system according to any one of configurations 1 to 3, further comprising display control means for controlling to display the determination result of the determination means.
(Configuration 5)
In configuration 4, the display control means controls to display a reason for non-conformity when the determination means determines that the stamp data does not conform to the guidelines. The image processing system described.
(Configuration 6)
The determining means determines that the stamp data generated by the generating means has an image format, an image size, that a character corresponding to the subject exists, that it is not too monotonous, that it has good visibility, and that it is contrary to morals and public order and morals. 6. The image processing system according to any one of configurations 1 to 5, characterized in that the image processing system determines whether or not the requirements of not violating copyright and portrait rights are satisfied.
(Configuration 7)
further comprising a photographing means for generating image data by photographing the subject;
7. The image processing system according to any one of configurations 1 to 6, wherein the generating means generates the stamp data by performing image processing on image data photographed by the photographing means.
(Configuration 8)
further comprising display control means for controlling to display the determination result of the determination means,
The display control means is characterized in that, when the determination means determines that the stamp data does not conform to the guidelines, the display control means controls to display an inquiry as to whether or not to take the image again. The image processing system according to configuration 7.
(Configuration 9)
8. The image processing system according to configuration 7, further comprising display control means for controlling to display an example of a pose for guiding the pose of the subject before the photographing means takes a picture.
(Configuration 10)
10. The image processing system according to configuration 9, wherein the display control means controls to display an example of stamp data generated when photographing with the example of the guiding pose.
(Configuration 11)
The photographing means detects the position of the subject within the angle of view to be photographed, guides the position of the subject, and prompts the user to press the release when the position of the subject reaches the position to be guided. The image processing system according to any one of configurations 7 to 10, characterized in that the image processing system performs an automatic release operation.
(Configuration 12)
12. The image processing system according to configuration 11, wherein the photographing means uses audio to guide the position of the subject.
(Configuration 13)
12. The image processing system according to configuration 11, wherein the photographing means guides the position of the subject by displaying graphics superimposed on the live view.
(Configuration 14)
further comprising a first selection means for selecting one of the plurality of stamp data styles according to a user operation;
According to configurations 1 to 13, the generation means generates the stamp data by performing image processing on photographed image data according to the stamp data style selected by the first selection means. The image processing system according to any one of the items.
(Configuration 15)
The generation by the generation means and the determination by the determination means are performed multiple times,
If the determination means determines that the predetermined number of stamp data conforms to the guidelines, the system further includes a requesting means for requesting a messenger application operating website for a sales review of the predetermined number of stamp data. 15. The image processing system according to any one of configurations 1 to 14, characterized in that:
(Configuration 16)
A configuration 15 further comprising display control means for controlling to display the number of remaining stamp data required until reaching a determination that the predetermined number of stamp data conforms to the guideline. The image processing system described.
(Configuration 17)
further comprising display control means for controlling to display the stamp data generated by the generation means and displaying an inquiry as to whether processing of the stamp data may be proceeded;
17. The image processing system according to any one of configurations 1 to 16, wherein the determination means performs the determination when there is an instruction to proceed with processing the stamp data.
(Configuration 18)
further comprising display control means for displaying the contents of the contract and displaying an inquiry as to whether or not to agree to the contents of the contract;
The image processing system according to any one of configurations 1 to 17, wherein when there is an instruction to agree to the contents of the contract, the generation means generates and the determination means makes a determination. .
(Configuration 19)
The determination means makes the determination using a network model,
16. The image processing system according to configuration 15, further comprising a learning means for learning the network model based on the stamp data and the examination result of the sales examination request.
(Configuration 20)
further comprising a second selection means for selecting one of the plurality of photographed image data according to a user operation,
The image according to any one of configurations 1 to 6, wherein the generation means generates the stamp data by performing image processing on the photographed image data selected by the second selection means. processing system.
(Configuration 21)
If the determination means determines that the stamp data does not conform to the guidelines, image data similar to the image data from which the stamp data was generated is selected from among the plurality of photographed image data. controlling to search and display the similar image data;
21. The image processing system according to configuration 20, wherein the second selection means selects one of the similar image data in response to a user operation.
(Configuration 22)
Receiving means for receiving photographed image data or stamp data that is generated based on the photographed image data and can be used in a messenger application;
A server device comprising: a determination unit that determines whether stamp data that is generated based on the photographed image data and that can be used in a messenger application conforms to a guideline.
(Configuration 23)
a transmitting means for transmitting photographed image data or stamp data that is generated based on the photographed image data and can be used in a messenger application to a server device;
receiving means for receiving from the server device a determination result as to whether stamp data available in the messenger application generated based on the photographed image data conforms to guidelines;
An imaging device comprising: display control means for controlling the display of the determination result.
(Method 1)
a generation step of generating stamp data that can be used in the messenger app by image processing the photographed image data;
A processing method for an image processing system, comprising: a determining step of determining whether the stamp data generated in the generating step conforms to a guideline.
(Method 2)
a receiving step of receiving photographed image data or stamp data that is generated based on the photographed image data and can be used in a messenger application;
A processing method for a server device, comprising: a determining step of determining whether stamp data that is generated based on the photographed image data and that can be used in a messenger application conforms to a guideline.
(Method 3)
a sending step of transmitting the photographed image data or the stamp data that is generated based on the photographed image data and can be used in the messenger application to the server device;
a receiving step of receiving from the server device a determination result as to whether stamp data available in the messenger application generated based on the photographed image data conforms to guidelines;
A processing method for an imaging device, comprising: a display control step for controlling to display the determination result.
(Program 1)
A program for causing a computer to function as each means of the image processing system, server device, or imaging device described in any one of Configurations 1 to 23.

Reference Signs List 100 image processing system, 200 imaging device, 300 server device, 400 database, 500 server device, 600 mobile terminal, 201 CPU, 202 primary storage section, 203 imaging section, 204 image processing section, 205 communication section, 206 recording and reproducing section, 207 display section, 208 audio processing section, 209 operation section, 210 CPU bus, 211 work memory, 212 memory bus, 301 CPU, 302 primary storage section, 303 image processing section, 304 communication section, 305 recording/playback section, 306 display section , 307 operation unit, 308 machine learning and inference processing unit, 309 CPU bus, 310 work memory, 311 memory bus

Claims (27)

a generation means for generating stamp data that can be used in a messenger application by processing the photographed image data;
and determining means for determining whether the stamp data generated by the generating means conforms to a guideline. The image processing system includes an imaging device and a server device,
The imaging device includes the generating means,
The image processing system according to claim 1, wherein the server device includes the determining means. The image processing system includes a server device,
The image processing system according to claim 1, wherein the server device includes the generating means and the determining means. The image processing system according to claim 1, further comprising display control means for controlling to display the determination result of the determination means. 4. The display control means controls to display the reason for non-conformity when the determination means determines that the stamp data does not conform to the guideline. The image processing system described in . The determining means determines that the stamp data generated by the generating means has an image format, an image size, that a character corresponding to the subject exists, that it is not too monotonous, that it has good visibility, and that it is contrary to morals and public order and morals. 2. The image processing system according to claim 1, wherein the image processing system determines whether or not the image processing system satisfies the requirements of not violating copyrights and portrait rights. further comprising a photographing means for generating image data by photographing the subject;
2. The image processing system according to claim 1, wherein the generating means generates the stamp data by performing image processing on image data photographed by the photographing means. further comprising display control means for controlling to display the determination result of the determination means,
The display control means is characterized in that, when the determination means determines that the stamp data does not conform to the guidelines, the display control means controls to display an inquiry as to whether or not to take the image again. The image processing system according to claim 7. 8. The image processing system according to claim 7, further comprising display control means for controlling to display pose examples for guiding the pose of the subject before the photographing means takes a picture. 10. The image processing system according to claim 9, wherein the display control means controls to display an example of stamp data generated when photographing with the example of the guiding pose. The photographing means detects the position of the subject within the angle of view to be photographed, guides the position of the subject, and prompts the user to press the release when the position of the subject reaches the position to be guided. The image processing system according to claim 7, wherein the image processing system performs an automatic release operation. 12. The image processing system according to claim 11, wherein the photographing means uses audio to guide the position of the subject. 12. The image processing system according to claim 11, wherein the photographing means guides the position of the subject by displaying graphics superimposed on the live view. further comprising a first selection means for selecting one from a plurality of stamp data styles in accordance with a user operation;
2. The generating means generates the stamp data by performing image processing on photographed image data according to the stamp data style selected by the first selecting means. image processing system. The generation by the generation means and the determination by the determination means are performed multiple times,
If the determination means determines that the predetermined number of stamp data conforms to the guidelines, the system further includes a requesting means for requesting a messenger application operating website for a sales review of the predetermined number of stamp data. The image processing system according to claim 1, characterized in that: Claim 15, further comprising display control means for controlling to display the number of remaining stamp data necessary to reach a determination that the predetermined number of stamp data conforms to a guideline. The image processing system described in . further comprising display control means for displaying the stamp data generated by the generation means and displaying an inquiry as to whether or not processing of the stamp data may be proceeded;
2. The image processing system according to claim 1, wherein the determining means performs the determination when there is an instruction to proceed with processing the stamp data. further comprising display control means for displaying the contents of the contract and displaying an inquiry as to whether or not to agree to the contents of the contract;
2. The image processing system according to claim 1, wherein when there is an instruction to agree to the contents of the contract, the generation means generates and the determination means makes a determination. The determination means makes the determination using a network model,
16. The image processing system according to claim 15, further comprising learning means for learning the network model based on the stamp data and the review result of the sales review request. further comprising a second selection means for selecting one of the plurality of photographed image data according to a user operation,
2. The image processing system according to claim 1, wherein the generation means generates the stamp data by performing image processing on the photographed image data selected by the second selection means. If the determination means determines that the stamp data does not conform to the guidelines, image data similar to the image data from which the stamp data was generated is selected from among the plurality of photographed image data. controlling to search and display the similar image data;
21. The image processing system according to claim 20, wherein the second selection means selects one of the similar image data in response to a user operation. Receiving means for receiving photographed image data or stamp data that is generated based on the photographed image data and can be used in a messenger application;
A server device comprising: a determination unit that determines whether stamp data that is generated based on the photographed image data and that can be used in a messenger application conforms to a guideline. a transmitting means for transmitting photographed image data or stamp data that is generated based on the photographed image data and can be used in a messenger application to a server device;
receiving means for receiving from the server device a determination result as to whether stamp data available in the messenger application generated based on the photographed image data conforms to guidelines;
and display control means for controlling the display of the determination result. a generation step of generating stamp data that can be used in the messenger app by image processing the photographed image data;
A processing method for an image processing system, comprising: a determining step of determining whether the stamp data generated in the generating step conforms to a guideline. a receiving step of receiving photographed image data or stamp data that is generated based on the photographed image data and can be used in a messenger application;
A processing method for a server device, comprising: a determining step of determining whether stamp data that is generated based on the photographed image data and that can be used in a messenger application conforms to a guideline. a sending step of transmitting the photographed image data or the stamp data that is generated based on the photographed image data and can be used in the messenger application to the server device;
a receiving step of receiving from the server device a determination result as to whether stamp data available in the messenger application generated based on the photographed image data conforms to guidelines;
A processing method for an imaging device, comprising: a display control step for controlling to display the determination result. A program for causing a computer to function as each means of an image processing system, a server device, or an imaging device according to any one of claims 1 to 23.

Images