JPWO2019058539A1 - Programs, information processing methods, and information processing equipment - Google Patents

Abstract

A first display step for displaying the first image on the information processing device, a step for acquiring the contour of the region to be cut out in the first image, and a plurality of editing points for correcting the contour are set on the contour, respectively. A second display step to be displayed, a third display step to enlarge and display the first image in response to a user operation, and to increase the number of the plurality of editing points to be displayed, and the plurality of display steps. The step of cropping the second image in the contour specified by the edit point is executed.

Description

The present disclosure relates to programs, information processing methods, and information processing devices.

Conventionally, a tool (program) capable of cropping (cutting and trimming) a part of an image by a user's operation is known (see, for example, Patent Document 1 and Non-Patent Document 1). In this tool, for example, the area to be cut out is designated by the user by an operation of tracing the outline of the area with a finger, a mouse cursor, or the like.

The cropped image can be used, for example, as a stamp or the like transmitted by an instant messaging service of SNS (Social Networking Service) or the like.

JP-A-2017-016670

GIMP, "2.7. Intelligent Scissors", [online], [Search on September 13, 2017], Internet <URL: https://docs.gimp.org/en/gimp-tool-iscissors.html>

However, in the prior art, for example, when the contour of an object such as a subject included in an image such as a photograph is specified by an operation of tracing with a finger, a mouse cursor, or the like, it may be difficult to accurately specify the contour. There's a problem.

The present disclosure has been made in view of the above problems, and an object of the present disclosure is to provide a technique capable of specifying an area of an object to be cut out from an image with relatively high accuracy and easily.

A program according to an embodiment of the present disclosure includes a first display step of displaying a first image on an information processing apparatus, a step of acquiring a contour of a region to be cut out in the first image, and a step of correcting the contour. A second display step of displaying a plurality of edit points on the contour, and an enlarged display of the first image in response to a user operation, and increasing the number of the plurality of edit points. A third display step to be displayed and a step to cut out a second image in the contour designated by the plurality of editing points are executed.

It is a figure which shows the structure of the communication system in one Embodiment of this disclosure. It is a flowchart which shows an example of the process at the time of cutting out a part of an image. It is a figure explaining the display example of the original image in a terminal. It is a figure explaining the display example of the confirmation screen of the cutout range in a terminal. It is a flowchart which shows an example of the process at the time of editing the cutout range which concerns on 1st Embodiment. It is a figure explaining an example of the method of displaying the edit point before the image is enlarged and displayed. It is a figure explaining an example of the method of increasing the number of edit points when the image is enlarged and displayed. It is a figure explaining an example of the method of reducing the number of edit points when the image is enlarged and then reduced and displayed. It is a figure which shows the structure of the communication system in 2nd Embodiment. It is a flowchart which shows an example of the process at the time of editing the cutout range which concerns on 2nd Embodiment. It is a figure explaining an example of the display screen of the 1st movement point group. It is a figure explaining an example of the data which edge detected the original image. It is a figure explaining the movement of the moving point when the housing of a terminal is tilted. It is a figure explaining the movement of the moving point when the housing of a terminal is tilted. It is a figure which shows an example of the display screen which fixed the position of the moving point by a user operation. It is a figure explaining an example of the display screen of the 2nd moving point group. It is a figure which shows an example of the data of the result of detecting the edge in the roughly specified area. It is a flowchart which shows an example of the process at the time of editing the cutout range which concerns on 3rd Embodiment. It is a figure explaining the process of moving and displaying an edit point by a flick operation. It is a figure explaining the process which the moving speed of an edit point is attenuated according to the strength of an edge, and is stopped and displayed. It is a figure explaining the process which the moving speed of an edit point is attenuated according to the strength of an edge, and is stopped and displayed.

<Compliance with legal matters>
It should be noted that the disclosures described herein are to be carried out in compliance with the legal matters of the implementing country regarding the implementation of this disclosure, such as secrecy of communications.

The program, the information processing method, and the embodiment for implementing the information processing apparatus according to the present disclosure will be described with reference to the drawings.

<System configuration>
FIG. 1 is a diagram showing a configuration of a communication system according to an embodiment of the present disclosure. As disclosed in FIG. 1, in a communication system, a server 10 and a terminal 20 (terminal 20A, terminal 20B, terminal 20C) are connected via a network 30. The server 10 provides a service for transmitting and receiving a message between terminals 20 to a terminal 20 owned by a user via a network 30. The number of terminals 20 connected to the network 30 is not limited.

The network 30 plays a role of connecting one or more terminals 20 and one or more servers 10. That is, the network 30 means a communication network that provides a connection route so that data can be transmitted and received after the terminal 20 connects to the server 10.

One or more parts of the network 30 may be a wired network or a wireless network. The network 30 is not limited, but by example, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), and a wireless network. LAN (wireless LAN: WLAN), wide area network (WAN), wireless WAN (wireless WAN: WWAN), metropolitan area network (MAN), part of the Internet, public exchange telephone network (Public) Switched Telephone Network: Part of PSTN, mobile network, ISDNs (integrated service digital networks), wireless LANs, LTE (long term evolution) CDMA (code division multiple access), Bluetooth (Bluetooth (registered trademark)), satellite communication Etc., or may include a combination of two or more of these. The network 30 may include one or more networks 30.

The terminal 20 (terminal 20A, terminal 20B, terminal 20C) may be any information processing terminal that can realize the functions described in each embodiment. The terminal 20 is not limited but, for example, a smartphone, a mobile phone (feature phone), a computer (not limited, for example, a desktop, a laptop, a tablet, etc.), a media computer platform (not limited, for example, a cable, a satellite set) Top boxes, digital video recorders), handheld computer devices (not limited to personal digital assistants, e-mail clients, etc.), wearable terminals (glasses devices, clock devices, etc.), or other types of computers , Or includes a communication platform. Further, the terminal 20 may be expressed as an information processing terminal.

Since the configurations of the terminal 20A, the terminal 20B, and the terminal 20C are basically the same, the terminal 20 will be described in the following description. Further, if necessary, the terminal used by the user X is expressed as the terminal 20X, and the user information in the predetermined service associated with the user X or the terminal 20X is expressed as the user information X. The user information is user information associated with an account used by the user in a predetermined service. The user information is not limited but, as an example, input by the user or given by a predetermined service, the user's name, the user's icon image, the user's age, the user's gender, the user's address, and the user's hobby. It includes information associated with the user such as a preference and a user's identifier, and may be any one or a combination of these.

The server 10 has a function of providing a predetermined service to the terminal 20. The server 10 may be any device as long as it is an information processing device that can realize the functions described in each embodiment. The server 10 is not limited, but by example, a server device, a computer (not limited, by example, a desktop, a laptop, a tablet, etc.), a media computer platform (not limited, by example, a cable, a satellite set top box, a digital video recorder). ), Handheld computer devices (for example, but not limited to PDAs, email clients, etc.), or other types of computers, or communication platforms. Further, the server 10 may be expressed as an information processing device.

<Hardware (HW) configuration>
The HW configuration of each device included in the communication system will be described with reference to FIG.

(1) HW configuration of the terminal

The terminal 20 includes a control device 21 (CPU: central processing unit), a storage device 28, a communication I / F 22 (interface), an input / output device 23, a display device 24, a microphone 25, a speaker 26, and a camera 27. Be prepared. Each component of the HW of the terminal 20 is connected to each other via bus B, for example, without limitation.

The communication I / F 22 transmits / receives various data via the network 30. The communication may be executed by wire or wirelessly, and any communication protocol may be used as long as mutual communication can be executed. The communication I / F 22 has a function of executing communication with the server 10 via the network 30. The communication I / F 22 transmits various data to the server 10 according to an instruction from the control device 21. Further, the communication I / F 22 receives various data transmitted from the server 10 and transmits the various data to the control device 21.

The input / output device 23 includes a device for inputting various operations to the terminal 20 and a device for outputting the processing result processed by the terminal 20. The input / output device 23 may be integrated with the input device and the output device, or may be separated into the input device and the output device.

The input device is realized by any or a combination of all kinds of devices capable of receiving an input from a user and transmitting information related to the input to the control device 21. The input device includes, but is not limited to, hardware keys such as a touch panel, a touch display, and a keyboard, a pointing device such as a mouse, a camera (operation input via a moving image), and a microphone (operation input by voice).

The output device is realized by any or a combination of all kinds of devices capable of outputting the processing result processed by the control device 21. The output device includes, for example, a touch panel, a touch display, a speaker (audio output), a lens (not limited, for example, 3D (three dimensions) output, a hologram output), a printer, and the like.

The display device 24 is realized by any or a combination of all kinds of devices capable of displaying according to the display data written in the frame buffer. The display device 24 includes a touch panel, a touch display, a monitor (not limited to an example, a liquid crystal display or an OELD (organic electroluminescence display)), a head mounted display (HDM), a projection mapping, and a hologram. , Includes a device capable of displaying images, text information, etc. in the air (may be vacuum). In addition, these display devices 24 may be able to display display data in 3D.

When the input / output device 23 is a touch panel, the input / output device 23 and the display device 24 may be arranged so as to face each other with substantially the same size and shape.

The control device 21 has a physically structured circuit for executing a function realized by a code or an instruction contained in a program, and is not limited to, but as an example, a data processing device built in hardware. Is realized by.

The control device 21 is not limited to, for example, a central processing unit (CPU), a microprocessor (microprocessor), a processor core (processor core), a multiprocessor (multiprocessor), an ASIC (application-specific integrated circuit), and an FPGA (field programmable). gate array) is included.

The storage device 28 has a function of storing various programs and various data required for the terminal 20 to operate. The storage device 28 includes various storage media such as HDD (hard disk drive), SSD (solid state drive), flash memory, RAM (random access memory), and ROM (read only memory).

The terminal 20 stores the program P in the storage device 28, and by executing the program P, the control device 21 executes the processing as each part included in the control device 21. That is, the program P stored in the storage device 28 causes the terminal 20 to realize each function executed by the control device 21.

The microphone 25 is used for inputting voice data. The speaker 26 is used for outputting audio data. The camera 27 is used for acquiring moving image data.

(2) Server HW configuration
The server 10 includes a control device 11 (CPU), a storage device 15, a communication I / F 14 (interface), an input / output device 12, and a display 13. Each component of the HW of the server 10 is connected to each other via bus B, for example, without limitation.

The control device 11 has a physically structured circuit for executing a function realized by a code or an instruction contained in a program, and is not limited to, but as an example, a data processing device built in hardware. Is realized by.

The control device 11 is typically a central processing unit (CPU), and may be a microprocessor, a processor core, a multiprocessor, an ASIC, or an FPGA. However, in the present disclosure, the control device 11 is not limited to these.

The storage device 15 has a function of storing various programs and various data required for the server 10 to operate. The storage device 15 is realized by various storage media such as HDD, SSD, and flash memory. However, in the present disclosure, the storage device 15 is not limited to these.

The communication I / F 14 transmits / receives various data via the network 30. The communication may be executed by wire or wirelessly, and any communication protocol may be used as long as mutual communication can be executed. The communication I / F 14 has a function of executing communication with the terminal 20 via the network 30. The communication I / F 14 transmits various data to the terminal 20 according to an instruction from the control device 11. Further, the communication I / F 14 receives various data transmitted from the terminal 20 and transmits the various data to the control device 11.

The input / output device 12 is realized by a device that inputs various operations to the server 10. The input / output device 12 is realized by any or a combination of all kinds of devices capable of receiving an input from a user and transmitting information related to the input to the control device 11. The input / output device 12 is typically realized by a hardware key represented by a keyboard or the like or a pointing device such as a mouse. The input / output device 12 is not limited to the input / output device 12, and may include, as an example, a touch panel, a camera (operation input via a moving image), and a microphone (operation input by voice). However, in the present disclosure, the input / output device 12 is not limited to these.

The display 13 is typically realized by a monitor (not limited to, for example, a liquid crystal display or an OELD (organic electroluminescence display)). The display 13 may be a head-mounted display (HDM) or the like. In addition, these displays 13 may be able to display display data in 3D. However, in the present disclosure, the display 13 is not limited to these.

The server 10 stores the program P in the storage device 15, and by executing the program P, the control device 11 executes the processing as each part included in the control device 11. That is, the program P stored in the storage device 15 causes the server 10 to realize each function executed by the control device 11.

Each embodiment of the present disclosure will be described as being realized by the CPU of the terminal 20 and / or the server 10 executing the program P.

The control device 21 of the terminal 20 and / or the control device 11 of the server 10 is not only a CPU but also a logic circuit formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)) or the like. Each process may be realized by (hardware) or a dedicated circuit. Further, these circuits may be realized by one or a plurality of integrated circuits, and a plurality of processes shown in each embodiment may be realized by one integrated circuit. Further, the LSI may be referred to as a VLSI, a super LSI, an ultra LSI, or the like depending on the degree of integration.

Further, the program P (software program / computer program) of each embodiment of the present disclosure may be provided in a state of being stored in a storage medium readable by a computer. The storage medium can store the program in a "non-temporary tangible medium".

When appropriate, the storage medium may be one or more semiconductor-based or other integrated circuits (ICs), such as, but not limited to, field programmable gate arrays (FPGAs) or application-specific ICs (ASICs). ), Hard disk drive (HDD), hybrid hard drive (HHD), optical disk, optical disk drive (ODD), magneto-optical disk, magneto-optical drive, floppy diskette, floppy disk drive (FDD), magnetic It can include tapes, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable storage medium, or any suitable combination of two or more of these. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate. The storage medium is not limited to these examples, and any device or medium may be used as long as the program P can be stored.

The server 10 and / or the terminal 20 can read the program P stored in the storage medium and execute the read program P to realize the functions of the plurality of functional units shown in each embodiment.

Further, the program P of the present disclosure may be provided to the server 10 and / or the terminal 20 via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. The server 10 and / or the terminal 20 realizes the functions of the plurality of functional units shown in each embodiment by executing the program P downloaded via the Internet or the like, as an example without limitation.

Each embodiment of the present disclosure can also be realized in the form of a data signal embedded in a carrier wave, in which the program P is embodied by electronic transmission.
At least a part of the processing in the server 10 and / or the terminal 20 may be realized by cloud computing composed of one or more computers.

At least a part of the processing in the terminal 20 may be performed by the server 10. In this case, the server 10 may perform at least a part of the processing of each functional unit of the control device 21 of the terminal 20.

The terminal 20 may perform at least a part of the processing on the server 10. In this case, the terminal 20 may perform at least a part of the processing of each functional unit of the control device 11 of the server 10.

Unless explicitly stated, the configuration of the determination in the embodiment of the present disclosure is not essential, and a predetermined process is operated when the determination condition is satisfied, or a predetermined process is performed when the determination condition is not satisfied. You may.

The program of this disclosure is not limited to, but examples include scripting languages such as ActionScript and JavaScript (registered trademark), object-oriented programming languages such as Objective-C and Java (registered trademark), and markup languages such as HTML5. Implemented using.

<First Embodiment>
The first embodiment is a point for correcting the contour of a region (range) to be cut out in the original image, and increases the number of editing points, which are points on the contour, when the image is enlarged and displayed. It is a form to be displayed. According to the first embodiment, the area of the object to be cropped from the image can be specified with relatively high accuracy and easily. The contents described in the first embodiment can be applied to any of the other embodiments described later.

<Functional configuration>
(1) Functional configuration of the terminal

As shown in FIG. 1, the terminal 20 has a reception unit 201, a display control unit 202, an image processing unit 203, and a control unit 204 as functions realized by the control device 21.

The reception unit 201 receives an operation from the user. The reception unit 201 receives, for example, an operation of designating the outline of the area to be cut out in the displayed original image.

The display control unit 202 causes the screen of the terminal 20 to display the display screen for trimming the image according to the instruction from the image processing unit 203 or the server 10. The display control unit 202 is, for example, a point for correcting the contour of a region (range) to be cut out in an image, and displays a plurality of editing points which are points on the contour. Further, the display control unit 202 enlarges and displays at least a part of the image in response to the user's operation, and increases the number of editing points to display.

The image processing unit 203 performs image processing for cutting out an image within the contour designated by a plurality of editing points.

The control unit 204 generates content including an image cut out by the image processing unit 203 and transmits the content to the server 10.
(2) Functional Configuration of Server As shown in FIG. 1, the server 10 has an acquisition unit 101, a display control unit 102, and a control unit 103 as functions realized by the control device 11.

The acquisition unit 101 acquires data from the terminal 20.

The display control unit 102 controls the display of the screen of the terminal 20.

The control unit 103 sells the content received from the terminal 20 to the user of each terminal 20 as a stamp or the like that can be used in, for example, an instant messaging service of SNS.
<Processing>
Next, with reference to FIG. 2, an example of processing for cropping a part of an image in the terminal 20 will be described. FIG. 2 is a flowchart showing an example of processing when cutting out a part of an image. FIG. 3A is a diagram illustrating a display example of an original image on the terminal 20. FIG. 3B is a diagram illustrating a display example of a cutout range confirmation screen in the terminal 20.

In step S1, the reception unit 201 of the terminal 20 receives an operation of selecting an image from the user.

Subsequently, the display control unit 202 of the terminal 20 displays the selected image (“first image”) on the screen (step S2). Here, as shown in FIG. 3A, for example, an image previously stored in the terminal 20 or an image taken by the terminal 20 is displayed. In the example of FIG. 3A, the image of the dog 301 is selected and displayed by the user.

Subsequently, the reception unit 201 and the display control unit 202 of the terminal 20 edit the cropping range in the selected image by an operation from the user (step S3).

Subsequently, the image processing unit 203 of the terminal 20 trims the image (“second image”) within the edited cropping range by the user's operation (step S4). In the example of FIG. 3B, the face portion 301A of the dog 301 within the edited cropping range is highlighted and displayed by reducing the brightness of the region outside the edited cropping range. In the example of FIG. 3B, when the confirmation button 302 is tapped, the image of the portion 301A within the edited cropping range is cropped and saved.

<< Trimming process of the first embodiment >>
Next, with reference to FIGS. 4 to 7, the process of editing the cutout range by the operation from the user in step S3 will be described. FIG. 4 is a flowchart showing an example of processing when editing the cropping range. FIG. 5 is a diagram illustrating an example of a method of displaying an edit point before the image is enlarged and displayed. FIG. 6 is a diagram illustrating an example of a method of increasing the number of editing points when the image is enlarged and displayed. FIG. 7 is a diagram illustrating an example of a method of reducing the number of edit points when the image is enlarged and then reduced and displayed.

In step S101, the reception unit 201 acquires the outline of the area to be cut out in the displayed original image in response to the user's operation.

Subsequently, the display control unit 202 displays a plurality of editing points on the contour (step S102). Here, for example, when the user performs an operation of tracing the screen with a finger or a mouse cursor, the display control unit 202 sets the contour as a line passing through each pixel traced by the user. In the example of FIG. 5, editing points 401, editing points 402, and the like adjacent to each other are displayed on the contour 400. Here, the display control unit 202 may display the editing points on the contour at predetermined intervals on the contour, for example. Alternatively, the display control unit 202 is located at a position where, for example, the length on the contour between two adjacent editing points is separated by a first threshold value (for example, 6 mm) or more, and each editing point is a feature on the contour. The editing points may be displayed at the positions of the points. The feature points on the contour will be described later.

Subsequently, the reception unit 201 receives from the user an operation for enlarging the displayed original image (step S103). Here, the reception unit 201 receives, for example, a pinch-out operation in which a thumb, an index finger, or the like is placed on the touch panel screen and spread by two fingers.

Subsequently, the display control unit 202 enlarges and displays the displayed original image, and increases the number of editing points to display (step S104).

In the example of FIG. 6, when the image of FIG. 5 is enlarged and the length on the contour between the editing point 401 and the editing point 402 is equal to or larger than the above-mentioned first threshold value, the display control unit 202 displays the image of FIG. The edit point 403 is additionally displayed on the contour between the edit point 401 and the edit point 402. Further, in the example of FIG. 6, the display control unit 202 has a second threshold value (for example,) on the contour between the edit point 401 and the edit point 402, and the length on the contour from the edit point 401 and the edit point 402, respectively. The edit point 403 is added and displayed at a predetermined position separated by 3 mm) or more.

The display control unit 202 may add the edit point 403 in the middle of the contour from the edit point 401 to the edit point 402. Alternatively, the display control unit 202 may add the edit point 403 to the feature points on the contour from the edit point 401 to the edit point 402. Here, the feature points on the contour are points representing the characteristics of the degree of bending of the contour, for example, singular points such as cusps when the contour is represented by an algebraic curve, and curvature in the algebraic curve. The extreme value point of the above may be used as a feature point on the contour.

Subsequently, the reception unit 201 receives from the user an operation of moving the position of the editing point to an arbitrary position on the screen (step S105). Here, the reception unit 201 receives, for example, a slide operation in which the editing point on the touch panel screen is touched by the finger and the finger is slid.

Subsequently, the display control unit 202 moves the position of the operated edit point in response to the operation (step S106). Here, for example, the display control unit 202 moves the operated edit point to the position where the finger is moved by a slide operation or the like, and the length on the contour from the operated edit point is a third threshold value. One or more other edit points (hereinafter referred to as "peripheral edit points") which are (for example, 1 cm) or less may also be moved in a direction according to the movement direction of the operated edit point. .. In this case, the display control unit 202 indicates that, for example, the moving distance of the peripheral editing point is proportional to the moving distance of the operated editing point, and the movement distance from the operated editing point to the peripheral editing point is on the contour. It may be inversely proportional to the length.

Subsequently, the display control unit 202 calculates the contour passing through the edited point after the movement, and displays the calculated contour after the change (step S107). Here, the display control unit 202 may calculate the contour passing through the edited edit points after the movement as an algebraic curve passing through each currently displayed edit point. Alternatively, the display control unit 202 may calculate the contour passing through the edited edit point after the movement as a straight line passing through each currently displayed edit point or the like.

Subsequently, the display control unit 202 adds or deletes the edit points to be displayed based on the currently displayed edit points and the changed contour (step S108). Here, when the display control unit 202 is located at a position where the length on the contour between two adjacent editing points is separated by a fourth threshold value (for example, 1 cm) or more due to the movement of the editing points, for example, each The edit point is added at the position where the edit point is the position of the feature point on the contour. Further, when the length on the contour between the two adjacent edit points becomes a position less than the above-mentioned second threshold value due to the movement of the edit points, the display control unit 202 sets the two adjacent edit points. Delete one of them (hide it).

Subsequently, the reception unit 201 receives from the user an operation for reducing the displayed original image (step S109). Here, the reception unit 201 receives, for example, a pinch-in operation in which a thumb, an index finger, or the like is placed on the touch panel screen and is pinched by two fingers.

Subsequently, the display control unit 202 reduces and displays the enlarged and displayed original image, and reduces the number of editing points to display (step S110).

Here, the display control unit 202 continues to display the edit point whose position has been changed based on the operation of moving the position of the edit point in step S105, and the position depending on the operation of moving the position of the edit point in step S105. The number of edit points that have not been changed may be reduced and displayed. As a result, even when the image is reduced, the edit points whose positions have been changed by the user operation, which are the feature points of the contour after the change, are continuously displayed, so that the shape of the contour can be made clearer to the user. Can be displayed.

In this case, the display control unit 202 has, for example, as an edit point whose position has been changed based on the operation of moving the position of the edit point in step S105, for example, only one edit point operated by the user. May be. Alternatively, in addition to the one editing point, the display control unit 202 adds the editing points around the one editing point automatically moved according to the movement of the one editing point in step S106, and added in step S108. It may include the edited points and the like. In the example of FIG. 7, the display control unit 202 reduces the edit points 601 and the edit points 602, and the edit points 603 whose positions are changed by the user's operation after being added when the display is enlarged. Even when it is displayed, it is continuously displayed.

Alternatively, the display control unit 202 may display a reduced number of edit points whose positions have been changed based on the operation of moving the position of the edit points in step S105. In this case, even if the display control unit 202 does not display the edit point displayed by the blue circle or the like, the contour line displayed by the white line or the like is a contour that passes through the edit point whose position has been changed. It may be displayed. In this case, the display control unit 202 may calculate and store the contour passing through the edited point after the movement in step S107 as an algebraic curve, for example. Then, the display control unit 202 erases the display of the edit points when the reduced display is performed, and when the enlarged display is performed again, the display control unit 202 becomes the feature points of the stored algebraic curve, etc., by the user operation in step S105. Display the edited point whose position has been changed again. As a result, the edited points that were erased when the display was reduced can be restored when the display is enlarged again.
<Effect of the first embodiment>
According to the first embodiment described above, when the image is enlarged and displayed, the number of editing points is automatically increased and displayed. As a result, the outline of the object to be cut out from the original image can be specified more accurately with a simpler operation. In addition, since the operation becomes easier, the processing load of the terminal 20 can be reduced as a result.

<Second embodiment>
The second embodiment is a mode in which a point designating a region to be cut out in the original image is moved and displayed in response to an operation of tilting the housing of the terminal 20 by the user. According to the second embodiment, the area of the object to be cropped from the image can be specified with relatively high accuracy and easily. Since the second embodiment is the same as the first embodiment except for a part, the description thereof will be omitted as appropriate. In the following, the parts common to the first embodiment will be omitted, and only the different parts will be described. The contents described in the second embodiment can be applied to any of the other embodiments.
<Hardware (HW) configuration>
FIG. 8 is a diagram showing a configuration of a communication system according to the second embodiment. As shown in FIG. 8, the terminal 20 according to the second embodiment further includes a tilt sensor 29. The tilt sensor 29 is, for example, an acceleration sensor, an angular velocity sensor, or the like, and is a sensor for detecting the tilt of the housing of the terminal 20.
<Functional configuration>
As shown in FIG. 8, the terminal 20 according to the second embodiment has a tilt detecting unit 205 as a function realized by the control device 21.

The tilt detection unit 205 detects the tilt of the housing of the terminal 20 based on the output from the tilt sensor 29.
<Processing>
<< Trimming process of the second embodiment >>
Next, with reference to FIGS. 9 to 13, the process of editing the cutout range by the operation from the user in step S3 will be described. FIG. 9 is a flowchart showing an example of processing when editing the cutout range according to the second embodiment. FIG. 10 is a diagram illustrating an example of a display screen of the first moving point group. FIG. 11 is a diagram illustrating an example of data in which the edge of the original image is detected. 12 and 13 are views for explaining the movement of the moving point when the housing of the terminal 20 is tilted. FIG. 14 is a diagram showing an example of a display screen in which the position of the moving point by the user operation is fixed. FIG. 15 is a diagram illustrating an example of a display screen of the second moving point group.

In step S201, the display control unit 202 is the first movement, which is one or more movement points for designating the area to be cut out in the displayed original image in response to the user's operation received by the reception unit 201. A point cloud (“first plurality of points”) is displayed from one side of the screen. In the example of FIG. 10, the display control unit 202 connects each movement point to a plurality of movement points 1001-1, 1001-2, or 1001-N arranged in the horizontal direction from the vicinity of the upper side of the screen of the terminal 20. Line 1002 (trimming line) is displayed.

Subsequently, the display control unit 202 detects the edge of the displayed original image (step S202). Here, the display control unit 202 calculates, for example, the magnitude of the gradient (edge strength) by first-order differentiating the image, predicts the local direction of the edge from the direction of the gradient, and the gradient in that direction is A method of searching for a locally maximum location may be used. Alternatively, the display control unit 202 may use, for example, a method of searching for a zero intersection in the second derivative obtained from the image. Alternatively, other known methods may be used. FIG. 11 shows an example of data 1101 as a result of edge detection of the original image shown in FIG. In the example of FIG. 11, the pixels whose detected edge strength is equal to or greater than a predetermined threshold value are shown in black. The display control unit 202 may use only the value of the detected edge strength that is equal to or greater than a predetermined threshold value.

Subsequently, the display control unit 202 sets the plurality of moving points 1001-1, 1001-2, or 1001-N and each moving point in response to the user's operation of tilting the housing of the terminal 20 in an arbitrary direction. The connecting line 1002 is moved in the tilted direction and displayed (step S203). Here, as the screen of the housing is tilted more from a predetermined position (for example, a horizontal position) by the user, each moving point may be moved and displayed at a faster speed.

Subsequently, the display control unit 202 temporarily stops each moving point at each portion of the edge and displays the moving point by a virtual force (adhesive force or frictional force) according to the strength of the edge of the original image (adhesive force or frictional force). Step S204). Here, the display control unit 202 indicates that the edge at the pixel position on the edge data as shown in FIG. 11 corresponds to the pixel position on the original image as shown in FIG. 10 where each moving point is currently located. Extract the strength value. Then, the display control unit 202 temporarily stops each moving point by a virtual force according to the strength of the extracted edge.

Further, the display control unit 202 displays a trimming line connecting the moving points by, for example, a curve or a straight line along the edge detected from the original image passing through the adjacent moving points.

The moving point once stopped is moved again by further tilting the housing. In this case, for example, when the housing is tilted by the user, the display control unit 202 is virtual according to the force in the direction in which the moving point slides down due to the gravity applied to the moving point and the strength of the edge. The virtual inertial force that the moving point tries to stop at the position on the edge is calculated by the adhesive force or the frictional force on the edge, which is a force. Then, even if the display control unit 202 displays the moving point so as to move in the sliding down direction when the force in the sliding down direction becomes larger than the inertial force trying to stop at the position on the edge. Good.

Also, if the housing is tilted sufficiently large that the force in the sliding direction is greater than the inertial force that tries to stop at a position on each edge, the moving point temporarily stops on each edge. Instead, it may be moved to the edge of the screen of the terminal 20 and displayed.

The process of moving the moving point in step S203 and the process of temporarily stopping each moving point by a virtual force according to the edge strength of step S204 are alternately repeated by the operation of tilting the housing by the user. Is done.

In the example of FIG. 12, the original image shown in FIG. 11 is virtual according to the value of the edge strength at the position of the pixel on the data in which the edge is detected, which corresponds to the position of the pixel on the original image shown in FIG. Each movement point is temporarily stopped by a strong force. As shown in FIG. 12, the moving point 1201 temporarily stopped on the edge of the ear portion of the dog 301 and the moving point 1202 temporarily stopped on the edge of the body portion of the dog 301 shown in FIG. , The moving point 1201 is temporarily stopped by a higher virtual force than the moving point 1202. Therefore, when the screen of FIG. 12 is displayed, if the housing is tilted to a certain degree so as to raise the upper part of the screen to a certain extent, the moving point 1201 remains stopped and the moving point 1202 moves toward the lower part of the screen. Moved to. Then, the moving point 1202 moves to the position of the lower part of the face of the dog 301 according to the edge strength as shown in FIG. 11 and is temporarily stopped as shown in the moving point 1302 shown in FIG.

In the examples of FIGS. 12 and 13, the display control unit 202 displays, for example, so that the moving points slide down due to the gravity applied to each moving point when the housing is tilted by the user. Then, when the length on the edge between adjacent moving points on the line 1002 connecting the moving points is equal to or greater than the above-mentioned first threshold value, the display control unit 202 puts it on the edge between the moving points. Movement points are added and displayed at the positions of feature points and the like. Further, in the examples of FIGS. 12 and 13, the display control unit 202 is tilted by the user in the vertical direction of the screen, so that the straight line connecting the moving points is stopped so as to hang down on the object from above or below. Is displayed. Then, the display control unit 202 is tilted by the user in the left-right direction of the screen, so that the straight line connecting the moving points is stopped and displayed so as to hang down from the left and right on the object.

Subsequently, when the display control unit 202 detects that the reception unit 201 has touched (pressed) one movement point included in each movement point by the user (step S205), the touch-operated movement point The position is fixed (step S206). In the example of FIG. 14, the display control unit 202 displays the moving point whose position is fixed in a manner that can be distinguished from the moving point whose position is not fixed. For example, when the moving point whose position is not fixed is a green circle, the moving point whose position is fixed is displayed by a double circle (round middle black), a blue circle, or the like.

Subsequently, the display control unit 202 responds to the operation of the user received by the reception unit 201, and in response to the operation of the user, the display control unit 202 specifies a second moving point group (“second plurality) for designating the area to be cut out in the displayed original image. Point ”) is displayed from the other side facing one side of the screen on which the first moving point group is displayed (step S207). In the example of FIG. 15, among the first moving point group, only the moving points whose positions are fixed by the process of step S206 are displayed. Then, from the vicinity of the lower side of the screen of the terminal 20, a plurality of moving points 1501-1, 1501-2, 1501-N arranged in the horizontal direction and a line 1502 (trimming line) connecting the moving points are displayed. There is.

Subsequently, the same processing as in steps S203 to S206 is performed on the second moving point group as in the first moving point group (steps S208 to S211). As a result, the moving points whose positions are fixed by the user's operation in the above-mentioned first moving point group and the moving points whose positions are fixed by the user's operation in the above-mentioned second moving point group Specifies the outline of the area to be cut out. In this case, the display control unit 202 may calculate the contour by a curve or a straight line along the edge detected from the original image passing through the adjacent moving points.
<First modification of the second embodiment>
Next, a modified example will be described with reference to FIG. FIG. 16 is a diagram showing an example of data as a result of detecting an edge in a roughly specified region. A combination of the process of designating the area to be cut out by the editing point in the first embodiment described above and the process of designating the area to be cut out by moving the moving point on the edge according to the inclination of the housing according to the second embodiment. May be good.

In this case, for example, by the process of the first embodiment shown in FIG. 4, the region to be cut out is roughly specified as shown in FIG. 7. Then, in the process of step S202 of the second embodiment shown in FIG. 9, as shown in FIG. 14, the edge is detected only in the designated area 1401 in the original image. As a result, in the processes after step S203 shown in FIG. 9, it is possible to more easily operate to specify the area to be cut out by moving it on the edge. In this case, the process of fixing the position of the moving point by the user's touch operation, such as step S205 in FIG. 9, may be omitted. Further, the processing of steps S207 to S211 of FIG. 9 using the second trimming line may be omitted.
<Second variant of the second embodiment>
After the process of moving the moving point on the edge to specify the area to be cut out according to the inclination of the housing according to the second embodiment described above, the area to be cut out is designated by the editing point in the first embodiment described above. The process may be enabled. In this case, the above-mentioned moving point is used as an editing point in the first embodiment as described below.

First, the display control unit 202 specifies an area to be cut out by moving the moving point on the edge according to the inclination of the housing by the process shown in FIG. 9 of the second embodiment. Then, for example, the display control unit 202 enlarges and displays the original image by the processing of steps S103 to S110 shown in FIG. 4 of the first embodiment described above, sets the moving points as editing points, and sets the number of editing points. Increase and display. Then, the editing point is moved by the operation of moving the editing point with a finger or the operation of tilting the housing.
<Effect of the second embodiment>
According to the second embodiment described above, a point designating a region to be cut out in the original image is moved and displayed according to the inclination of the housing. As a result, the outline of the object to be cut out from the original image can be specified more accurately with a simpler operation. In addition, since the operation becomes easier, the processing load of the terminal 20 can be reduced as a result.
<Third embodiment>
The third embodiment is a mode in which, in response to an operation such as a flick operation or a swipe operation of the user, a point for designating a region to be cut out in the original image is moved to an edge position in the original image and displayed. According to the third embodiment, the area of the object to be cropped from the image can be specified with relatively high accuracy and easily. Since the third embodiment is the same as the first embodiment or the second embodiment except for a part, the description thereof will be omitted as appropriate. In the following, the description of the parts common to the first embodiment or the second embodiment will be omitted, and only the different parts will be described. The contents described in the third embodiment can be applied to any of the other embodiments.
<Processing>
<< Trimming process of the third embodiment >>
Next, with reference to FIGS. 17 to 19, the process of editing the cutout range by the operation from the user in step S3 will be described. FIG. 17 is a flowchart showing an example of processing when editing the cutout range according to the third embodiment. FIG. 18 is a diagram illustrating a process of moving and displaying an edit point by a flick operation. 19A and 19B are diagrams for explaining a process in which the moving speed of the editing point is attenuated according to the strength of the edge, and the editing point is stopped and displayed.

In step S301, the reception unit 201 acquires the outline of the area to be cut out in the displayed original image in response to the user's operation. Subsequently, the display control unit 202 displays a plurality of editing points on the contour (step S302).

Here, in the processing of step S301 and step S302, even if the contour traced by the user with a finger or the like is acquired as in the processing of step S101 and step S102 shown in FIG. 4 of the first embodiment described above. Good. Alternatively, the number of edited points or the like increased with the enlarged display in any of the processing stages of steps S103 to S110 shown in FIG. 4 of the first embodiment described above may be displayed. Alternatively, the moving point in any of the processing stages of step S201 to step S211 shown in FIG. 9 of the second embodiment described above may be used as the editing point.

Subsequently, the reception unit 201 accepts a user's flick operation (or swipe operation) for one edit point (hereinafter referred to as "first edit point") included in the plurality of edit points (step S303). Here, the flick operation is, for example, an operation such as quickly moving or flipping a finger on a screen having a touch panel. The swipe operation is an operation in which a finger is slid on a screen having a touch panel.

Subsequently, the display control unit 202 moves the first edit point and the edit points around the first edit point (hereinafter, referred to as "second edit point") in the direction of the flick operation (step S304). ). Here, the display control unit 202 moves the first edit point at a speed corresponding to the speed of the flick operation, and determines the speed of the flick operation and the distance between the first edit point and the second edit point on the contour. The second edit point is moved at a speed corresponding to. The direction of the flick operation is, for example, the direction in which the finger is moved by the flick operation. The speed of the flick operation is, for example, the speed at which the finger is moved by the flick operation.

In the example of FIG. 18, the display control unit 202 determines that the distance on the contour from the flick-operated first edit point 1801 is equal to or less than a predetermined threshold value, or the number of adjacent edit points from the first edit point 1801 is predetermined. Editing points 1802, editing points 1803, and editing points 1804 that are equal to or less than the threshold are selected. Here, the predetermined threshold value may be a value including at least other editing points adjacent to the first editing point.

Then, the display control unit 202 calculates the distance on the contour between the first editing point 1801 and the selected editing point 1802, sets the speed of the flick operation and the speed corresponding to the calculated distance as the magnitude, and flicks. Calculate the velocity vector 1802A with the direction of operation as the orientation. Here, the display control unit 202 may set the size to, for example, a size proportional to the speed of the flick operation and inversely proportional to the calculated distance.

Further, the display control unit 202 calculates the speed vector 1803A and the speed vector 1804A for the selected edit points 1803 and 1804, respectively.

Then, the display control unit 202 moves the first edit point 1801 with the speed vector 1801A having the speed corresponding to the speed of the flick operation and the direction of the flick operation, and the edit point 1802, the edit point 1803, and the edit point 1803. The edit point 1804 is moved by the velocity vector 1803A and the velocity vector 1804A.

In the example of FIG. 18, the distance on the contour from the first editing point 1801 increases in the order of editing point 1802, editing point 1803, and editing point 1804. In this case, the velocity vector 1802A, the velocity vector 1803A, and the velocity vector 1804A have the same direction, and the velocity decreases in the order of the velocity vector 1802A, the velocity vector 1803A, and the velocity vector 1804A.

Subsequently, when the display control unit 202 moves to the position of the edge of the original image in which each moved editing point is displayed, a virtual force (adhesive force or frictional force) according to the strength of the edge of the original image is obtained. ) Stops and displays each edit point at a position on the edge (step S305). Here, the display control unit 202 may detect an edge in the original image in advance. Alternatively, the display control unit 202 may detect only the edges in the original image along the moving direction from the moving source position of each moved editing point. As a result, the processing load on the terminal 20 can be reduced.

Here, the display control unit 202 attenuates the moving speed of each editing point according to, for example, the speed of the flick operation and the strength of the edge on the path in which each editing point moves in response to the flicking operation. You may let it stop. In this case, the display control unit 202 moves the editing points 1901 by the speed vector 1901A for each moving editing point, as shown in FIG. 19A. Then, when passing through the edge 1903 on the movement direction 1902, the display control unit 202 attenuates the speed of movement of the edit point 1901 by a virtual force corresponding to the strength of the edge 1903. Then, as shown in FIG. 19B, the display control unit 202 moves the edit point 1901 with the speed vector 1901B whose moving direction is invariant and whose speed is attenuated from the speed vector 1901A. Then, the display control unit 202 stops the editing point 1901 at the position of the edge 1904 on the moving direction 1902 by a virtual force corresponding to the strength of the edge 1904 on the moving direction 1902.

Subsequently, the display control unit 202 calculates the contour passing through the edited point after the movement, and displays the calculated contour after the change (step S306). Subsequently, the display control unit 202 adds or deletes the edit points to be displayed based on the currently displayed edit points and the changed contour (step S307). The processing of steps S306 to S307 may be the same as the processing of steps S107 to S108 of FIG. 4 of the first embodiment described above.
<First modification of the third embodiment>
When the display control unit 202 receives the pinch-out operation or the like after moving the editing point to the edge position in response to the flick operation or the like, the display control unit 202 receives steps S103 to S110 shown in FIG. 4 of the first embodiment described above. May be performed. That is, the original image is enlarged and displayed, the moving points are set as editing points, and the number of editing points is increased to be displayed. Then, in the enlarged display state, the editing point is moved to the position where the finger is released by the slide operation of moving the editing point with the finger.
<Effect of the third embodiment>
According to the third embodiment described above, in response to an operation such as a flick operation or a swipe operation of the user, a point that specifies a region to be cut out in the original image is moved to a position of an edge in the original image and displayed. As a result, the outline of the object to be cut out from the original image can be specified more accurately with a simpler operation. In addition, since the operation becomes easier, the processing load of the terminal 20 can be reduced as a result.

<Others>
In the above-described embodiment, an example of performing image processing in an application installed on the terminal 20 has been described, but the server 10 may perform image processing. In this case, for example, the reception unit 201 of the terminal 20 notifies the server 10 of the operation content by the user, the display control unit 102 of the server 10 controls the display screen of the terminal 20, and the control unit 103 of the server 10 trims. The image processing such as may be performed.

The reception unit 201 is an example of the “acquisition unit”. The image processing unit 203 is an example of a “cutting unit”. The server 10 and the terminal 20 are examples of "information processing devices", respectively.

Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and modifications based on the present disclosure. Therefore, it should be noted that these modifications and modifications are within the scope of this disclosure. As an example, not limited, the functions included in each means, each step, etc. can be rearranged so as not to be logically inconsistent, and a plurality of means, steps, etc. can be combined or divided into one. Is possible. Further, the configurations shown in each embodiment may be appropriately combined.

10 Server 101 Acquisition unit 102 Display control unit 103 Control unit 20 Terminal 201 Reception unit 202 Display control unit 203 Image processing unit 204 Control unit 205 Tilt detection unit

Claims (8)

For information processing equipment
The first display step of displaying the first image and
The step of acquiring the outline of the area to be cut out in the first image, and
A second display step of displaying a plurality of edit points for modifying the contour on the contour, respectively.
A third display step in which the first image is enlarged and displayed in response to a user operation and the number of the plurality of editing points is increased and displayed.
A step of cropping a second image within the contour specified by the plurality of editing points,
A program that executes. In the second display step, the first edit point and the second edit point are displayed.
In the third display step, when the length on the contour from the first editing point to the second editing point is equal to or greater than the first threshold value, the third display step is on the contour from the first editing point to the second editing point. To display the third edit point,
The program according to claim 1. In the third display step, on the contour from the first editing point to the second editing point, the length on the contour from the first editing point and the second editing point is equal to or larger than the second threshold value. Display the third edit point at a distant position.
The program according to claim 2. In the third display step, the third edit point is displayed at the feature points on the contour from the first edit point to the second edit point.
The program according to claim 2 or 3. In the third display step, the third edit point is displayed at an intermediate point on the contour from the first edit point to the second edit point.
The program according to claim 2 or 3. A step of changing the position of one or more edit points among the plurality of edit points displayed by the third display step in response to a user operation.
In response to the user's operation, the first image enlarged and displayed by the third display step is reduced and displayed, and the editing points whose positions have been changed by the changing step continue to be displayed. A fourth display step in which the number of edit points whose positions have not been changed by the change step is reduced and displayed.
The program according to any one of claims 1 to 5. Information processing device
The first display step of displaying the first image and
The step of acquiring the outline of the area to be cut out in the first image, and
A second display step of displaying a plurality of edit points for modifying the contour on the contour, respectively.
A third display step in which the first image is enlarged and displayed in response to a user operation and the number of the plurality of editing points is increased and displayed.
A step of cropping a second image in the contour designated by the plurality of editing points in the first image, and
Information processing method to execute. An acquisition unit that acquires the outline of the area to be cropped in the displayed first image,
A plurality of editing points for modifying the contour are displayed on the contour, respectively, and the first image is enlarged and displayed in response to a user operation, and the number of the plurality of editing points is increased. The display control unit that lets you display
A cutout portion for cutting out the second image in the contour designated by the plurality of editing points in the first image, and
Information processing device with.