JP2017056607A - Data processing device, data processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it easier to manage a plurality of objects regarding a subject by synthesizing the objects as a batch.SOLUTION: An object processing device 100 acquires three-dimensional data from a server 101 and acquires a plurality of objects regarding a subject from the three-dimensional data. An overlapping volume calculation part 104 calculates an overlapping amount of regions where the plurality of objects overlap with each other as an overlapping volume. An overlapping volume setting part 105 accepts designation of an overlapping volume requested by a user and sets an overlapping volume value. An object synthesizing part 112 synthesizes the plurality of objects when the overlapping volume calculated by the overlapping volume calculation part 104 reaches a prescribed value or reaches the value set by the overlapping volume setting part 105. A liquid crystal panel 114 displays an image of the object after the regions of the overlapping volume are colored and synthesized.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for processing three-dimensional data for printing with a 3D (three-dimensional) printer.

  In the 3D printer, the user can select and print a desired scene from the 3D moving image data. Patent Document 1 discloses a method of generating a composite image by laying out the cut out subject images so as not to overlap as a method of displaying the movement of the subject images in an easy-to-see manner. The image recording apparatus disclosed in Patent Literature 1 detects an image area where a moving subject exists from moving image data, sets a cut-out area, extracts it at a predetermined sampling interval, and arranges them in a horizontal or vertical direction and combines them. Thus, a composite image is generated.

JP 2012-99876 A

In Patent Document 1, when a plurality of objects related to the same subject that are continuously cut out in time series are arranged and printed by a 3D printer, a solid object (product) of each object is generated separately, which makes management difficult. is there.
An object of the present invention is to facilitate management by combining a plurality of objects related to a subject into a group.

  An apparatus according to an embodiment of the present invention includes: a first acquisition unit that acquires three-dimensional data; a second acquisition unit that acquires a plurality of objects related to a subject from the three-dimensional data; Computation means for computing the amount of overlap of the region where the objects overlap, and composition means for composing the plurality of objects that overlap each other when the amount of overlap computed by the computation means is not zero.

  According to the present invention, management is facilitated by combining a plurality of objects related to a subject into a group.

It is a block diagram which shows the function structure of the object processing apparatus which concerns on 1st Embodiment of this invention. It is a flowchart of the production | generation process of the three-dimensional data which concerns on 1st Embodiment of this invention. It is a flowchart which shows the process which follows FIG. It is a flowchart of the registration process of the reference | standard object which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the production | generation process of the three-dimensional data which concerns on 1st Embodiment of this invention. It is a flowchart of the production | generation process of the three-dimensional data which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the process which follows FIG. It is a figure for demonstrating the production | generation process of the three-dimensional data which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating the operation | movement which sets an overlap volume in 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each embodiment, an object processing device will be described as an example of a three-dimensional data processing device.
[First Embodiment]
FIG. 1 is a block diagram illustrating a configuration example of an object processing apparatus 100 according to the first embodiment of the present invention. The object processing apparatus 100 can communicate with an external server 101. The storage device of the server 101 stores 3D moving image data.

  The 3D data acquisition unit 102 acquires 3D moving image data from the server 101. The subject object acquisition unit 103 detects and extracts a subject based on the frame number and the coordinate information of the frame, and acquires 3D data that can be printed by a 3D printer. Hereinafter, this 3D data (subject object data) is abbreviated as OBJ data. The overlap volume calculation unit 104 calculates a volume (hereinafter referred to as an overlap volume) as an overlap amount of a portion where the subject objects overlap. The overlap volume is calculated using a logical product operation. The overlap volume setting unit 105 sets an overlap volume between objects of the subject.

  The microcomputer 106 is a control center that controls the entire system. A microcomputer (hereinafter referred to as a control unit) 106 includes a CPU (Central Processing Unit), a nonvolatile memory (ROM) for storing programs, and a volatile memory (RAM) serving as a work area. The control unit 106 has at least an external bus for transferring data with other hardware and accessing a control register, and a timer for measuring time. Each block (see reference numerals 102 to 106, 108, 111, 112, and 115) is connected to the bus 107. A bus 107 is a transmission path for transferring data according to the control of the control unit 106.

  The interface unit 108 controls data transfer between the control unit 106, the memory 109, and the data transmission unit 110 under the control of the control unit 106. The memory 109 is a storage device in which each block can be used for work, and is also used as buffer means for temporarily storing 3D moving image data input from the server 101. The data transmission unit 110 transmits stereoscopic data (three-dimensional data) to a 3D printer (not shown).

  The 2D conversion unit 111 performs processing for converting 3D moving image data into 2D (two-dimensional) image data and reproducing it. The object combining unit 112 acquires and combines a plurality of OBJ data, and outputs the combined OBJ data. The switch operation unit 113 includes a switch for a user to operate the object processing apparatus 100, and the control unit 106 receives an operation instruction by a user operation. For example, the switch operation unit 113 includes an operation switch for reproducing 3D moving image data and operation buttons for changing various parameters.

  The liquid crystal panel 114 constitutes a display unit and displays images of objects and virtual buttons on the screen. The touch panel operation unit 115 includes an operation device that is used integrally with the liquid crystal panel 114. When the user instructs the object processing apparatus 100 using the touch panel operation unit 115 while the liquid crystal panel 114 displays thumbnail images and virtual button images on the screen, the control unit 106 receives the operation instructions.

Next, a process for generating three-dimensional data for printing with a 3D printer will be described with reference to FIGS. 2 to 4 are flowcharts showing processing examples. When the user sets the playback mode using the switch operation unit 113, the control unit 106 executes the following processing.
In S201 of FIG. 2, the control unit 106 sets a reference object (reference object). The reference object setting process will be described with reference to the flowchart of FIG.

  In S301 of FIG. 4, the 3D data acquisition unit 102 acquires content data to be reproduced from the server 101 in accordance with an instruction from the control unit 106. The content data is 3D moving image data. The acquired 3D moving image data is reproduced by 2D conversion by the 2D conversion unit 111 in accordance with an instruction from the control unit 106, and an image is displayed on the liquid crystal panel 114. In S <b> 302, when the user touches the liquid crystal display screen with the touch panel operation unit 115, the control unit 106 temporarily stops the reproduction of the 2D conversion unit 111. The control unit 106 holds the frame number in the memory 109 using the temporarily stopped frame as a reference frame.

  In S303, a reference object (hereinafter also referred to as reference OBJ) selection process is performed. The reference OBJ is selected when the user touches the liquid crystal display screen with the touch panel operation unit 115 on the image of the subject existing in the frame set in S302. The control unit 106 holds data of coordinates selected by a user operation in the memory 109. In step S304, the control unit 106 registers the selected object. In object registration, the control unit 106 reads out the frame number and coordinates held in the memory 109 and sets them in the 3D data acquisition unit 102 and the subject object acquisition unit 103. The 3D data acquisition unit 102 acquires the 3D moving image data of the corresponding frame from the server 101 and inputs it to the subject object acquisition unit 103. The subject object acquisition unit 103 performs subject detection and extraction processing from the coordinates set by the control unit 106, and acquires OBJ data. The control unit 106 registers the acquired OBJ data in the object composition unit 112. The object composition unit 112 holds the input OBJ data in the memory 109. A specific example of processing will be described with reference to FIG.

  FIG. 5A illustrates a state in which the moving image that has been 2D-converted by the 2D conversion unit 111 is displayed on the screen of the liquid crystal panel 114, and illustrates how the subject moves from the right side to the left side of the screen. The user registers the subject object of the desired scene as a reference object after performing an operation to pause the moving image being played back.

  In S202 to S206 of FIG. 2, a process for searching for an object having a small overlapping volume is executed. In S202, the control unit 106 instructs the 3D data acquisition unit 102 to advance the frame of the 3D data acquisition unit 102 whose reproduction has been paused by 1V. 1V represents a vertical period corresponding to the frame rate. The 3D data acquisition unit 102 acquires 3D moving image data of the corresponding frame from the server 101 and inputs it to the subject object acquisition unit 103. The subject object acquisition unit 103 acquires OBJ data from the newly input 3D moving image data and the coordinates calculated from the coordinates of the image before 1V. For the coordinates, the control unit 106 calculates the face image of the subject before 1V from the color difference or the like. Thereby, the data of the object (hereinafter referred to as comparison OBJ) to be compared by the control unit 106 can be acquired.

  In S203 of FIG. 2, the overlapping volume calculation unit 104 calculates the overlapping volume between the reference OBJ and the comparison OBJ set in S202. The overlap volume calculation unit 104 calculates the overlap volume between the objects of the subject using a logical product that is a known method. In S204, the control unit 106 determines whether it is the first calculation. If the current calculation is the first calculation, the process proceeds to S205, and if it is the second or subsequent calculation, the process proceeds to S207. In step S205, the comparison OBJ data is held in the memory 109. In S207, comparison OBJ data with a small overlap volume is held in the memory 109 based on the calculation result of S203.

  After the processing in S205 or S207, in S206, the control unit 106 determines whether or not there is no overlap between the reference OBJ and the comparison OBJ. In the determination process of whether or not there is an overlap, the control unit 106 determines whether or not the overlap volume between objects is zero. If the objects overlap each other, the process returns to S203, and the calculation process of the overlapping volume between the comparison OBJ after 1V and the reference OBJ is executed. The processing from S203 to S206 is repeatedly executed. Thereby, the comparison OBJ data in which the overlapping volume is minimized with respect to the reference OBJ data among the plurality of comparison OBJ data is detected.

  On the other hand, if it is determined in S206 that the objects do not overlap, the process proceeds to S208. In step S <b> 208, the control unit 106 adds the comparison OBJ data having a small overlapping volume held in the memory 109 to the object for printing. The print object addition process is performed by registering the OBJ data of the comparison object held in the memory 109 in the object composition unit 112. The object synthesis unit 112 synthesizes the registered comparison OBJ data with the reference OBJ data and holds it on the memory 109. A specific description will be given with reference to FIG. The reference object is located at the center of FIG. 5B, and the objects A1 and A2 are shown on the left side and the objects M1 and M2 are shown on the right side. These objects A1, A2, M1, and M2 are objects of the same subject as that of the reference object. The object A1 after the elapse of several V from the frame of the reference object is extracted and added to the print object.

  In S209 of FIG. 2, the control unit 106 changes the OBJ data of the object A1 of FIG. 5B to the reference OBJ data and holds it in the memory 109. In S210, a determination process is performed as to whether or not the moving image reproduction has ended. If the moving image reproduction has not ended, the process returns to S202, the object A1 is changed to the reference object, and search processing for an object having a small overlapping volume is executed. Thereby, an object A2 having a small overlapping volume is extracted from the object A1 shown in FIG. The extracted OBJ data of the object A2 is registered in the object composition unit 112 and added to the print object. Through the processes of S203 to S210, OBJ data obtained by combining the reference object and the objects A1 and A2 is generated.

  If it is determined in S210 that the moving image has ended, the process proceeds to S211 in FIG. In step S211, the control unit 106 returns the setting of the reference OBJ, and holds the frame number and object coordinates of the reference OBJ set in step S201 in FIG. In step S212, the control unit 106 instructs the 3D data acquisition unit 102 to return the frame of the 3D data acquisition unit 102 whose playback has been paused to 1V. The 3D data acquisition unit 102 acquires 3D moving image data of the corresponding frame from the server 101 and inputs it to the subject object acquisition unit 103. The subject object acquisition unit 103 acquires OBJ data from newly input 3D moving image data and coordinates calculated from the coordinates of the image after 1V. For the coordinates, the control unit 106 calculates the face image of the subject after 1V from the color difference or the like.

  The processes of S213 to S220 correspond to the processes of S203 to S210, but are executed on a frame temporally prior to the frame of the reference object. That is, except that the processing targets are different, the same processing is performed for the other parts, and details thereof are omitted.

  As a result of the processing of S213 to S220, objects M1 and M2 shown in FIG. 5B are set. The extracted M1 and M2 OBJ data is registered in the object composition unit 112. The object combining unit 112 combines the OBJ data of M1 and M2 with the combined data stored in the memory 109 and stores the combined data in the memory 109. OBJ data obtained by combining the reference object and the objects M1 and M2 is generated. As a result, OBJ data in which the reference object and the objects A 1, A 2, M 1, and M 2 are combined is created and stored in the memory 109.

  If the end of the moving image is determined in S220, the process ends. As shown in FIG. 5B, the reference OBJ corresponding to the movement of the subject and the objects A1, A2, M1, and M2 are displayed on the screen of the liquid crystal panel 114. When printing with a 3D printer, the control unit 106 acquires the synthesized OBJ data from the memory 109 and transfers it from the data transmission unit 110 to the 3D printer. The 3D printer generates a group of three-dimensional objects as shown in FIG.

For the purpose of presenting the overlapping volume of multiple objects in a way that is easy for the user to understand, for areas where objects overlap, the display color, brightness, etc. are displayed separately from areas where the objects do not overlap . For example, a region where the objects overlap is distinguished by coloring and displayed on the liquid crystal panel 114. In FIG. 5C, the reference object is shown in the center, and the objects A1 and M1 are shown on the left and right, respectively. In the overlapping area K1 between the reference object and the object M1, the overlapping volume area is displayed in different colors. Similarly, for the overlapping area K2 between the reference object and the object A1, the overlapping volume area is displayed in different colors.
According to this embodiment, based on the amount of overlap between a plurality of objects related to a subject acquired from three-dimensional data, product management is performed by combining a plurality of objects that overlap each other into a single object through a synthesis process. Becomes easier. Further, by combining a plurality of objects having the smallest amount of overlap, it is possible to avoid the objects from overlapping too much.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the present embodiment, an example in which the user sets the overlapping volume will be described. An operation for generating three-dimensional data for printing by a 3D printer will be described with reference to FIGS. 6 and 7 are flowcharts for explaining processing examples. FIG. 8 is a diagram for explaining a process of generating three-dimensional data. 8A shows the movement of the subject, and FIG. 8B shows the reference object and a plurality of objects related to the same subject. FIG. 8C is a schematic diagram showing a product of an object. FIG. 9 is an explanatory diagram of the overlap volume setting process.

In S501 of FIG. 6, a setting process is performed for the overlapping volume of the subject objects. The control unit 106 notifies the user that the overlapping volume value can be selected on the display screen of the liquid crystal panel 114. The user uses the touch panel operation unit 115 to select whether to increase or decrease the overlapping volume. When detecting that the user has selected the overlapping volume, the control unit 106 sets the overlapping volume value in the overlapping volume setting unit 105. The overlapping volume setting unit 105 holds the overlapping volume value set by the control unit 106 in the memory 109. The overlapping volume value is a value when at least a part of the objects of the subject overlap, that is, a value larger than zero.
Each process of S502 to S521 in FIGS. 6 and 7 corresponds to each process of S201 to S220 in FIGS. Since the same processing is performed except that an object close to the overlapping volume value set by the user's operation instruction is extracted (see S508 and S518), detailed description thereof will be omitted and differences will be described.

  When the end of the moving image is determined in S521, in S522, the control unit 106 determines whether or not to update the overlapping volume of the subject objects. If the control unit 106 determines to update the overlap volume, the process returns to S501 in FIG. FIG. 9 shows the reference object in the center of the screen, and objects B1 and N1 on the left and right, respectively. Objects B1 and N1 are objects of the same subject as the reference object. The user gives an instruction to change the subject image displayed on the screen of the liquid crystal panel 114 by a touch operation. In the example of FIG. 9, the object N1 overlaps on the right side of the reference object. When the user touches the position of the object N1 with the touch panel operation unit 115, the control unit 106 recognizes the touched coordinates and changes the overlapping volume of the object N1 with respect to the reference object. When the user moves the finger to the right, the overlapping volume between the reference object and the object N1 decreases, and when the user moves the finger to the left, the overlapping volume between the reference object and the object N1 increases. The overlapping volume setting range is limited such that at least a part of the object selected by the user overlaps the reference object.

If the control unit 106 does not update the overlapping volume in S522, the process ends. As shown in FIG. 8B, the reference OBJ corresponding to the movement of the subject and the objects A3, A4, M3, and M4 are displayed on the screen of the liquid crystal panel 114. When printing with a 3D printer, the control unit 106 acquires the combined OBJ data from the memory 109 and transmits the data from the data transmission unit 110 to the 3D printer. The 3D printer generates a group of three-dimensional objects shown in FIG.
In the present embodiment, the amount of overlap between objects can be arbitrarily set according to designation by a user operation, so that convenience for the user is improved.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

102 3D data acquisition unit 103 Subject object acquisition unit 104 Overlap volume calculation unit 105 Overlap volume setting unit 112 Object composition unit

Claims (8)

First acquisition means for acquiring three-dimensional data;
Second acquisition means for acquiring a plurality of objects related to the subject from the three-dimensional data;
A calculation means for calculating an overlapping amount of the area where the plurality of acquired objects overlap;
A data processing apparatus comprising: a combining unit that combines the plurality of objects that overlap each other when the amount of overlap calculated by the calculating unit is not zero.   The synthesizing unit includes: a first object of the plurality of objects; and a second object that is an object of a frame different from the first object and has the minimum overlap amount calculated by the calculation unit. The data processing apparatus according to claim 1, wherein the data processing apparatus combines the data. A setting unit configured to set an overlapping amount of the plurality of objects by a user operation;
2. The composition means, when the calculation result of the overlap amount by the calculation means is the overlap amount set by the setting means, combines the plurality of objects overlapping each other. Or the data processing apparatus of 2.   The data processing apparatus according to claim 1, wherein the calculation unit calculates a volume as the overlap amount. Display means for displaying images of the plurality of objects;
5. The data processing according to claim 1, wherein the display unit distinguishes and displays a region where the plurality of objects overlap and a region where the plurality of objects do not overlap. 6. apparatus.   The data processing apparatus according to claim 5, wherein the display unit displays an area in which the plurality of objects overlap with color. A data processing method executed by a data processing apparatus for processing three-dimensional data,
Obtaining a plurality of objects related to the subject from the three-dimensional data;
Calculating an overlap amount of an area where the plurality of objects overlap;
Determining whether the amount of overlap is zero; and
A data processing method comprising: combining the plurality of objects overlapping each other when it is determined that the overlap amount is not zero. A program for causing a computer of the data processing apparatus to execute each step according to claim 7.

Images