JP6517609B2 - Encoding method and encoding program - Google Patents

Description

  The present invention relates to an encoding method and an encoding program.

  In recent years, with the spread of broadband services, it has become easier to view moving image content via a network. It is also said that more than half of the data on the network becomes moving image data because it has become easier to view moving image content.

  In addition, there is an image service which gives the viewer a high sense of reality by displaying the whole sky image so as to have a viewing angle of 360 degrees. The moving image includes a plurality of frames (still images). In the whole sky image, the frames are displayed adjacent to each other so as to have spatial continuity. That is, in the full sky image, the end of the frame is displayed adjacent to the end of the other frame. For example, in the full sky image, the right end of the frame is displayed adjacent to the left end of another frame. For example, in the full sky image, the upper end of a frame is displayed adjacent to the lower end of another frame.

  In addition, in H.263, which is one of moving image compression standards, non-restricted motion compensation technology such as out-of-screen reference is adopted (see Non-Patent Document 1).

ITU-T H.263 (ANNEX D)

  The conventional encoding device has a problem that it is not possible to improve the efficiency of image coding when the images are displayed adjacent to each other with spatial continuity.

  In view of the above circumstances, the present invention provides an encoding method and an encoding program that can improve the efficiency of image encoding when the images are displayed adjacent to each other with spatial continuity. The purpose is to

  One aspect of the present invention is a coding method in a coding device, which is determined in a row direction or a column direction in the image according to a step of acquiring an image to be coded and a predetermined element. Defining a score in a plurality of regions, and repositioning the region according to whether the region is relatively high in the image if the image is encoded into an intra frame, And D. encoding the image in which the region has been rearranged.

  One embodiment of the present invention is the encoding method described above, wherein, when the image is encoded in an inter frame, the areas of the ends facing each other in the image are defined as a motion search range, and the motion search And D. encoding the image based on the result.

  One aspect of the present invention is an encoding program for causing a computer to execute the above encoding method.

  According to the present invention, it is possible to improve the coding efficiency of frames of a moving image when the moving image is displayed adjacent to each other with spatial continuity.

It is a figure which shows the example of a structure of the encoding apparatus in 1st Embodiment. FIG. 7 is a diagram illustrating an example of rearrangement of block lines in the column direction in the first embodiment. It is a figure which shows the example of rearrangement of the block line of row direction in 1st Embodiment. It is a flowchart which shows the example of the procedure of pre-processing in 1st Embodiment. It is a flowchart which shows the example of the procedure of pre-processing in 2nd Embodiment. It is a flowchart which shows the example of the procedure of pre-processing in 3rd Embodiment. It is a flowchart which shows the example of the procedure of pre-processing in 4th Embodiment.

Embodiments of the present invention will be described in detail with reference to the drawings.
First Embodiment
FIG. 1 is a diagram showing an example of the configuration of the coding apparatus 10. As shown in FIG. The encoding device 10 includes a pre-processing unit 11, a storage unit 12, an encoding unit 13, and an output unit 14. Software that a processor such as a CPU (Central Processing Unit) functions by executing a program stored in a memory, for example, part or all of the preprocessing unit 11, the encoding unit 13, and the output unit 14 It is a functional unit. Further, some or all of these functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

  The preprocessing unit 11 acquires a frame (original image) which is an entire sky image and to be encoded. The image may be a still image or a moving image. The coding standard is not limited to a specific standard. For example, a still image may be encoded based on the JPEG (Joint Photographic Experts Group) standard. For example, moving pictures may be encoded based on the Moving Picture Experts Group (MPEG) standard or the H.264 standard.

  In the following, the preprocessing unit 11 acquires a moving image as an example. The moving image includes a plurality of frames (still images). In the whole sky image, the frames are displayed adjacent to each other so as to have spatial continuity. That is, in the full sky image, the end of the frame is displayed adjacent to the end of the other frame. The preprocessing unit 11 performs preprocessing on a moving image including a frame to be encoded. The pre-processing unit 11 transfers the pre-processed moving image to the encoding unit 13.

  The preprocessing unit 11 processes a frame to be encoded into a frame that is easy to encode by the encoding unit 13. A frame that is easy to encode is, for example, a frame with high image redundancy. The encoding unit 13 can improve the efficiency of encoding by performing encoding processing using image redundancy on a frame (original image).

  Hereinafter, a unit area of a frame to be encoded is referred to as a "block". Hereinafter, a plurality of blocks arranged in one direction in the frame will be referred to as "block lines".

  The preprocessing unit 11 defines a plurality of block lines in the row direction or the column direction in the frame. The preprocessing unit 11 may acquire a frame in which a plurality of block lines are predetermined in the row direction or the column direction. The preprocessing unit 11 determines a score based on the element of the image in the block line as the block line. An element of the image is, for example, the activity of the image. The activity is, for example, the complexity or flatness of the texture as a pixel value. For example, the preprocessing unit 11 sets a high score on the block line as the texture in the block line is more complicated. That is, as the texture in the block line is flatter, the preprocessing unit 11 sets a lower score on the block line.

  Further, the elements of the image are expressed, for example, by whether or not gradation exists in the image. For example, the pre-processing unit 11 determines, in the block line in which the gradation is present, a higher score than the block line in which the gradation is not present. Elements of the image are represented, for example, by luminance as a pixel value. For example, the pre-processing unit 11 sets a higher score on a block line in which a region with high luminance is present than in a block line in which a region having high luminance is not present.

  When the first block line, the second block line, and the third block line are lined up in this order in the frame, the pre-processing unit 11 is a second unit that satisfies all of the first to third conditions. Detect block lines. The preprocessing unit 11 may detect a second block line that satisfies at least one of the first condition to the third condition.

  The first condition is that the difference between the score of the first block line and the score of the second block line is equal to or less than a threshold. The second condition is that the difference between the score of the third block line and the score of the second block line is equal to or greater than the threshold. The third condition is that the difference between the score of the first block line and the score of the third block line is the largest in the frame as compared to the difference of the scores of other block lines.

  FIG. 2 is a diagram showing an example of rearrangement of block lines in the column direction in the first embodiment. In FIG. 2, the frame to be encoded is composed of block lines 1 a to 6 a in the column direction. In FIG. 2, as an example, the difference between the score of the block line 2a and the score of the block line 3a is equal to or less than the threshold. The difference between the score of block line 3a and the score of block line 4a is equal to or greater than the threshold. The difference between the score of the block line 2a and the score of the block line 4a is the largest in the frame as compared to the difference of the scores of other block lines.

  The preprocessing unit 11 separates the first block line having a small difference in score from the detected second block line from the detected second block line. In FIG. 2, the preprocessing unit 11 separates the block line 2 a from the block line 3 a. The preprocessing unit 11 may separate the block line group from the other block line group. In FIG. 2, when separating the block line 2a and the block line 3a, the preprocessing unit 11 separates the block lines 1a and 2a from the block lines 3a to 6a.

  The preprocessing unit 11 rearranges the block line group in the frame by joining the separated block lines to other block lines. The pre-processing unit 11 causes the block lines having high scores to be adjacent to each other by joining the block lines having high scores. In FIG. 2, the preprocessing unit 11 arranges the block line 1 a and the block lines 4 a to 6 a adjacent to each other except the end of the frame. The encoding unit 13 can improve the efficiency of encoding by performing encoding processing using image redundancy on frames in which block lines having high scores are adjacent to each other.

  The preprocessing unit 11 rearranges block lines with low scores at the end of the frame. The preprocessing unit 11 may rearrange a plurality of block lines with low scores at both ends of the frame. In FIG. 2, the preprocessing unit 11 arranges the block line 2 a at the right end of the frame. The preprocessing unit 11 arranges the block line 3a at the left end of the frame. In the full sky image, spatial redundancy is present between the leftmost block of the frame and the rightmost block of the other frames, so image redundancy is high. The encoding unit 13 can improve the encoding efficiency by performing encoding processing using image redundancy on a frame in which a plurality of block lines with low scores are arranged at both ends.

  FIG. 3 is a diagram showing an example of rearrangement of block lines in the row direction in the first embodiment. In FIG. 3, the frame to be encoded is composed of block lines 1b to 6b in the row direction. In FIG. 3, as an example, the difference between the score of the block line 4b and the score of the block line 5b is equal to or less than the threshold. The difference between the score of block line 5b and the score of block line 6b is equal to or greater than the threshold. The difference between the score of block line 4b and the score of block line 6b is the largest in the frame as compared to the difference of the scores of other block lines.

  The preprocessing unit 11 separates the first block line having a small difference in score from the detected second block line from the detected second block line. In FIG. 3, the preprocessing unit 11 separates the block line 4 b from the block line 5 b. The preprocessing unit 11 may separate the block line group from the other block line group. In FIG. 3, when separating the block line 4b and the block line 5b, the preprocessing unit 11 separates the block lines 1b to 4b and the block lines 5b and 6b.

  The preprocessing unit 11 rearranges the block line group in the frame by joining the separated block lines to other block lines. The pre-processing unit 11 causes the block lines having high scores to be adjacent to each other by joining the block lines having high scores. In FIG. 3, the preprocessing unit 11 arranges the block line 6 b and the block lines 1 b to 3 b adjacent to each other except the end of the frame. The encoding unit 13 can improve the efficiency of encoding by performing encoding processing using image redundancy on frames in which block lines having high scores are adjacent to each other.

  The preprocessing unit 11 rearranges block lines with low scores at the end of the frame. The preprocessing unit 11 may rearrange a plurality of block lines with low scores at both ends of the frame. In FIG. 3, the preprocessing unit 11 arranges the block line 1 b at the lower end of the frame. The preprocessing unit 11 arranges the block line 5b at the upper end of the frame. In the whole sky image, spatial redundancy exists between the block at the lower end of the frame and the block at the upper end of the other frames, so image redundancy is high. The encoding unit 13 improves encoding efficiency by performing encoding processing using image redundancy on frames in which a plurality of block lines with low scores are arranged at both ends (ends facing each other) It can be done.

  The storage unit 12 includes, for example, a non-volatile storage medium (non-transitory storage medium) such as a read only memory (ROM), a flash memory, or a hard disk drive (HDD). The storage unit 12 may have, for example, a volatile storage medium such as a random access memory (RAM) or a register. The storage unit 12 may store, for example, a program for causing a software function unit to function. The storage unit 12 may store, for example, parameters used for the encoding process. The storage unit 12 may store, for example, parameters used for preprocessing. The storage unit 12 may store a threshold.

  The encoding unit 13 acquires, from the preprocessing unit 11, a moving image including a plurality of frames subjected to preprocessing. The encoding unit 13 encodes a plurality of pre-processed frames. In the first embodiment, the encoding unit 13 encodes a frame by intra coding. The encoding unit 13 encodes a frame, for example, by executing an encoding program. The encoding unit 13 transmits the encoded frame to the output unit 14. The encoding unit 13 may store the encoded frame in the storage unit 12.

  The output unit 14 outputs the encoded frame to the display device. The display device decodes the encoded frame and displays a moving image including the decoded frame as a full sky image. That is, the output unit 14 outputs a moving image to the display device such that a plurality of frames are displayed adjacent to each other.

  FIG. 4 is a flowchart showing an example of the procedure of pre-processing. Steps S <b> 103 to S <b> 106 illustrated in FIG. 4 are performed for at least one of block lines in the column direction and block lines in the row direction.

  The preprocessing unit 11 acquires a frame (original image) to be encoded (step S101). The preprocessing unit 11 determines whether the acquired frame is a frame displayed adjacent to have spatial continuity. That is, the preprocessing unit 11 determines whether the acquired frame is a full sky image (step S102).

  If the acquired frame is not a full sky image (step S102: No), the preprocessing unit 11 proceeds to step S107. When the acquired frame is the whole sky image (step S102: Yes), the preprocessing unit 11 determines a score based on an element for each block line (step S103). The preprocessing unit 11 determines whether a block line satisfying the third condition from the first condition is detected (step S104).

  When the block line satisfying the first condition to the third condition is not detected (step S104: No), the preprocessing unit 11 proceeds the process to step S107. When a block line satisfying the first condition to the third condition is detected (step S104: Yes), the preprocessing unit 11 detects a block line for which a score having a small difference from the detected block line score is determined, and It separates from the block line that has been sent (step S105). The preprocessing unit 11 arranges at the end of the frame a block line in which a score having a small difference from the detected block line score is determined (step S106). The encoding unit 13 encodes the frame of the moving image transmitted from the preprocessing unit 11 by intra coding (step S107).

  The decoded frame may be subjected to processing for restoring the block line rearranged before encoding to the original arrangement before rearrangement.

  As described above, the encoding method according to the first embodiment is the encoding method in the encoding apparatus 10, which includes the steps of acquiring an image (frame) to be encoded and a predetermined element. And determining the score in a plurality of areas (block lines) defined in the row direction or the column direction in the image, and if the image is encoded into an intra frame, is the area having a relatively high score in the image? Re-arranging the region depending on whether or not to encode the image in which the region has been re-arranged.

  This enables the encoding device 10 of the first embodiment to improve the efficiency of image encoding when the images are displayed adjacent to each other with spatial continuity.

Second Embodiment
The second embodiment is different from the first embodiment in that the encoding unit 13 encodes a frame by inter encoding. In the second embodiment, only differences from the first embodiment will be described.

  In the whole sky image, the frames are displayed adjacent to each other so as to have spatial continuity. That is, in the full sky image, the end of the frame is displayed adjacent to the end of the other frame. The encoding unit 13 sets block lines at the ends facing each other in the frame as a motion search range. Also, the encoding unit 13 may set one of the end block lines facing each other in the frame as the motion search range. Further, the encoding unit 13 may set a range including one of the end block lines facing each other in the frame as the motion search range. In the second embodiment, the encoding unit 13 encodes a frame by inter coding.

  FIG. 5 is a flowchart showing an example of the procedure of pre-processing in the second embodiment. The preprocessing unit 11 acquires a frame (original image) to be encoded (step S201). The preprocessing unit 11 determines whether the acquired frame is a frame displayed adjacent to have spatial continuity. That is, the preprocessing unit 11 determines whether the acquired frame is a full sky image (step S202).

  When the acquired frame is not a full sky image (step S202: No), the preprocessing unit 11 proceeds with the process to step S204. If the acquired frame is a full sky image (step S202: Yes), the encoding unit 13 sets the block lines at the ends facing each other in the frame as the motion search range (step S203). The encoding unit 13 encodes the frame of the moving image transmitted from the preprocessing unit 11 by inter encoding (step S204).

  As described above, the encoding method according to the second embodiment is the encoding method in the encoding device 10, and when the image is encoded into the inter frame, the areas of the ends facing each other in the image (frame) (Block lines) are defined as the range of motion search, and the step of encoding the image based on the result of the motion search is included. The encoding unit 13 may set one of the end regions facing each other in the image as the motion search range. Further, the encoding unit 13 may set a range including one of the end regions facing each other in the image as the motion search range.

  By this, the coding apparatus 10 of the second embodiment can further improve the coding efficiency of the image when the images are displayed adjacent to each other with spatial continuity.

Third Embodiment
The third embodiment is different from the first embodiment in that the preprocessing unit 11 performs preprocessing when an object to be encoded is not an inter frame (predicted frame). In the third embodiment, only differences from the first embodiment will be described.

  The preprocessing unit 11 performs preprocessing on the intra frame and the inter frame subsequent to the intra frame. When the moving image stream has a Closed GOP (Group Of Pictures) structure, the preprocessing unit 11 performs preprocessing on a plurality of frames in units of GOPs. When the stream of moving images has an Open GOP (Group Of Pictures) structure, the preprocessing unit 11 performs preprocessing on a plurality of frames in scene units.

  The encoding unit 13 encodes the intra frame of the moving image subjected to the pre-processing by the pre-processing unit 11 by intra coding. The encoding unit 13 encodes the inter frame of the moving image not subjected to the preprocessing by the preprocessing unit 11 by inter coding.

  FIG. 6 is a flowchart showing an example of the procedure of pre-processing in the third embodiment. Steps S <b> 303 to S <b> 306 illustrated in FIG. 6 are performed for at least one of block lines in the column direction and block lines in the row direction.

  The preprocessing unit 11 acquires a frame (original image) to be encoded (step S301). The preprocessing unit 11 determines whether an object to be encoded is to be encoded into either an inter frame or an intra frame (step S302). When an object to be encoded is encoded into an intra frame (step S302: No), the preprocessing unit 11 determines a score based on an element for each block line (step S303). The preprocessing unit 11 determines whether a block line satisfying the third condition from the first condition is detected (step S304).

  When the block line satisfying the first condition to the third condition is not detected (step S304: No), the preprocessing unit 11 proceeds the process to step S308. When a block line satisfying the first condition to the third condition is detected (step S304: Yes), the preprocessing unit 11 detects a block line for which a score having a small difference from the detected block line score is determined, and The block line is separated from the block line (step S305). The preprocessing unit 11 arranges at the end of the frame a block line in which a score having a small difference from the detected block line score is determined (step S306).

  When the target to be encoded is encoded into the inter frame (step S302: Yes), the encoding unit 13 sets the block lines at the ends facing each other in the frame as the motion search range (step S307). The encoding unit 13 encodes the frame of the moving image transmitted from the pre-processing unit 11 by intra coding or inter coding (step S308).

  As described above, the encoding method according to the third embodiment is the encoding method in the encoding apparatus 10, which includes the steps of acquiring an image (frame) to be encoded and a predetermined element. And determining the score in a plurality of areas (block lines) defined in the row direction or the column direction in the image, and if the image is encoded into an intra frame, is the area having a relatively high score in the image? Re-arranging the region depending on whether or not to encode the image in which the region has been re-arranged. The encoding method of the third embodiment is the encoding method in the encoding device 10, and is the encoding method in the encoding device 10, and when the image is encoded into an inter frame, the image (frame) In the above, the end regions (block lines) facing each other are defined as the motion search range, and the image is encoded based on the result of the motion search. The encoding unit 13 may set one of the end regions facing each other in the image as the motion search range. Further, the encoding unit 13 may set a range including one of the end regions facing each other in the image as the motion search range.

  This enables the coding device 10 of the third embodiment to further improve the coding efficiency of the image when the images are displayed adjacent to each other with spatial continuity.

Fourth Embodiment
The fourth embodiment is different from the third embodiment in that the encoding unit 13 encodes an inter frame by intra coding or inter coding. In the fourth embodiment, only differences from the third embodiment will be described.

  FIG. 7 is a flowchart showing an example of the procedure of pre-processing in the fourth embodiment. Steps S <b> 403 to S <b> 406 illustrated in FIG. 7 are performed for at least one of block lines in the column direction and block lines in the row direction. In the third embodiment, the process of encoding an inter frame is subdivided.

  The preprocessing unit 11 acquires a frame (original image) to be encoded (step S401). The preprocessing unit 11 determines whether an object to be encoded is to be encoded into either an inter frame or an intra frame (step S402). When an object to be encoded is encoded into an intra frame (step S402: No), the preprocessing unit 11 determines a score based on an element for each block line (step S403). The preprocessing unit 11 determines whether a block line satisfying the third condition from the first condition is detected (step S404).

  When the block line satisfying the first condition to the third condition is not detected (step S404: No), the preprocessing unit 11 proceeds with the process to step S409. When a block line satisfying the first condition to the third condition is detected (step S404: Yes), the preprocessing unit 11 detects a block line for which a score having a small difference from the detected block line score is determined, and It separates from the block line that has been sent (step S405). The preprocessing unit 11 arranges at the end of the frame a block line in which a score having a small difference from the detected block line score is determined (step S406).

  When an object to be encoded is encoded into an inter frame (step S402: Yes), the encoding unit 13 determines whether a block to be encoded is a block to be inter encoded (step S407). If the block to be encoded is a block to be intra-coded (step S407: No), the encoding unit 13 proceeds with the process to step S409.

  If the block to be encoded is a block to be inter encoded (step S407: Yes), the encoding unit 13 sets block lines at the ends facing each other in the frame as the motion search range (step S408). ). The encoding unit 13 encodes the frame of the moving image transmitted from the pre-processing unit 11 by intra coding or inter coding (step S409).

  As described above, the encoding method of the fourth embodiment is the encoding method in the encoding apparatus 10, and when the block to be encoded is the block to be inter-encoded, the frames in the frame are mutually positive. It has a step which makes a block search line of an end opposite a motion search range.

  This enables the coding device 10 of the fourth embodiment to further improve the coding efficiency of the image when the images are displayed adjacent to each other with spatial continuity.

  At least a part of the encoding device in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer readable recording medium, and the program recorded in the recording medium may be read and executed by a computer system. Here, the “computer system” includes an OS and hardware such as peripheral devices. The term "computer-readable recording medium" refers to a storage medium such as a flexible disk, a magneto-optical disk, a ROM, a portable medium such as a ROM or a CD-ROM, or a hard disk built in a computer system. Furthermore, “computer-readable recording medium” dynamically holds a program for a short time, like a communication line in the case of transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include one that holds a program for a certain period of time, such as volatile memory in a computer system that becomes a server or a client in that case. Further, the program may be for realizing a part of the functions described above, or may be realized in combination with the program already recorded in the computer system. It may be realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1a ... Block line, 2a ... Block line, 3a ... Block line, 4a ... Block line, 1b ... Block line, 2b ... Block line, 3b ... Block line, 4b ... Block line, 5b ... Block line, DESCRIPTION OF SYMBOLS 10 ... Encoding device, 11 ... Pre-processing part, 12 ... Storage part, 13 ... Encoding part, 14 ... Output part

Claims (3)

A coding method in the coding device,
Obtaining an image to be encoded;
Defining, for each region defined in a row direction or a column direction in the image, the higher the texture of the region is complex, or the higher the score if gradation is present in the region;
If the image is encoded into an intra frame, rearranging the region and encoding the image with the rearranged region based on the score;
I have a,
In the encoding step, when the first area, the second area, and the third area in the image are arranged in this order, the score of the first area and the score of the second area And the difference between the score of the third area and the score of the second area are equal to or greater than a predetermined value as a second condition. From the first condition to the third condition, where the third condition is that the difference between the score of the third region and the score of the third region is the largest as compared to the difference of the scores of the other regions in the image. To detect the second region satisfying at least one of the above, and separating the detected second region and the first region to join the separated region group to another region group. An encoding method for rearranging the area . The image is an all-around image,
When the whole sky image is encoded into an inter frame, the areas at both ends of the frame of the whole sky image are set as a motion search range, and the image is coded based on the result of the motion search. When,
The encoding method according to claim 1 , further comprising: An encoding program for causing a computer to execute the encoding method according to any one of claims 1 to 2 .

Images