JP7685064B2 - Communication device and data communication method - Google Patents

Description

The present invention relates to a communication device and a data communication method for compressing and transmitting data.

Due to the spread of communication networks and the Internet, huge amounts of data are sent and received on a daily basis. In order to transmit data efficiently, it is common for the data to be transmitted to be compressed. For example, various compression methods are applied depending on the type of data, such as compression from BMP format to JPEG format for images, compression in color space from RGB format to YUV format for video images, and compression between frames such as H.264 and H.265.

Guilin Liu and 5 others, “Image Inpainting for Irregular Holes Using Partial Convolutions”, [online], December 2018, Internet <URL: https://arxiv.org/pdf/1804.07723.pdf> Norihiko Kawai and two others, “Image inpainting considering brightness change and spatial locality of textures”, [online], January 2008, Internet <URL: http://yokoya.naist.jp/paper/data/1029/visapp_cameraready.pdf>

In recent years, the age of IoT (Internet of Things), in which various phenomena are digitized, has progressed, and the amount of data being communicated is on the rise. Therefore, in order to reduce the amount of data being communicated, there is a demand to compress data as efficiently as possible.

The present invention has been made in consideration of the above-mentioned conventional circumstances, and aims to improve data compression efficiency and realize data communication with a low communication load.

In order to achieve the above object, a transmitting communication device according to one aspect of the present invention is configured as follows. That is, the transmitting communication device that performs data compression communication is characterized by having a function of masking a part of the data to be transmitted, a function of losslessly compressing or lossily compressing the masked data, and a function of transmitting the losslessly or lossily compressed data to a receiving communication device.

Here, the transmitting communication device may be configured to mask the data to be transmitted according to a predetermined mask pattern that specifies the position and size of the mask area.

In addition, the transmitting communication device can be configured to mask the data to be transmitted according to a mask pattern in which the position and size of the mask area are randomly determined.

In addition, the transmitting communication device may have a function of repairing the mask area of the masked data, and may be configured to re-mask the data to be transmitted according to a different mask pattern if the degree of restoration of the repaired mask area does not satisfy a specified standard.

The receiving communication device, which is another aspect of the present invention, is configured as follows. That is, the receiving communication device performs compressed data communication, and is characterized by having a function of receiving data transmitted from a transmitting communication device that transmits losslessly or lossily compressed data by masking a portion of the data to be transmitted, a function of decompressing the losslessly compressed received data, and a function of repairing a masked area of the decompressed data or a masked area of the lossily compressed received data.

Here, the receiving communication device can be configured to determine that a data portion of the data to be repaired in which a predetermined value is stored is a mask area.

The receiving communication device may also be configured to identify a mask area in the data to be repaired based on information indicating a mask pattern that is transmitted along with the received data.

A data communication method according to yet another aspect of the present invention is configured as follows. That is, in a data communication method for performing compressed data communication between a transmitting communication device and a receiving communication device, the transmitting communication device masks a portion of the data to be transmitted, compresses the masked data losslessly or lossily, and transmits the losslessly or lossily compressed data to the receiving communication device, and the receiving communication device receives the data transmitted from the transmitting communication device, and if the received data is losslessly compressed, decompresses the received data and restores the masked area of the decompressed data, and if the received data is lossily compressed, restores the masked area of the received data.

According to the present invention, it is possible to improve data compression efficiency and realize data communication with a low communication load.

1 is a diagram showing an overview of a data communication system using a communication device according to an embodiment of the present invention. 1 is a diagram illustrating an example of the configuration of a communication device according to an embodiment of the present invention; 3 is a diagram illustrating an example of the configuration of a data transmission/reception unit in the communication device of FIG. 2. 4 is a diagram illustrating an example of the configuration of a determination unit in the data transmission/reception unit in FIG. 3 . 1A and 1B are diagrams illustrating an improvement in compression efficiency by the compression method according to the present proposal.

An embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows an overview of a data communication system using a communication device according to an embodiment of the present invention. In the data communication system shown in FIG. 1, a plurality of communication devices 101 are connected via a network N. Each communication device 101 can transmit and receive data to and from each other. The communication device 101 is, for example, a computer equipped with hardware resources such as a processor and a memory, and is configured such that the processor executes a program related to each function according to the present invention. Examples of the form of the communication device 101 include a personal computer, a tablet, a smartphone, a wireless device, an analysis server, an IP camera, and an IoT sensor. Examples of data transmitted and received by the communication devices 101 include video, images, audio, text, and numerical values obtained from a sensor.

Figure 2 shows an example of the configuration of a communication device 101. The communication device 101 in the figure includes a data transmission/reception unit 201 that transmits and receives data to and from other communication devices 101, an analysis/processing unit 202 that analyzes and processes data within the communication device 101, and a storage device 203 in which input/output data for the data transmission/reception unit 201 and the analysis/processing unit 202 are stored. Here, the functions realized by the analysis/processing unit 202 are not particularly limited, and various functions such as a calculation function, an image acquisition function, an image display function, an alarm function, and a laser irradiation function are envisioned.

Figure 3 shows an example of the configuration of the data transmission/reception unit 201. The data transmission/reception unit 201 has a transmission function for transmitting data to a receiving communication device, and a reception function for receiving data from a transmitting communication device. The transmission function is realized using a reading unit 301, a masking unit 302, a determination unit 303, a compression unit 304, a transmission unit 305, and a repair model 310. The reception function is realized using a reception unit 306, a decompression unit 307, a repair unit 308, a writing unit 309, and a repair model 310.

First, the transmission function of the data transmission/reception unit 201 of the communication device 101 will be described.
Data to be transmitted is stored in the storage device 203. The reading unit 301 reads the data to be transmitted from the storage device 203 and outputs it to the masking unit 302. The masking unit 302 performs a masking process to mask the data (data to be transmitted) input from the reading unit 301. In the masking process, a part of the input data is missing by replacing a value of the input data with a predetermined value (for example, 0). The mask area in the input data is not limited to one location, and may be multiple locations.

In this example, the mask unit 302 applies a mask to the input data according to a predetermined mask pattern in which the position and size of the mask area are specified. Other elements of the mask pattern may also be specified depending on the data format. For example, the shape of the mask area may also be specified for the mask pattern for image data. The mask unit 302 selects one of a plurality of mask patterns prepared in advance and uses it for masking the input data. In addition, if the mask pattern used is not appropriate, the mask unit 302 can re-mask the input data using a different mask pattern. Note that a mask pattern in which the position and size of the mask area are randomly determined may be generated during mask processing without preparing a mask pattern in advance.

The determination unit 303 determines whether the original data before masking can be restored to a practically useful level from the data masked by the mask unit 302. FIG. 4 shows an example of the configuration of the determination unit 303. The determination unit 303 in this example has a repair unit 401 and a comparison unit 402. The repair unit 401 performs a repair process on the masked data to repair the masked area in the same manner as the repair unit 308 described below (i.e., using the same repair model 310 as on the receiving side).

The comparison unit 402 compares the restored data with the original data and calculates an index value indicating the comparison result, for example, the signal-to-noise ratio or similarity of only the masked area. The closer the signal-to-noise ratio or similarity is to 0, the more similar the compared objects are to each other, and the farther away from 0, the more dissimilar the compared objects are to each other.

In the case of image data, the SNR can be expressed as an index such as SNR (Signal to Noise Ratio) or PSNR (Peak Signal to Noise Ratio). The similarity can be expressed as an index such as SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), NCC (Normalized Cross-Correlation), ZNCC (Zero-means Normalized Cross-Correlation), or SSIM (Structural Similarity).

The comparison unit 402 calculates an index value indicating the comparison result between the restored data and the original data, and then compares the index value with a preset threshold value. The threshold value is set in advance, for example, according to the reliability required for the data to be transmitted. The threshold value may be set to a different value depending on the data format. If the index value is equal to or less than the threshold value, the comparison unit 402 determines that the original data can be restored, and notifies the compression unit 304 of that effect. If the calculated index value exceeds the threshold value, the comparison unit 402 determines that the original data cannot be restored. In this case, the comparison unit 402 outputs a mask re-execution instruction to the mask unit 302 to re-mask the input data using a different mask pattern. If it is not determined that the original data can be restored even after performing mask re-execution a predetermined number of times, the comparison unit 402 notifies the compression unit 304 that the original data cannot be restored.

The compression unit 304 compresses the data masked by the mask unit 302 or the original data before masking using a predetermined compression method (e.g., Huffman coding, run-length compression, etc.). Whether the masked data or the original data before masking is compressed is determined based on the judgment result by the comparison unit 402. That is, if it is judged that the original data can be restored from the masked data, the compression unit 304 compresses the masked data. On the other hand, if it is judged that the original data cannot be restored from the masked data, the compression unit 304 compresses the original data before masking. The compression method is selected according to the data format. For example, in the case of image data in BMP format, a compression method from BMP format to JPEG format or PNG format is selected. The compression method may be a lossless compression method or a lossy compression method.

The transmitting unit 305 transmits the data compressed by the compressing unit 304 to the receiving communication device. As described above, the masked data is transmitted and received only when it is determined that the original data can be substantially restored from the masked data, thereby ensuring the reliability of the data. If necessary, the transmitting unit 305 may additionally transmit mask information indicating the position and size of the mask (mask information indicating "no mask area" when transmitting the original data without masking). By transmitting such mask information, the receiving communication device can easily identify the masked portion in the received data.

Next, the receiving function of the data transmitting/receiving unit 201 of the communication device 101 will be described.
The receiving unit 306 receives data (and mask information) transmitted from the transmitting communication device. The decompression unit 307 decompresses the received data when the data received by the receiving unit 306 is losslessly compressed. The restoration unit 308 receives the data decompressed by the decompression unit 307 or the lossy compressed received data. The restoration unit 308 performs restoration processing to restore the mask area of the input data. That is, when the data received by the receiving unit 306 is losslessly compressed, the mask area of the data obtained by decompressing the received data is restored. On the other hand, when the data received by the receiving unit 306 is lossy compressed, the mask area of the received data is restored. The writing unit 309 writes the data restored by the restoration unit 308 to the storage device 203.

Here, the restoration process by the restoration unit 308 can be performed using any restoration technique according to the data format. For example, when restoring image data, the Inpainting method can be used. A representative Inpainting method is a machine learning method using a restoration model such as that in Non-Patent Document 1, but a method not based on machine learning such as that in Non-Patent Document 2 may also be used. The restoration unit 308 in this example uses a restoration model 310 prepared in advance to perform restoration of a mask area on the input data.

If mask information is attached to the received data, the position and size of the mask area in the input data can be identified based on this mask information. If mask information is not attached to the received data, a data portion of the received data in which a predetermined value (e.g., 0) is stored may be determined to be the mask area.

Next, the improvement in compression efficiency achieved by the compression method proposed herein will be described with reference to Fig. 5. In conventional compression methods, image data 501 in BMP format is converted (compressed) into image data 502 in JPEG or PNG format without masking. On the other hand, in the compression method proposed herein, masking is performed on image data 501 in BMP format, and the masked image data 503 is converted (compressed) into image data 504 in JPEG or PNG format.

The data capacity of the original image data 501 and the masked image data 503 is both 768 KB. The data capacity of the image data 502 after compression using the conventional compression method is 100 KB for JPEG format and 674 KB for PNG format. The data capacity of the image data 504 after compression using the compression method proposed in this proposal is 91.7 KB for JPEG format and 611 KB for PNG format. Thus, according to this proposal, the data capacity is smaller than before and the compression efficiency is improved. This is because, focusing on the fact that the compression efficiency improves as the same pattern appears more frequently in data, by masking part of the data with the same value (e.g., 0), the entire masked area can be compressed as the same pattern.

As described above, in the data communication system of this example, the transmitting communication device has a masking unit 302 that masks a portion of the data to be transmitted, a compression unit 304 that performs lossless or lossy compression on the masked data, and a transmitting unit 305 that transmits the lossless or lossy compressed data to the receiving communication device, and the receiving communication device has a receiving unit 306 that receives data transmitted from the transmitting communication device, a decompression unit 307 that decompresses the losslessly compressed received data, and a restoration unit 308 that restores the masked area of the decompressed data or the masked area of the lossy compressed received data. With this configuration, by masking and compressing a portion of the data to be transmitted, the compression efficiency of the communication data can be improved, making it possible to realize data communication with a small communication load.

In addition, in the data communication system of this example, the transmitting communication device has a repair unit 401 that repairs the masked area of the masked data, and is configured to re-mask the data to be transmitted according to a different mask pattern if the degree of restoration of the repaired masked area does not satisfy a predetermined standard. In this way, by configuring the masked data to be transmitted and received only when it is determined that the original data can be substantially restored from the masked data, it is possible to ensure the reliability of the data.

Here, in the above explanation, image data has been used as an example, but the present invention can also be applied to the transmission and reception of other types of data, such as video, audio, text, and numerical values obtained from a sensor. Also, in the above explanation, the communication device 101 has a function as a transmitting communication device according to the present invention and a function as a receiving communication device according to the present invention, but these may be configured as separate devices.

Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention and its equivalents described in the claims.

Furthermore, the present invention can be provided not only as devices such as those described above or as systems composed of these devices, but also as methods executed by these devices, programs for causing a processor to realize the functions of these devices, and storage media for storing such programs in a computer-readable format.

The present invention can be used in communication devices that compress and transmit/receive data.

101: Communication device, 201: Data transmission/reception unit, 202: Analysis/processing unit, 203: Storage device, 301: Reading unit, 302: Masking unit, 303: Determination unit, 304: Compression unit, 305: Transmission unit, 306: Reception unit, 307: Decompression unit, 308: Restoration unit, 309: Writing unit, 310: Restoration model, 401: Restoration unit, 402: Comparison unit

Claims (4)

In a transmitting communication device that performs compressed data communication,
A function to mask a part of the data to be transmitted according to a mask pattern;
repairing a masked area of the masked data;
a function of determining whether the degree of restoration of the repaired mask region satisfies a predetermined criterion;
A function of losslessly compressing or lossily compressing the masked data or the data to be transmitted before being masked;
a function of transmitting the losslessly compressed or losslessly compressed data to a receiving communication device;
if it is determined that the degree of restoration does not satisfy the criterion, re-executing a process of masking the data to be transmitted according to a different mask pattern and repairing the masked area;
If it is determined that the degree of restoration satisfies the criterion, the data masked with the mask pattern is losslessly or lossily compressed and transmitted to the receiving communication device;
A communication device characterized in that if the degree of restoration is not determined to satisfy the criterion even after repeating the re-execution a predetermined number of times, the data to be transmitted before being masked is losslessly or lossily compressed and transmitted to the receiving communication device. 2. The communication device according to claim 1,
A communication device which masks the data to be transmitted in accordance with a predetermined mask pattern that defines the position and size of a mask area. 2. The communication device according to claim 1,
A communication device which masks the data to be transmitted according to a mask pattern in which the position and size of a mask area are randomly determined. A data communication method for compressing and communicating data between a transmitting communication device and a receiving communication device, comprising:
The transmitting communication device,
A part of the data to be transmitted is masked according to a mask pattern,
Inpainting a masked region of the masked data;
determining whether the degree of restoration of the repaired mask region satisfies a predetermined criterion;
if it is determined that the degree of restoration does not satisfy the criterion, re-executing a process of masking the data to be transmitted according to a different mask pattern and repairing the masked area;
When it is determined that the degree of restoration satisfies the criterion, the data masked with the mask pattern is losslessly compressed or lossily compressed;
When it is not determined that the degree of restoration satisfies the criterion even after the re-execution is repeated a predetermined number of times, the data to be transmitted is losslessly compressed or lossily compressed before being masked;
Transmitting the losslessly compressed or losslessly compressed data to a receiving communication device;
The receiving communication device,
receiving data transmitted from the transmitting communication device;
If the received data is losslessly compressed, decompressing the received data and repairing a masked area of the decompressed data;
A data communication method, comprising the steps of: if the received data has been lossy compressed, restoring a masked area of the received data without decompressing the received data.

Images