JP7229822B2 - Display image data acquisition device, display image data acquisition method, and display image data acquisition program - Google Patents

Description

The present invention relates to a technique for acquiring display image data of a computer.

With the widespread use of computers such as PCs (Personal Computers), frauds using computers are also increasing. Therefore, it has become necessary to record display image data of a computer in addition to user monitoring by a monitoring camera.

As in Patent Document 1, it is conceivable to monitor the usage status of a computer by taking a log of the operation details of the computer. However, although the details of computer operations may be recorded in the log, there are also detailed operations that are not recorded in the log. Other times, the computer crashes and it's not clear what was going on. Therefore, it is effective to monitor or record the display image data of the computer.

Japanese Patent Application Laid-Open No. 2009-032078

If display image data is constantly recorded while the computer is in operation, the amount of data increases. Therefore, it is difficult to store display image data for a long period of time. On the other hand, if the data amount of the display image data is reduced recklessly, it may happen that necessary confirmation cannot be performed when confirming the display image data.
SUMMARY OF THE INVENTION An object of the present invention is to enable appropriate confirmation while reducing the amount of data when recording display image data while a computer is in operation.

A display image data acquisition device according to the present invention includes:
a content identification unit that identifies display content, which is the content displayed on the screen by the target device;
a quality determining unit that determines the quality of image data to be acquired according to the display content specified by the content specifying unit;
a data acquisition unit that acquires display image data, which is image data displayed on a screen by the target device, according to the quality determined by the quality determination unit;

The content specifying unit specifies an active window, which is an active window, as the display content,
The quality determining unit determines the quality according to which application the active window is.

The quality determining unit refers to application information storing quality for each application, and determines the quality by specifying the quality corresponding to the application serving as the active window.

The content specifying unit specifies a displayed character string as the display content,
The quality determination unit determines the quality according to a keyword included in the character string.

The quality determination unit determines the quality by referring to keyword information in which quality is stored for each keyword and specifying the quality corresponding to the keyword included in the character string.

The quality determination unit determines the quality according to at least one of the day of the week and the time.

The quality determination unit refers to schedule information storing quality for at least one of the day of the week and time, and specifies the quality corresponding to at least one of the current day of the week and the current time. determine quality.

The quality is at least one of a rate indicating the number of acquisitions of the display image data per unit time and a compression ratio.

A display image data acquisition method according to the present invention includes:
a content specifying unit specifying display content, which is the content displayed on the screen by the target device;
A quality determination unit determines the quality of image data to be acquired according to the specified display content,
A data acquisition unit acquires display image data, which is image data displayed on a screen by the target device, according to the determined quality.

A display image data acquisition program according to the present invention comprises:
content identification processing for identifying display content, which is the content displayed on the screen by the target device;
a quality determination process for determining the quality of image data to be acquired according to the display content specified by the content specification process;
A computer is caused to function as a display image data acquisition device that performs a data acquisition process for acquiring display image data, which is image data to be displayed on a screen by the target device, according to the quality determined by the quality determination process.

In this invention, quality is determined according to display contents, and display image data is acquired according to the determined quality. Therefore, it is possible to obtain high-quality display image data for important display contents, and to obtain low-quality display image data for unimportant display contents. As a result, it is possible to reduce the amount of data when the display image data is recorded while the computer is running, and to enable appropriate confirmation.

1 is a configuration diagram of a display image data acquisition device 10 according to Embodiment 1. FIG. 4 is an explanatory diagram of application information 31 according to the first embodiment; FIG. 4 is an explanatory diagram of keyword information 32 according to the first embodiment; FIG. 4 is an explanatory diagram of schedule information 33 according to Embodiment 1. FIG. 4 is a flowchart showing the overall operation of the display image data acquisition device 10 according to Embodiment 1; 4 is a flowchart of quality determination processing according to the first embodiment; FIG. 4 is an explanatory diagram of a specific example of the operation of the display image data acquisition device 10 according to Embodiment 1; FIG. 11 is a configuration diagram of a display image data acquisition device 10 according to Modification 3; FIG. 11 is a diagram showing another configuration of the display image data acquisition device 10 according to Modification 3; FIG. 11 is a configuration diagram of a display image data acquisition device 10 according to Modification 4; FIG. 11 is a configuration diagram of a display image data acquisition device 10 according to Modification 6;

Embodiment 1.
*** Configuration description ***
A configuration of a display image data acquisition device 10 according to Embodiment 1 will be described with reference to FIG.
The display image data acquisition device 10 is a computer.
The display image data acquisition device 10 acquires display image data, which is image data displayed on the screen by the target device 51 , and writes it in the data recording device 52 .

The display image data acquisition device 10 includes hardware including a processor 11 , a memory 12 , a storage 13 , and a communication interface 14 . The processor 11 is connected to other hardware via signal lines and controls these other hardware.

The processor 11 is an IC (Integrated Circuit) that performs processing. Specific examples of the processor 11 are a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit).

The memory 12 is a storage device that temporarily stores data. Specific examples of the memory 12 include SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory).

The storage 13 is a storage device that stores data. A specific example of the storage 13 is an HDD (Hard Disk Drive). In addition, the storage 13 is SD (registered trademark, Secure Digital) memory card, CF (Compact Flash, registered trademark), NAND flash, flexible disk, optical disk, compact disk, Blu-ray (registered trademark) disk, DVD (Digital Versatile Disk), etc. It may be a portable recording medium.

The communication interface 14 is an interface for communicating with external devices such as the target device 51 and the data recording device 52 . Specific examples of the communication interface 14 are Ethernet (registered trademark), USB (Universal Serial Bus), and HDMI (registered trademark, High-Definition Multimedia Interface) ports.

The display image data acquisition device 10 includes a content identification unit 21, a quality determination unit 22, and a data acquisition unit 23 as functional components. The function of each functional component of the display image data acquisition device 10 is implemented by software.
The storage 13 stores a program that implements the function of each functional component of the display image data acquisition device 10 . This program is read into the memory 12 by the processor 11 and executed by the processor 11 . Thereby, the function of each functional component of the display image data acquisition device 10 is realized.

The storage 13 stores application information 31 , keyword information 32 and schedule information 33 .

Only one processor 11 is shown in FIG. However, there may be a plurality of processors 11, and the plurality of processors 11 may cooperate to execute programs that implement each function.

***Description of operation***
The operation of the display image data acquisition device 10 according to the first embodiment will be described with reference to FIGS. 2 to 7. FIG.
The operation of the display image data acquisition device 10 according to the first embodiment corresponds to the display image data acquisition method according to the first embodiment. Also, the operation of the display image data acquisition device 10 according to the first embodiment corresponds to the processing of the display image data acquisition program according to the first embodiment.

Pre-setting that is a prerequisite for the operation of the display image data acquisition device 10 according to the first embodiment will be described. In the presetting, application information 31, keyword information 32, and schedule information 33 are set by the user.

The application information 31 according to the first embodiment will be described with reference to FIG.
The application information 31 is information that stores quality for each application.
An application includes an application name and a path. An application name is a name for identifying an application. The path indicates the location where the executable file of the application is stored.
The quality is at least one of a rate indicating the number of acquisitions per unit time (here, one second) and a compression ratio. In Embodiment 1, quality is rate and compression ratio. In the first embodiment, the display image data is converted into JPEG format and stored, and the compression ratio is the quality specified by the argument when converting into JPEG format. Note that the compression rate is not limited to the quality specified by the argument when converting to the JPEG format, and may be, for example, the resolution of the display image data.

A higher quality is set in the application information 31 for an application that is more likely to be fraudulent. In FIG. 2, high quality is set for applications such as a customer management ledger and order management, which are recognized as having a high possibility of illegal acquisition of information. High quality means a rate with a large number of acquisitions per unit time and a low compression rate.
As for the rate, by increasing the number of acquisitions per unit time, the flow of operations can be checked in detail, and by lowering the compression rate, the data becomes clearer and even fine characters can be checked. The meanings of rate and compression ratio are different.

Keyword information 32 according to the first embodiment will be described with reference to FIG.
The keyword information 32 is information in which quality is stored for each keyword. The quality is the same as the quality in application information 31 .
A keyword is a string that specifies a particular quality. In FIG. 3, character strings such as confidential, strictly confidential, sales, profit and loss, and contact network are set as keywords. This is because document data containing these character strings is highly likely to be illegally obtained.

The schedule information 33 according to Embodiment 1 will be described with reference to FIG.
The schedule information 33 is information in which quality is stored for at least one of the day of the week and the time. The quality is the same as the quality in application information 31 .
The day of the week and time of day are the days of the week and time of day that specify special qualities. In FIG. 4, days of the week and times such as Sunday (holiday) and nighttime are set. This is because there are few people in the office on these days and times, and it is recognized that there is a high possibility that information will be obtained illegally.

The overall operation of the display image data acquisition device 10 according to the first embodiment will be described with reference to FIG.
The display image data acquisition device 10 periodically executes the process of FIG. 5, for example, every 3 seconds.

(Step S1: content specifying process)
The content specifying unit 21 specifies the display content, which is the content displayed on the screen by the target device 51 . For example, the content identification unit 21 identifies display content by communicating with an agent program installed in the target device 51 .
Specifically, the content specifying unit 21 specifies, as the display content, the application of the active window that is the window that is active, that is, is the window to be operated. At this time, the content specifying unit 21 specifies the application of the active window by specifying the path of the execution file of the application that is the active window from the handle of the active window.
Further, the content specifying unit 21 specifies the displayed character string as the display content. At this time, the content specifying unit 21 acquires a captured image of the screen of the target device 51 as display image data, extracts the character string from the display image data using a character recognition technique, and converts the displayed character string into Identify.

(Step S2: quality determination processing)
The quality determination unit 22 determines the quality of the image data to be acquired according to the display content specified in step S1. In Embodiment 1, the quality determination unit 22 determines the rate and compression rate as quality. Details of the quality determination process will be described later.

(Step S3: Data Acquisition Processing)
The data acquisition unit 23 acquires display image data, which is image data displayed on the screen by the target device 51 , and writes the acquired display image data to the data recording device 52 according to the quality determined in step S<b>2 .
In the first embodiment, the data acquisition unit 23 acquires the display image data of the acquisition number indicated by the rate determined in step S2 per unit time. At this time, the data acquisition unit 23 writes the display image data to the data recording device 52 after compressing the display image data at the compression ratio determined in step S2. That is, the data acquisition unit 23 designates the quality indicating the compression rate determined in step S2 as an argument, converts the display image data into JPEG, and writes the JPEG-converted display image data to the data recording device 52. . The data acquisition unit 23 writes the display content specified in step S1, the day of the week, and the time into the data recording device 52 together with the JPEG-converted display image data.

The quality determination process (step S2 in FIG. 5) according to the first embodiment will be described with reference to FIG.
(Step S21: Application determination process)
The quality determining unit 22 determines quality depending on which application's window is the active window.
Specifically, the quality determination unit 22 refers to the application information 31 and determines the quality by specifying the quality corresponding to the path of the execution file of the application of the active window specified in step S1.

(Step S22: Keyword determination processing)
The quality determination unit 22 determines quality according to the keyword included in the character string.
Specifically, the quality determining unit 22 refers to the keyword information 32 and determines the quality by specifying the quality corresponding to the keyword included in the character string specified in step S1. If the keyword included in the character string specified in step S1 is not set in the keyword information 32, the quality determination unit 22 determines the standard value as the quality.
At this time, if the identified character string includes a plurality of keywords, the quality determination unit 22 identifies the highest quality among the qualities corresponding to the plurality of keywords. In Embodiment 1, quality is rate and compression ratio. Therefore, the highest quality is specified for each rate and compression ratio. That is, the quality determination unit 22 specifies the rate with the highest number of acquisitions per unit time among the rates corresponding to the multiple keywords, and specifies the lowest compression rate among the compression rates corresponding to the multiple keywords. .

(Step S23: Schedule determination process)
The quality determining unit 22 refers to the schedule information 33 and determines the quality by specifying the quality corresponding to at least one of the current day of the week and the current time. If a schedule corresponding to at least one of the current day of the week and the current time is not set in the schedule information 33, the quality determination unit 22 determines the standard value as the quality.

(Step S24: comprehensive judgment processing)
The quality determination unit 22 adopts the highest quality among the qualities determined in steps S21 to S23.
In Embodiment 1, quality is rate and compression ratio. Therefore, the quality determination unit 22 adopts the rate with the largest number of acquisitions per unit time among the rates determined in steps S21 to S23. Also, the quality determination unit 22 adopts the lowest compression rate among the compression rates determined in steps S21 to S23.

A specific example of the operation of the display image data acquisition device 10 according to the first embodiment will be described with reference to FIG.
In the target device 51, it is assumed that the application of the active window is a customer management ledger, and character strings such as general, international, overseas, economy, and confidential are displayed. Also assume that the current time is 9:30 on Tuesday.
Then, in step S1 of FIG. 5, the content specifying unit 21 specifies that the application of the active window is a customer management ledger, and that character strings such as General, International, Overseas, Economy, and Confidential are displayed. be. In step S21 of FIG. 6, rate "5" and JPEG quality "60", which is a compression rate, are specified as the quality corresponding to the customer management ledger. Further, in step S22 of FIG. 6, the rate "7" corresponding to internal confidentiality and the JPEG quality "50", which is the compression ratio, are specified. Further, in step S23 of FIG. 6, the rate "2" corresponding to 9:30 on Tuesday and the JPEG quality "40", which is the compression rate, are specified. As a result, in step S24 of FIG. 6, rate "7" and JPEG quality "60" are adopted.
Then, in step S3 of FIG. 5, the data acquisition unit 23 acquires the display image data seven times per second according to the rate "7". Then, the data acquisition unit 23 converts the display image data into a JPEG image according to the JPEG quality of “60” and writes it to the data recording device 52 .

*** Effect of Embodiment 1 ***
As described above, the display image data acquisition apparatus 10 according to the first embodiment determines the quality according to the display content and the day of the week/time, and acquires the display image data according to the determined quality. Therefore, it is possible to obtain display image data with high quality for important display contents or days of the week/times where there is a high possibility of unauthorized information acquisition, etc., while obtaining display image data with low quality for unimportant display contents and days of the week/times. It becomes possible to obtain display image data with high quality.
As a result, it is possible to reduce the amount of data when the display image data is recorded while the computer is running, and to enable appropriate confirmation. That is, by recording important display image data in a smooth and clear state, detailed confirmation is possible. On the other hand, the amount of data is reduced by recording unimportant display image data in a somewhat rough and unclear state.

In addition, the display image data acquisition device 10 according to Embodiment 1 stores the display content, the day of the week and the time together with the display image data. Therefore, it is possible to narrow down the display image data that needs to be confirmed based on the display content, the day of the week, and the time. As a result, it is possible to check the display image data efficiently.

***Other Configurations***
<Modification 1>
In Embodiment 1, the display image data acquisition device 10 is provided outside the target device 51 . The display image data acquisition device 10 may be mounted inside the target device 51 . In this case, a program for realizing the function of each functional component of the display image data acquisition device 10 is executed by the processor of the target device 51 to realize the function of each functional component.

<Modification 2>
In Embodiment 1, the display image data acquisition device 10 changes the quality of the entire display image data according to the display content, the day of the week and the time. The display image data acquisition device 10 may change the quality of each part of the display image data according to the display content, the day of the week and the time.
For example, if the application of the active window is a specific application, the quality of the active window may be high while the quality of other parts may be low. As a specific example, the active window may be rate "7" and the other portions may be rate "0". A rate of "0" means that no display image data is acquired.

<Modification 3>
In the first embodiment, the display image data acquisition device 10 acquires display image data and stores it in the data recording device 52 . As shown in FIG. 8 , the display image data acquisition device 10 may acquire display image data, and also acquire image data by capturing an image of the surroundings of the target device 51 with a camera 53 . Then, the display image data acquisition device 10 may associate the display image data and the photographed data acquired at the same time and write them in the data recording device 52 .
By storing photographed data in association with display image data, it becomes possible to identify fraud that cannot be identified only by display image data. For example, while the target device 51 is displaying confidential information, the user may take a picture of the display device of the target device 51 to illegally obtain the confidential information. In this case, only the display image data can identify only that the confidential information was displayed. It becomes possible to specify.

Incidentally, as shown in FIG. 9, the photographed data may be written directly from the camera 53 to the data recording device 52 .

<Modification 4>
In the first embodiment, it has been described that the display image data is stored in the data recording device 52 . As shown in FIG. 10, the display image data stored in the data recording device 52 may be transmitted in real time to a monitoring terminal 54 equipped with a viewer and displayed. Moreover, the display image data stored in the data recording device 52 may be transmitted to the monitoring terminal 54 and displayed according to a request from the monitoring terminal 54 . It is conceivable that the monitoring terminal 54 designates the display image data to be acquired, for example, according to the day of the week or the time, and causes the data recording device 52 to transmit the data.

<Modification 5>
In Embodiment 1, there is only one target device 51 . A plurality of target devices 51 may be provided. In this case, the display image data acquisition device 10 stores application information 31 , keyword information 32 and schedule information 33 for each target device 51 . If the same settings are to be made for two or more target devices 51, the application information 31, the keyword information 32, and the schedule information 33 may be copied and set.

<Modification 6>
In Embodiment 1, each functional component is realized by software. However, as Modified Example 6, each functional component may be implemented by hardware. Regarding this modification 6, the differences from the first embodiment will be described.

The configuration of the display image data acquisition device 10 according to Modification 6 will be described with reference to FIG. 11 .
When each functional component is implemented by hardware, the display image data acquisition device 10 includes an electronic circuit 15 instead of the processor 11 , memory 12 and storage 13 . The electronic circuit 15 is a dedicated circuit that realizes the functions of each functional component, memory 12 and storage 13 .

The electronic circuit 15 includes a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field-Programmable Gate Array). is assumed.
Each functional component may be implemented by one electronic circuit 15, or each functional component may be implemented by being distributed among a plurality of electronic circuits 15. FIG.

<Modification 7>
As a modified example 7, some functional components may be implemented by hardware, and other functional components may be implemented by software.

The processor 11, the memory 12, the storage 13 and the electronic circuit 15 are called a processing circuit. That is, the function of each functional component is realized by the processing circuit.

The embodiments and modifications of the present invention have been described above. Some of these embodiments and modifications may be combined and implemented. Also, any one or some may be partially implemented. It should be noted that the present invention is not limited to the above embodiments and modifications, and various modifications are possible as required.

10 display image data acquisition device 11 processor 12 memory 13 storage 14 communication interface 15 electronic circuit 21 content identification unit 22 quality determination unit 23 data acquisition unit 31 application information 32 keyword information 33 schedule information , 51 target device, 52 data recording device, 53 camera, 54 monitoring terminal.

Claims (10)

a content identification unit that identifies display content, which is the content displayed on the screen by the target device;
a quality determining unit that determines the quality of image data to be acquired according to the display content specified by the content specifying unit;
a data acquisition unit that acquires display image data, which is image data displayed on a screen by the target device, according to the quality determined by the quality determination unit ;
The content specifying unit specifies an active window, which is an active window, as the display content,
the quality determining unit, if the active window is a window of a specific application, sets the quality of the active window higher than the quality of portions other than the active window;
The data acquisition unit acquires display image data for the active window according to the quality of the active window, and acquires display image data for a portion other than the active window according to the quality for a portion other than the active window. Display image data acquisition device to acquire . 2. The display image data acquisition device according to claim 1, wherein the quality determining unit determines the quality according to which application the active window is. 3. The display according to claim 2, wherein the quality determination unit determines the quality by referring to application information storing quality for each application and specifying the quality corresponding to the application serving as the active window. Image data acquisition device. The content specifying unit specifies a displayed character string as the display content,
4. The display image data acquisition device according to any one of claims 1 to 3, wherein the quality determining unit determines the quality according to a keyword included in the character string. 5. The display image according to claim 4, wherein the quality determination unit determines the quality by referring to keyword information storing quality for each keyword and specifying the quality corresponding to the keyword included in the character string. data acquisition device. 6. The display image data acquisition device according to any one of claims 1 to 5, wherein the quality determination unit determines the quality according to at least one of a day of the week and a time. The quality determination unit refers to schedule information storing quality for at least one of the day of the week and time, and specifies the quality corresponding to at least one of the current day of the week and the current time. 7. The display image data acquisition device of claim 6, wherein quality is determined. 8. The display image data acquisition device according to any one of claims 1 to 7, wherein said quality is at least one of a rate indicating the number of acquisitions of said display image data per unit time and a compression rate. A content identification unit provided in the display image data acquisition device identifies display content, which is the content displayed on the screen by the target device,
A quality determination unit included in the display image data acquisition device determines the quality of image data to be acquired according to the specified display content,
a data acquisition unit included in the display image data acquisition device acquires display image data, which is image data displayed on a screen by the target device, according to the determined quality ;
The content specifying unit specifies an active window, which is an active window, as the display content,
the quality determining unit, if the active window is a window of a specific application, sets the quality of the active window higher than the quality of portions other than the active window;
The data acquisition unit acquires display image data for the active window according to the quality of the active window, and acquires display image data for a portion other than the active window according to the quality for a portion other than the active window. Display image data acquisition method to acquire . content identification processing for identifying display content, which is the content displayed on the screen by the target device;
a quality determination process for determining the quality of image data to be acquired according to the display content specified by the content specification process;
causing a computer to function as a display image data acquisition device that performs a data acquisition process for acquiring display image data, which is image data to be displayed on a screen by the target device, according to the quality determined by the quality determination process ;
In the content specifying process, an active window that is an active window is specified as the display content,
In the quality determination process, if the active window is a window of a specific application, the quality of the active window is set higher than the quality of portions other than the active window;
In the data acquisition process, display image data for the active window is acquired according to the quality of the active window, and display image data for the portion other than the active window is acquired according to the quality for the portion other than the active window. Display image data acquisition program to acquire .

Images