JP5689771B2 - Image management system, client terminal side program, server side program, and image management method - Google Patents

Description

  The present invention relates to a technique for managing an image acquired by photographing a document.

  2. Description of the Related Art Conventionally, there has been known a technique for obtaining an image by photographing a document required for business with a camera mounted on a terminal. The acquired image is stored in the storage unit of the terminal and transmitted to the server.

  The following technologies are disclosed as related technologies.

JP 2008-305292 A

  However, when the terminal keeps the acquired image, the image may be leaked due to loss of the terminal, theft, via a network, or the like. Further, there is a possibility that the image may be lost due to a failure or a disaster of the terminal.

  Further, when an image is transmitted from the terminal to the server, the image may be damaged due to a failure in the transmission process. Or, if the image stored in the terminal is erased by mistake before the valid image is stored in the server, it will be necessary to re-obtain the necessary documents and spend unnecessary labor. Become.

  The problem to be solved by the present invention is to provide a technique for ensuring the security of an acquired image and preventing the loss of the acquired image.

  In order to solve the above-described problem, the image management system according to the present embodiment includes an acquisition unit, a legibility determination unit, a storage unit, and an erasure instruction unit. The acquisition unit acquires an image obtained by photographing a document and stored in the first storage unit. The readability determination unit evaluates the readability of the acquired image, and determines whether or not readability is possible based on the evaluation result. The image storage unit stores a legible image in the second storage unit. When the readable image is stored in the second storage unit, the deletion instruction unit instructs to delete the image stored in the first storage unit.

It is a block diagram which shows the hardware constitutions of the image management system of 1st Embodiment. It is a block diagram which shows the function structure of the image management system of 1st Embodiment. It is a flowchart which shows operation | movement of the image management system of 1st Embodiment. It is a flowchart which shows the image acquisition operation | movement of the image management system of 1st Embodiment. It is a figure which shows a mode that a document is image | photographed with the camera which a client terminal has. It is a figure which shows a mode that the edge of a document is detected. It is a figure which shows the image of the preserve | saved document. It is a figure which shows the image after trapezoid correction and trimming of the acquired image. It is a flowchart which shows the image selection operation | movement of the image management system of 1st Embodiment. It is a block diagram which shows the function structure of the image management system of 2nd Embodiment. It is a flowchart which shows operation | movement of the image management system of 2nd Embodiment. It is a figure which shows the list | wrist of a document name required for a procedure. It is a figure which shows the result of an OCR process. It is a figure which shows the character string candidate for determining whether a document is legible and whether it is an appropriate document. It is a block diagram which shows the function structure of the image management system of 3rd Embodiment. It is a flowchart which shows operation | movement of the image management system of 3rd Embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
A hardware configuration of the image management system according to the first embodiment will be described.

  FIG. 1 is a block diagram illustrating a hardware configuration of an image management system 1 according to the first embodiment. The image management system 1 according to the first embodiment includes a client terminal 10 and a server 20.

  The client terminal 10 includes a CPU (Central Processing Unit) 11, a camera 12, a communication unit 13, a storage unit 14, a display unit 15, and an input unit 16. A digital camera, PC, etc. are mentioned. The CPU 11 is a device that realizes the function of the client terminal 10 by operating according to a program such as a document image acquisition application stored in the storage unit 14. The camera 12 is a device that captures a document or the like. The communication unit 13 is a device that communicates data and the like with the server 20. The storage unit 14 is a device for storing a program for operating the CPU 11 and information passed from the CPU 11. The display unit 15 is a device that displays information passed from the CPU 11, and examples thereof include a liquid crystal display and an organic EL display. The input unit 16 is a device that detects an operation by the user and passes it to the CPU 11, and is a pointing device, a keyboard, a touch panel, or the like.

  The server 20 includes a CPU 21, a communication unit 22, and a storage unit 23. The CPU 21 is a device that realizes the function of the server 20 by operating in accordance with a program stored in the storage unit 23. The communication unit 22 is a device that communicates data and the like with the client terminal 10. The storage unit 23 is a device that stores programs for operating the CPU 11 and information passed from the CPU 21.

  Next, the functional configuration of the image management system 1 will be described. FIG. 2 is a block diagram illustrating a functional configuration of the image management system 1 according to the first embodiment. The client terminal 10 and the server 20 have the functions of the image management system 1 of the first embodiment.

  A functional configuration of the client terminal 10 will be described. The client terminal 10 includes an acquisition unit 101, a selection unit 102, a communication unit 103, an image deletion unit 104 (deletion unit), and a message display unit 105.

  The acquisition unit 101 acquires a plurality of images obtained by photographing a predetermined document and stored in the storage unit 14 (first storage unit). The acquisition unit 101 may acquire a plurality of images obtained by photographing an external terminal and stored in an external storage unit (first storage unit) via a network. The selection unit 102 selects a predetermined image from the images acquired by the acquisition unit 101. The communication unit 103 communicates various information with the server 20. The image erasure unit 104 erases the image stored in the storage unit 14. The message display unit 105 displays a message on the input unit 16 of the client terminal 10.

  A functional configuration of the server 20 will be described. The server 20 includes an image acquisition start cooperation unit 201 (image acquisition permission information transmission unit), a communication unit 202, a legibility determination unit 203, a message notification unit 204 (erasure instruction unit), and an image storage unit 205. .

  The image acquisition start cooperation unit 201 (shooting permission information transmission unit) starts the document image acquisition application in cooperation with the acquisition unit 101. The communication unit 202 has the same function as the communication unit 103. The legibility determination unit 203 evaluates the legibility of the selected image and determines whether or not the legibility is legible. The message notification unit 204 transmits an image deletion command and a reacquisition message to the client terminal 10 via the communication unit 202 based on the determination result of the legibility determination unit 203. The image storage unit 205 stores the selected image in the storage unit 23.

  Next, the operation of the image management system 1 of the first embodiment will be described. FIG. 3 is a flowchart showing the operation of the image management system 1 of the first embodiment. This will be described with reference to FIGS.

  First, the acquisition unit 101 acquires a plurality of images obtained by photographing a predetermined document (S1). S1 will be described in detail below. FIG. 4 is a flowchart showing an image acquisition operation of the image management system 1 of the first embodiment. Here, as shown in FIG. 2, the acquisition unit 101 includes an activation processing unit 101a (identification information transmission unit), an edge detection unit 101b, an ideal frame display unit 101c, an image acquisition unit 101d, and a trapezoid correction unit 101e. And have. Further, it is assumed that the document image acquisition application has an unauthorized shooting prevention function, and shooting cannot be started only by being activated.

  The activation processing unit 101a cooperates with the client terminal 10 and the server 20 to activate the document image acquisition application (S1a). Specifically, the activation processing unit 101a performs the following operation. When the document image acquisition application is started, the activation processing unit 101 a uses the information such as the client terminal ID (identification information) stored in the storage unit 14 and the image acquisition start cooperation unit 201 of the server 20 via the communication unit 103. Send to.

  When the image acquisition start cooperation unit 201 of the server 20 receives the client terminal ID from the activation processing unit 101 a, the image acquisition start cooperation unit 201 collates the received client terminal ID with the client terminal ID stored in the storage unit 23 in advance. In addition, when the image acquisition start cooperation unit 201 receives the client terminal ID, the image acquisition start cooperation unit 201 collates the current time with a time range (predetermined condition) that is stored in the storage unit 23 and is scheduled to be captured.

  The image acquisition start cooperation unit 201 activates the image acquisition permission information via the communication unit 202 when it is determined that the client terminal IDs match and the current time is included in the scheduled time range as a result of the collation. It transmits to the processing unit 101a. The activation processing unit 101a causes the document image acquisition application to read the received image acquisition permission information, thereby canceling the unauthorized photographing prevention function of the document image acquisition application. The document image acquisition application operates only for a certain period by canceling the unauthorized photographing prevention function, and after that period, the document image acquisition application returns to a setting in which photographing cannot be performed by the unauthorized photographing prevention function.

  FIG. 5 is a diagram illustrating a state in which a document is photographed by the camera 12 included in the client terminal 10. FIG. 6 is a diagram illustrating a state in which the edge of the document is detected. When the continuous shooting of the camera 12 of the client terminal 10 starts as shown in FIG. 5, the edge detection unit 101b searches for the edge of the image from the luminance gradient of the shot image as shown in FIG. 6 (S1b).

  When the edge of the image is detected, the ideal frame display unit 101c causes the display unit 15 to display the ideal frame on the captured image as shown in FIG. 6 (S1c). The ideal frame is displayed on the display unit 15 of the client terminal 10 based on coordinate information corresponding to the display screen of the camera 12.

  FIG. 7 is a diagram showing an image of a saved document. FIG. 8 is a diagram showing an image after trapezoidal correction and trimming of the acquired image. The image acquisition unit 101d continuously captures, for example, 30 images per second by the camera 12 of the client terminal 10, and acquires n images when the captured image enters the ideal frame (S1d). During continuous shooting, the image acquisition unit 101d counts down and displays the remaining number of shots on the preview screen of the display unit 15. The image acquired in this way may be distorted into a trapezoid depending on the orientation of the camera 12, as shown in FIG. Therefore, the keystone correction unit 101e corrects the keystone of the image acquired by the image acquisition unit 101d (S1e). Through such processing, the acquisition unit 101 acquires a plurality of images shown in FIG.

  Next, the selection unit 102 selects an image satisfying a predetermined condition from the plurality of images acquired by the acquisition unit 101 (S2). S2 will be described in detail below. FIG. 9 is a flowchart showing an image selection operation of the image management system 1 of the first embodiment. Here, as illustrated in FIG. 2, the acquisition unit 101 includes an image selection unit 102a, an image correction unit 102b, and an image storage unit 102c.

  The image selection unit 102a calculates a luminance change rate for each adjacent pixel for each of the plurality of images acquired by the acquisition unit 101, and calculates an average value (sharpness) of the change rate in the image as an evaluation value. . The image selection unit 102a selects the image with the highest calculated evaluation value as an image with less blur due to camera shake or focus shift at the time of shooting (S2a). The image correcting unit 102b performs trimming / contrast correction on the selected image (S2b). The image storage unit 102c stores the corrected image after trimming / contrast correction in the storage unit 14 (S2c).

  Next, the communication unit 103 transmits the corrected image to the communication unit 202 of the server 20 (S3). The communication unit 202 of the server 20 receives the corrected image (S4). The legibility determination unit 203 calculates the sharpness of the corrected image as an evaluation value. And the legibility determination part 203 acquires the threshold value preserve | saved beforehand at the memory | storage part 23, and determines whether an evaluation value is larger than a threshold value, ie, whether it is legible (S5).

  When the evaluation value is larger than the threshold value (S5, yes), the legibility determination unit 203 passes the determination result and the corrected image to the message notification unit 204, and passes the corrected image to the image storage unit 205. The image storage unit 205 stores the corrected image in the storage unit 23 (S6). Further, the message notification unit 204 transmits an image deletion command to the client terminal 10 via the communication unit 202 based on the determination result (S7). The communication unit 103 of the client terminal 10 receives the image erasing command (S8). The image erasure unit 104 erases the corrected image stored in the storage unit 14 based on the image erasure command (S9).

  On the other hand, when the evaluation value is smaller than the threshold value in S5 (S5, no), the message notification unit 204 transmits a reacquisition message to the client terminal 10 based on the determination result (S10), and returns to S1. At this time, the message display unit 105 displays an image reacquisition message on the display unit 15 of the client terminal 10.

  As described above, according to the first embodiment, since the corrected image on the client terminal 10 side is deleted after the corrected image is stored in the server 20, the security of the acquired image is ensured and the loss of the acquired image is prevented. can do. Further, since the client terminal 10 and the server 20 are linked, unauthorized photographing by the user can be prevented.

  Further, according to the first embodiment, after selecting an image with less blur from a plurality of images acquired by the client terminal 10, the server 20 evaluates the legibility by the sharpness. For this reason, the image management system 1 can reduce the image processing burden of the server 20.

(Second Embodiment)
Next, a second embodiment will be described. The second embodiment can be realized by the same configuration as the first embodiment except that an OCR (Optical Character Reader) processing function is added to the server 20 of the first embodiment.

  A functional configuration of the second embodiment will be described. FIG. 10 is a block diagram illustrating a functional configuration of the image management system 2 according to the second embodiment. The description of the same contents as in the first embodiment is omitted.

  The second embodiment includes an OCR processing unit 206 and a document type confirmation unit 207 in addition to the functional configuration of the first embodiment. The OCR processing unit 206 reads a character string or the like indicated by the corrected image by the OCR processing function of the server 20. The document type confirmation unit 207 confirms the document type of the document indicated by the corrected image.

  The operation of the image management system 2 of the second embodiment will be described. FIG. 11 is a flowchart showing the operation of the image management system 2 of the second embodiment. FIG. 12 shows a list of document names necessary for the procedure. FIG. 13 is a diagram illustrating a result of the OCR process. FIG. 14 is a diagram showing character string candidates for determining whether or not a document is legible and appropriate. In the following description, FIGS. 1 and 2 may be used.

  First, the user presses the document name of a document to be acquired from the screen displayed on the display unit 15 of the client terminal 10. When one document name is selected from the list of document names as shown in FIG. 12, a document image acquisition application is activated. Thereafter, the operations of S1 and S2 are performed as in the first embodiment. The communication unit 103 transmits the document name information selected by the user and previously stored in the storage unit 14 to the server 20 in addition to the corrected image (S3). The communication unit 202 of the server 20 receives these images and information (S4). The OCR processing unit 206 performs OCR processing on the corrected image, reads the character string of the corrected image, and acquires an OCR result as shown in FIG. 13 (S5a). The legibility determination unit 203 acquires information on a Japanese dictionary stored in advance in the storage unit 23 of the server 20 and determines whether the OCR result is a character string that makes sense as Japanese (S5b). At this time, the legibility determination unit 203 may acquire and determine information on character string candidates stored in advance in the storage unit 23 as shown in FIG. 14 instead of the Japanese dictionary.

  If the character string is meaningful in Japanese (S5b, yes), the document type confirmation unit 207 reads out the character string candidate related to the insurance payment service shown in FIG. Then, it is confirmed whether or not a character string candidate is included in the result of the OCR process (S5c). Here, even when the character string candidate is not included in the result of the OCR process, the document type confirmation unit 207 performs the OCR process and the character string candidate search by rotating the corrected image by 90 degrees. The document type confirmation unit 207 repeats these three times. As a result, when a character string candidate is included (S5c, yes), the document type confirmation unit 207 stores the corrected image in the storage unit 23 of the server 20 as in the first embodiment (S6), An image deletion command for instructing to delete the corrected image is transmitted to the client terminal 10 (S7). The client terminal 10 receives the message (S8), and deletes the corrected image stored in the storage unit 14 (S9).

  On the other hand, when the character string does not make sense as Japanese (S5b, no), and when the character string candidate is not included in the result of the OCR process (S5c, no), the message notification unit 204 displays the image reacquisition message. The data is transmitted to the client terminal 10 (S10), and the process returns to S1. At this time, the message display unit 105 displays an image reacquisition message on the display unit 15 of the client terminal 10. Note that the legibility may be determined by any one of the processes S5b and S5c.

  As described above, since the image management system 2 evaluates the readability of the corrected image in the server 20 by performing the OCR process, it is possible to directly and quickly determine whether or not the image can be read. Also, since the document type is automatically determined, when an image of a document that is completely different from the desired document is acquired, it can be immediately determined that the document is different from the desired document.

(Third embodiment)
An image management system 3 according to the third embodiment will be described. The image management system 3 of the third embodiment can be realized with the same hardware configuration as the image management system 2. In addition to the above, the storage unit 23 of the server 20 stores necessary document information required for each business procedure such as stored membership procedure business and insurance payment business, customer visit date / time information, and workflow for each business procedure. Are associated and stored in advance. Other than this, the hardware configuration is the same as that of the second embodiment.

  A functional configuration of the third embodiment will be described. FIG. 15 is a block diagram illustrating a functional configuration of the image management system 3 according to the third embodiment. The description of the same contents as in the second embodiment is omitted. The third embodiment includes a workflow registration unit 208 and a procedure progress management unit 209 in addition to the functional configuration of the second embodiment. The workflow registration unit 208 registers the workflow stored in the storage unit 23 of the server 20 in the schedule table. The workflow indicates the work content necessary for performing a plurality of procedures included in the workflow, and the scheduled work date or scheduled work time of each work content. The procedure progress management unit 209 manages the progress status of the workflow indicating the necessary procedure based on the information received from the client terminal 10. Work schedule information and a schedule are stored in the storage unit 23 of the server 20 in advance. Further, it is assumed that the work content is registered in the schedule table of the server 20 by the scheduled work date.

  The operation of the image management system 3 of the third embodiment will be described. FIG. 16 is a flowchart showing the operation of the image management system 3 of the third embodiment.

  First, the workflow registration unit 208 registers in advance a workflow stored in the storage unit 23 of the server 20 in the schedule table before the acquisition unit 101 acquires an image (S0). Then, similarly to the operation of the image management system 2, the user presses the document name of the document to be acquired from the screen displayed on the display unit 15. Thereafter, similarly to the second embodiment, the operations from S1 to S5c are performed.

  The image storage unit 205 stores the corrected image in the storage unit 23 (S6a). After the corrected image is saved, the procedure progress management unit 209 advances the workflow to the next procedure by saving that the required document acquisition procedure is completed among the workflows registered in the schedule, and updates the workflow (S6b). Thereafter, as in the second embodiment, the operations from S7 to S10 are performed, and the operation is terminated.

  In this way, the image management system 3 manages a necessary workflow according to a desired business schedule from a customer, and advances the workflow to the next procedure after the acquisition of necessary documents is completed. For this reason, according to the image management system 3, it is not necessary for the user to self-manage necessary business procedures, so that the necessary business can be easily performed, and business deficiencies can be resolved.

  In the first to third embodiments, the client terminal 10 and the server 20 have the functions of the image management systems 1 to 3 as shown in FIGS. 2, 10, and 15. The legibility determination unit 203 may be included. In this case, the server 20 stores the image whose legibility is determined by the legibility determination unit 203 of the client terminal 10 in the storage unit 23. The image storage unit described in the claims may be a communication unit that transmits an image determined to be legible to the server 20, and the erasure instruction unit described in the claims is transmitted from the server 20. It may be a communication unit that receives the erasing instruction command and causes the image erasing unit 104 to erase the image. Therefore, the image management systems 1 to 3 may be applied to the client terminal 10. On the other hand, the server 20 may include the selection unit 102 of the client terminal 10. In this case, the client terminal 10 need only acquire and delete the image. The acquisition unit described in the claims may be a communication unit that receives an image captured by the client terminal 10. Accordingly, the image management systems 1 to 3 may be applied to the server 20.

  Further, in the image management systems 1 to 3, the information acquired by the image acquisition unit 101 d may be transmitted to the image acquisition start cooperation unit 201 of the server 20 via the communication unit 103. If the image acquisition start cooperation unit 201 does not receive the information about the start of shooting even after the elapse of a predetermined period after the transmission of the shooting permission information, the image acquisition start cooperation unit 201 may notify the business management department to that effect.

  In the selection units 102 of the image management systems 1 to 3, the selected image is corrected by the image correction unit 102 b after selecting the selected image, but the order is not limited. That is, the selection unit 102 may select all images obtained by the acquisition unit 101 after trimming and contrast correction by the image correction unit 102b, and then select an image with a high evaluation value by the image selection unit 102a. In this case, an image with less blur and blur can be reliably selected from the images acquired by the acquisition unit 101.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  1-3 Image management system, 10 Client terminal, 11 and 21 CPU, 12 Camera, 13 and 22 Communication unit, 14 and 23 Storage unit, 15 Display unit, 16 Input unit, 20 Server, 101 Acquisition unit, 101a Startup processing unit , 102 selection unit, 103 and 202 communication unit, 104 image deletion unit, 105 message display unit, 201 image acquisition start cooperation unit, 203 legibility determination unit, 204 message notification unit, 205 image storage unit, 206 OCR processing unit, 207 Document type confirmation unit, 208 workflow registration unit, 209 procedure progress management unit.

Claims (8)

A client terminal and a server connected to the client terminal via a network;
The client terminal is
An identification information transmitting unit that transmits identification information of the client terminal to the server;
By receiving the image acquisition permission information from the server, an acquisition unit to start shooting documents, the image stored in the first storage section obtained by photographing the document, and acquires,
An image transmission unit that transmits the image acquired by the acquisition unit to the server;
An erasure unit for erasing an image stored in the first storage unit by receiving an erasure instruction from the server;
With
The server
A collation unit that collates the identification information transmitted from the client terminal with identification information stored in advance, and collates a predetermined condition indicating a current time and a scheduled shooting time;
An image acquisition permission information transmitting unit that transmits the image acquisition permission information to the client terminal when the collated results match;
A readability determination unit that evaluates the readability of an image transmitted from the client terminal and determines whether readability is possible based on the result of the evaluation;
An image storage unit for storing the legible image in the second storage unit;
If possible image of the read is stored in the second storage unit, an image management system, wherein the client terminal, and a deletion instruction unit which transmits the deletion instruction. The image management system according to claim 1, wherein the evaluation is performed based on a sharpness of an image transmitted from the client terminal . The image management system according to claim 1, wherein the evaluation is performed by recognizing a character string indicated by an image transmitted from the client terminal by OCR processing. The server further includes:
The image management system according to claim 3, further comprising: a document type confirmation unit configured to confirm a type of the photographed document based on a character string candidate related to a predetermined document and the character string. The server further includes:
A workflow registration unit for registering a workflow indicating the work content necessary for performing a predetermined procedure;
5. A workflow management unit that stores, after the readable image is stored in the second storage unit, the completion of the operation of acquiring the image of the document among the work contents. The image management system according to any one of the above. A client terminal connected to the server via the network
An identification information transmitting unit that transmits identification information of the client terminal to the server;
The server collates the identification information transmitted from the client terminal with previously stored identification information, collates the current time with a predetermined condition indicating the scheduled shooting time, and the collation results match. In this case, the image acquisition permission information transmitted to the client terminal is received to start capturing the document, and an acquisition unit that acquires the image obtained by capturing the document and stored in the first storage unit When,
An image transmission unit that transmits the image acquired by the acquisition unit to the server;
The server evaluates the readability of the image transmitted from the client terminal, determines whether or not the image can be read based on the result of the evaluation, stores the readable image in a second storage unit, and reads the read When an erasable image is stored in the second storage unit, an erasure unit that erases the image stored in the first storage unit by receiving an erasure instruction transmitted to the client terminal
Client-side program to function as a client. A server connected to the client terminal via the network
A collation unit that collates the identification information of the client terminal transmitted from the client terminal and the identification information stored in advance, and collates the current time with a predetermined condition indicating the scheduled shooting time;
An image acquisition permission information transmitting unit that transmits image acquisition permission information to the client terminal when the collated results match;
Image acquisition permission information received by the client terminal, shooting of a document is started, an image obtained by shooting the document and acquired in the first storage unit is acquired and transmitted to the server A readability determination unit that evaluates the readability of the image and determines whether readability is possible based on the result of the evaluation;
An image storage unit for storing the legible image in the second storage unit;
When the legible image is stored in the second storage unit, an erasure instruction unit that transmits an erasure instruction for erasing the image stored in the first storage unit to the client terminal
Server-side program to function as The client terminal
Transmitting identification information of the client terminal to a server connected to the client terminal via a network;
The server is
Collating the identification information transmitted from the client terminal with previously stored identification information, collating the current time with a predetermined condition indicating the scheduled shooting time,
If the collated results match, the image acquisition permission information is transmitted to the client terminal,
The client terminal is
Receiving the image acquisition permission information from the server, to start shooting the document, to acquire the image obtained by shooting the document and stored in the first storage unit,
Sending the acquired image to the server;
The server is
Evaluate the legibility of the image transmitted from the client terminal, determine whether or not to interpret based on the result of the evaluation,
Storing the readable image in the second storage unit;
When the legible image is stored in the second storage unit, an erasure instruction is transmitted to the client terminal,
The client terminal is
The image stored in the first storage unit is deleted by receiving the deletion instruction from the server.
An image management method characterized by the above.

Images