JP7734802B2 - Image communication device, control method for image communication device, communication system, and control method for communication system - Google Patents

Description

The present invention relates to an image communication device, etc.

Conventions that notify the reception of a facsimile are known in the past for facsimiles, which are a type of image communication device.

For example, a technology has been proposed for an image forming device that creates a notification message notifying the recipient that a facsimile has been received and sends it to the recipient, where the notification message includes a URL (Uniform Resource Locator) for displaying a preview of the facsimile data and a form that allows the recipient to specify how to process the facsimile data (see, for example, Patent Document 1).

JP 2008-160531

However, the technology described in Patent Document 1 has the problem that it is not possible to explicitly confirm that a reply has been sent.

In view of the above-mentioned problems, the present application aims to provide a terminal device or the like that clearly displays that a reply has been sent to an image received by an image communication device.

In order to solve the above-mentioned problems, the first embodiment of the present disclosure comprises:
an acquisition unit that acquires a received image received by an image communication device that transmits and receives an image and information that can identify the received image that has been returned;
a display control unit that controls displaying a thumbnail image based on the received image and displaying identification information together with the thumbnail image based on the returned received image;
The terminal device is characterized by comprising:

A second embodiment of the present disclosure comprises:
A system including an image communication device that transmits and receives images, a server device that can communicate with the image communication device, and a terminal device that can communicate with the server device,
The server device
a management unit that manages received images received by the image communication device and information that can identify the received images that have been returned,
The terminal device
an acquisition unit that acquires information that can identify the received image and the returned image from the server device;
a display control unit that controls displaying a thumbnail image based on the received image and displaying identification information together with the thumbnail image based on the returned received image;
The system is characterized by comprising:

A third embodiment of the present disclosure comprises:
acquiring a received image received by an image communication device that transmits and receives an image and information that can identify the received image that has been returned;
a step of controlling to display a thumbnail image based on the received image and to display identification information together with the thumbnail image based on the returned received image;
The display method is characterized by comprising:

A fourth embodiment of the present disclosure comprises:
On the computer,
a function of acquiring a received image received by an image communication device that transmits and receives images and information that can identify the received image that has been returned;
a function of controlling display of a thumbnail image based on the received image and displaying identification information together with the thumbnail image based on the returned received image;
It is a program characterized by realizing the above.

According to the present disclosure, as one example, it is possible to explicitly display that a reply has been made to a received image.

FIG. 1 is a diagram for explaining the overall configuration of a system according to a first embodiment. FIG. 1 is a diagram for explaining an overview of a system according to a first embodiment. FIG. 2 is a diagram illustrating the functional configuration of an image communication apparatus according to the first embodiment. FIG. 2 is a diagram illustrating a functional configuration of a server device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a data configuration of timeline data in the first embodiment. FIG. 4 is a diagram illustrating an example of a data configuration of group information in the first embodiment. FIG. 2 is a diagram illustrating a functional configuration of a terminal device according to the first embodiment. FIG. 4 is a diagram for explaining processing of a terminal device in the first embodiment. FIG. 10 is a diagram for explaining a timeline display process in the first embodiment. FIG. 4 is a diagram for explaining processing of a server device in the first embodiment. FIG. 10 is a diagram for explaining an example of operation in the first embodiment. FIG. 10 is a diagram for explaining an example of operation in the first embodiment. FIG. 10 is a diagram for explaining an example of operation in the first embodiment. FIG. 10 is a diagram for explaining an example of operation in the first embodiment. FIG. 10 is a diagram for explaining an example of operation in the first embodiment. FIG. 10 is a diagram for explaining an example of operation in the first embodiment. FIG. 10 is a diagram for explaining processing of a server device in the second embodiment. FIG. 10 is a diagram for explaining an example of operation in the second embodiment. FIG. 11 is a diagram for explaining processing of a terminal device in the third embodiment. FIG. 11 is a diagram for explaining an example of operation in the third embodiment. FIG. 11 is a diagram for explaining an example of operation in the third embodiment. FIG. 11 is a diagram for explaining an example of operation in the third embodiment. FIG. 13 is a diagram for explaining an example of operation in the fourth embodiment. FIG. 13 is a diagram for explaining an example of operation in the fifth embodiment.

Embodiments of the present disclosure will be described below with reference to the drawings. Note that in this embodiment, a system including a terminal device to which the present invention is applied will be described as an example.

1. First embodiment
[1.1 Overall Description]
[1.1.1 Overall Configuration]
1 is a diagram illustrating an entire system 1. In the system 1, an image communication apparatus 10, a server apparatus 20, and a terminal apparatus 30 are connected via a network NW1.

The image communication device 10 is a device that has at least facsimile (FAX/facsimile) functionality. The image communication device 10 may be installed in an image forming device such as a digital multifunction peripheral (MFP; Multi-Function Peripheral/Printer), or may be a standalone device. The image communication device 10 can also be connected to a telephone line network NW2, such as an analog line or a digital line (ISDN).

The server device 20 is a device that provides a predetermined service (e.g., an IoT platform) and is installed, for example, on the cloud. In this embodiment, the server device 20 manages a group made up of an image communication device 10 and one or more users who use the image communication device, and provides a service that allows users belonging to the group to share information about images received by the image communication device 10.

The terminal device 30 is a device used by a user, and is, for example, an information processing device configured as a mobile terminal such as a smartphone or tablet, or a PC (Personal Computer).

Network NW1 is a network that connects each device. For example, it is realized by a LAN (Local Area Network) or a WAN (Wide Area Network). It may also be configured by combining these networks.

In this specification, "receiving a fax," "receiving a fax," and "fax reception" refer to receiving an image based on a standard using facsimile communication. Also, "sending a fax," "sending a fax," and "sending a fax" refer to sending an image based on a standard using facsimile communication. Facsimile communication is electrical communication for sending and receiving images that include text, graphics, etc., and the transmission path used for facsimile communication is called a facsimile communication network.

[1.1.2 System Operation Overview]
Next, an overview of the operation of the system 1 will be described with reference to Fig. 2. In Fig. 2, User A is a user who sends a fax to the image communication apparatus 10 using a device with facsimile functions. A device that sends a fax to the image communication apparatus 10 is called a sending device. Also in Fig. 2, User B is a user of a service provided by the server apparatus 20, and is a user who belongs to the same group as the group to which the image communication apparatus 10 belongs.

First, with reference to FIG. 2(a), we will explain the case where the image communication device 10 receives a fax. User A uses the sending device to send a fax to the image communication device 10. As a result, an image is sent from the sending device to the image communication device 10.

The image communication device 10 receives images from the transmission source device. In this embodiment, the images (fax data) received by the image communication device 10 from the transmission source device are referred to as "received images."

The image communication device 10 transmits received data including the received image and information about the received image to the server device 20. The information about the received image is, for example, information about the sending device extracted from the received image (e.g., the fax number of the sending device, etc.).

The server device 20 authenticates the image communication device 10 and receives the received data from the image communication device 10. The server device 20 manages (stores) the received images so that they can be acquired by a device used by user B (e.g., terminal device 30).

The terminal device 30 obtains the received image from the server device 20 and displays a notification (reception notification) indicating that the image communication device 10 has received a fax, or displays the received image. This allows user B to check the received image without printing it, or to download it to the terminal device 30. In this way, the received image is transferred (sent) to the server device 20 by the image communication device 10, and is obtained by the terminal device 30 via the server device 20. Therefore, by connecting the terminal device 30 to the server device 20, user B can check the receipt of the received image in a timely manner, regardless of their location, such as at their desk or while out and about.

User B may download the received images from server device 20 to a device other than terminal device 30 (for example, a mobile terminal or a personal computer). User B may also check data such as the received images and print only the necessary images from a device such as a printer. The printer may obtain the received images from server device 20, or from the device that downloaded the received images (for example, terminal device 30).

Furthermore, the server device 20 manages (stores) information indicating the operation (response) performed on the received image and the status of the received image (for example, status information such as whether a reply has been made) in association with the received image, and makes it available for acquisition by the terminal device 30. By displaying the information associated with the received image, the terminal device 30 makes it possible for the operation performed on the received image and the status of the received image to be shared with users (multiple users) in the group to which the image communication device 10 belongs. Therefore, a user who wishes to acquire information about the image communication device 10 simply needs to belong to the same group to which the image communication device 10 belongs in the services provided by the server device 20.

In this embodiment, an operation on a received image, in which a specific image corresponding to the received image (for example, an image indicating that a fax has been received) is sent to the device that sent the received image, is referred to as a "reply." Also, in this embodiment, an image sent to the device that sent the received image as a result of the reply operation is referred to as a "reply image."

The server device 20 may also manage received images for a certain period of time. This allows the user to check past data (received images) and perform operations on past data (received images).

The terminal device 30 also displays the received images and the operations performed on the received images as a timeline-style user interface (UI), allowing the user to operate the received images from the timeline-style UI. This allows the user to easily check the received images and operate the received images using the timeline-style UI.

Next, referring to Figure 2(b), we will explain the case where a user sends a reply. After User B checks the received image, he or she can immediately send a reply FAX to the sending device (the destination for User B) to inform them that the FAX has been received.

User B selects the received image to be returned and sends a request for a return to server device 20. When server device 20 receives the return request, it authenticates the user who sent the request and generates return data that includes at least the return image and information about the device that sent the received image. The information about the device is the destination of the return image when returning.

For example, the server device 20 generates a reply image based on a preset template. Furthermore, the server device 20 sets the information about the destination (sender device) of the reply image based on the information about the sender device sent from the image communication device 10 along with the received image.

User B can edit the reply data. For example, User B can edit the information about the sending device, or select or enter an original comment (comment for the reply data) to be added to the reply image. Furthermore, when User B is ready to send a reply image to User A, he or she sends a request to execute the reply to server device 20.

When user B requests that a reply be sent, the server device 20 sends reply data to the image communication device 10. The image communication device 10 faxes the reply image included in the reply data to the sending device based on the information about the sending device included in the reply data. As a result, the sending device of the received image that is the target of the reply receives the fax from the image communication device 10 and outputs the reply image. User A can check the reply image.

If the server device 20 successfully sends a reply image to the sender device, it updates the status of the received image that was the subject of the reply to "reply completed." This allows users who belong to the same group as the user who sent the reply to share information about the received image that has been replied to.

In addition, when re-replying to a received image whose status is "Replied," the terminal device 30 checks the message to prevent duplicate reply operations.

[1.2 Functional configuration]
[1.2.1 Image communication device]
The functional configuration of the image communication device 10 will be described with reference to Fig. 3. As shown in Fig. 3, the image communication device 10 is configured to include a control unit 100, an image reading unit 120, an image forming unit 130, a display unit 140, an input unit 150, a storage unit 160, a line communication unit 170, and a network communication unit 180.

The control unit 100 is a functional unit for controlling the entire image communication device 10. The control unit 100 realizes various functions by reading and executing various programs stored in the storage unit 160, and is composed of one or more arithmetic units (e.g., CPUs (Central Processing Units)).

By executing programs, the control unit 100 functions as a fax control unit 102, image processing unit 104, reception processing unit 106, and transmission processing unit 108.

The FAX control unit 102 sends and receives FAXes. For example, the FAX control unit 102 transmits an image of a document scanned by the image scanning unit 120 to another device via a telephone line from the line communication unit 170. In addition, in the case of Internet FAX, the FAX control unit 102 transmits the image to another device via the Internet from the NW communication unit 180.

In other words, the FAX control unit 102 sends and receives images to and from other devices by performing facsimile communication using the normal G3/G4 standard, or by performing facsimile communication using Internet FAX. The FAX control unit 102 may perform facsimile communication using any known standard or protocol.

FAX control unit 102 also receives images based on facsimile communication signals via line communication unit 170 or NW communication unit 180. The FAX control unit 102 stores the received image as image data in image data storage area 162.

The image data is, for example, a PDF (Portable Document Format) file. The image data may also be in a file format such as TIFF (Tagged Image File Format) or JPEG (Joint Photographic Experts Group). When the fax control unit 102 receives multiple images at once, it may store image data consisting of multiple pages, with each received image stored on a different page, in the image data storage area 162.

The image processing unit 104 performs various image-related processes. For example, the image processing unit 104 performs sharpening and tone conversion processes on images read from the image reading unit 120.

When a fax is received by the fax control unit 102, the reception processing unit 106 generates received data including image data in which the received image is stored and information related to the received image. The reception processing unit 106 transmits the received data and the authentication information stored as authentication information 164 to the server device 20 via the network communication unit 180.

The information relating to the received image is, for example, the following information: Note that all or any of the following information may be included.
- Date and time of reception - Number of received pages - Image data size - Information on the sending device

The information about the sending device may be information included in the protocol for facsimile communication, a telephone number notified by a sending telephone number notification service, information set in the sending device, etc. For example, in the case of the G3 protocol, the receiving processor 106 obtains the transmitting subscriber identification (TSI) and uses this as information about the sending device. Note that the receiving processor 106 may also extract information about the sending device from the received image by recognizing the image.

The transmission processing unit 108 receives reply data from the server device 20 via the NW communication unit 180. Furthermore, upon receiving the reply data, the transmission processing unit 108 controls the FAX control unit 102 to transmit the reply image included in the reply data to the sending device based on the information about the sending device included in the reply data.

The image reading unit 120 reads an original document and outputs image data. For example, the image reading unit 120 uses a CCD scanner to read an original document placed on the table of the image communication device 10 and outputs the image. The image reading unit 120 stores the image in the memory unit 160. The image reading unit 120 is configured, for example, by a scanner device, but may also read images stored in, for example, a USB memory.

The image forming unit 130 forms an image on recording paper, for example. That is, the image is printed on recording paper by the image forming unit 130. The image forming unit 130 is configured, for example, by a laser printer unit or the like.

The display unit 140 is a functional unit for displaying various types of information and is configured, for example, by an LCD (Liquid Crystal Display). The input unit 150 is a functional unit for accepting operations by the user of the image communication device 10. The input unit 150 is configured, for example, by a touch panel. The method for detecting input on the touch panel may be any common detection method, such as a resistive film method, an infrared method, an electromagnetic induction method, or a capacitive method. The input unit 150 may also be a hardware key, or may be configured by a touch panel and hardware keys.

The storage unit 160 is a functional unit that stores various programs and data necessary for the operation of the image communication device 10. The storage unit 160 is configured, for example, by semiconductor memory such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive).

The storage unit 160 has an image data storage area 162 and stores authentication information 164. The authentication information 164 stores a device ID, password, etc., required for authentication when using a service provided by the server device 20. The device ID is an ID used to identify a device that uses a service provided by the server device 20.

The line communication unit 170 is a functional unit that can communicate with telephone lines and communicates with other devices via telephone lines.

The NW communication unit 180 is a functional unit that connects to, for example, a LAN (Local Area Network) to send and receive information and images with other devices. The NW communication unit 180 uses a communication method such as Ethernet (registered trademark) LAN, LTE (Long Term Evolution), 4G, or 5G.

1.2.2 Server Device
The functional configuration of the server device 20 will be described with reference to Fig. 4. As shown in Fig. 4, the server device 20 includes a control unit 200, a NW communication unit 210, and a storage unit 220.

The control unit 200 is a functional unit for controlling the entire server device 20. The control unit 200 realizes various functions by reading and executing various programs stored in the memory unit 220, and is composed of one or more computing devices (e.g., CPUs).

By executing the program, the control unit 200 functions as a timeline data processing unit 202, a received data processing unit 204, and an authentication processing unit 206.

The timeline data processing unit 202 performs processing related to timeline data. Timeline data refers to data for displaying information about received images, such as received images and the details of operations performed on the received images, in timeline format. The timeline data processing unit 202 performs processing for transmitting and storing timeline data.

When the timeline data processing unit 202 receives a request to acquire timeline data from a terminal device 30, it transmits the timeline data stored in the timeline data storage area 222 to the terminal device 30 that sent the request. For example, the timeline data processing unit 202 transmits timeline data, in which the group ID of a group to which a user authenticated by the authentication processing unit 206 belongs is specified as the posting destination, to the terminal device 30 that sent the request to acquire timeline data. The group ID of the group to which the user belongs is acquired by the timeline data processing unit 202 reading, from the group information storage area 232, group information in which the user ID is stored, and then reading the group ID stored in the read group information. Furthermore, when the timeline data processing unit 202 receives a request to acquire image data from the terminal device 30, it reads the corresponding image data from the image data storage area 354 and transmits it to the terminal device 30.

The timeline data processing unit 202 may notify the terminal device 30 when timeline data is generated or updated, and prompt the terminal device 30 to acquire the timeline data. The timeline data processing unit 202 may also transmit the timeline data using a push method. In this way, the terminal device 30 can acquire the latest timeline data in real time.

When the timeline data processing unit 202 receives a request from the terminal device 30 to post a message in response to a received image, it stores timeline data based on the information included in the posting request in the timeline data storage area 222. For example, the timeline data processing unit 202 stores timeline data in which the message type is "user message" and includes predetermined information such as date and time in addition to the information included in the posting request (e.g., message, title, poster user ID, destination group ID, image ID). Note that "posting" refers to a user sending a message to the server device 20 via the terminal device 30. Also, a "message" refers to the specific content (e.g., text data) displayed on the terminal device 30.

The received data processing unit 204 receives received data from the image communication device 10 and generates timeline data related to the received images.

Specifically, the received data processing unit 204 executes the following processing to generate timeline data.
(1) Storage of Image Data The received data processing unit 204 stores the image data included in the received data in the image data storage area 224 .

(2) Generation of Image ID The received data processing unit 204 generates an image ID that uniquely identifies the received image (image data). The image ID may be a sequential number, a code in a predetermined format, a hash value of the image data, a UUID (Universally Unique Identifier), or the like.

(3) Obtaining the group ID of the image communication device 10 The received data processing unit 204 obtains the group ID of the group to which the image communication device 10 that transmitted the received image belongs. For example, the received data processing unit 204 reads out group information in which the device ID included in the authentication information transmitted from the image communication device 10 is stored from the group information storage area 232. The received data processing unit 204 sets the group ID stored in the read group information as the group ID to which the image communication device 10 belongs.

(4) Generation of Timeline Data The received data processing unit 204 generates timeline data related to the received image and stores it in the timeline data storage area 222. For example, the received data processing unit 204 generates timeline data by including information about the received image in a message, setting the date and time of receiving the received data from the image communication device 10 or the date and time of receiving the received image as the date and time, and using information about the sending device as the sender. Here, the received data processing unit 204 includes in the timeline data the file name of the image data stored in (1), the image ID generated in (2), and the group ID acquired in (3). Furthermore, the received data processing unit 204 sets the poster user ID to be included in the timeline data as an ID indicating the server device 20, the message type as a "received message," and the reply status as "no reply."

The authentication processing unit 206 authenticates users and devices that use services provided by the server device 20. For example, when the authentication processing unit 206 receives a request to obtain or post timeline data from the terminal device 30, it authenticates the user based on the authentication information sent along with the request. If the authentication processing unit 206 can authenticate the image communication device 10, it permits the timeline data processing unit 202 to execute a predetermined process. Similarly, the authentication processing unit 206 authenticates the image communication device 10 based on the authentication information sent from the image communication device 10 along with the received data. If the authentication processing unit 206 can authenticate the image communication device 10, it permits the received data processing unit 204 to execute the generation of timeline data.

The authentication processing unit 206 searches the authentication information storage area 226 for authentication information that contains the same information as the authentication information transmitted from the terminal device 30 or image communication device 10. If found, it authenticates the terminal device 30 or image communication device 10.

The NW communication unit 210 is a functional unit that connects to, for example, a LAN (Local Area Network) and transmits and receives information to and from other devices. The NW communication unit 210 uses a communication method such as Ethernet (registered trademark) LAN, LTE (Long Term Evolution), 4G, or 5G.

The storage unit 220 is a functional unit that stores various programs and data necessary for the operation of the server device 20. The storage unit 350 is configured, for example, from semiconductor memory such as an SSD or HDD.

The storage unit 220 has a timeline data storage area 222, an image data storage area 224, an authentication information storage area 226, a user information storage area 228, a device information storage area 230, and a group information storage area 232.

The timeline data storage area 222 is an area for storing timeline data. As shown in FIG. 5, the timeline data may include, for example, a poster user ID indicating the user who posted the message (e.g., "faxreceived"), a destination group ID indicating the group to which the message was posted (e.g., "grp1"), the date and time of the message (e.g., "November 6, 2019, 9:17:39"), the message title (e.g., "A new fax has been received"), the message (e.g., "Fax number 091434..."), the image ID (e.g., "1"), the image data in which the received image is stored (e.g., "FAX_201911060917_091434.pdf"), the sender of the received image (e.g., "091434"), the user ID indicating the user performing the reply, the message type (e.g., "Received message"), a reply status indicating whether a reply image corresponding to the received image has been sent (e.g., "Replied"), and an edit status indicating whether the message has been edited (e.g., "Edited").

When a message is posted by a user, the user ID of the user who posted the message is stored as the poster user ID. Furthermore, when timeline data is generated by the server device 20, a predetermined ID indicating the server device 20 (for example, "faxreceived") is stored as the poster user ID.

The date and time is the reference date and time when sorting timeline data in chronological order. For example, the date and time could be the date and time when a request to post a message was sent, or the date and time when the message was created.

The sender's fax number, etc., is stored as the sender's information. The image data is stored as information (e.g., a file name) that identifies the image data stored in the image data storage area 224. Note that the image data may also be included in the timeline data.

The message type is information indicating the type of message content. For example, the following information is stored as the message type:
Received message: A message indicating that a fax has been received. The message may include, for example, the fax number of the sending device, the date and time of reception, the number of received pages, and the file name of the image data. The message may also include content that explicitly indicates that a fax has been received (for example, text data such as "A fax has been received") or content that prompts a reply operation (for example, text data such as "Do you want to reply to the received fax?"). The content that explicitly indicates that a fax has been received may be included in the title of the timeline data.
- Reply operation message A reply operation message indicating the inquiry about the reply data to the user who is performing the reply operation. The message includes content that prompts the user to confirm or edit the reply data. Specifically, the message includes content that prompts the user to enter a comment to be added to the reply image, and content that prompts the user to confirm or edit the information about the sender device of the received image that is the target of the reply (the destination to which the reply image is sent).
System Message: A message generated by the server device 20 to inform the user of the status of the reply operation. The message includes the status of the operation on the received image and the result of the reply operation.
User Message: A message posted by a user. The message includes the content posted by the user.

Note that timeline data whose message type is a reply message stores the user ID of the user performing the reply operation.

The reply status is stored in timeline data whose message type is "received message." If no reply has been made to the image data (received image) stored in the timeline data, the reply status is stored as "no reply," and if a reply has been made, the reply status is stored as "replied."

In this way, in timeline data where the message type is "received message," received images are associated with whether or not a reply has been sent to the received image. A device that acquired the timeline data (e.g., terminal device 30) can determine whether a reply has been sent to a received image based on timeline data where the message type is "received message."

The authentication information storage area 226 stores information (authentication information) required to authenticate users and devices that use services provided by the server device 20. The authentication information stores a user ID that uniquely identifies a user or a device ID that uniquely identifies a device, in association with a password.

The user information storage area 228 stores information (user information) about users who use services provided by the server device 20. The user information stores a user ID, user name, an icon representing the user, and other information that can uniquely identify the user.

The device information storage area 230 stores information (device information) about devices (e.g., image communication devices 10) that use services provided by the server device 20. The device information stores a device ID that can uniquely identify the device, the device name, an icon representing the device, etc.

The group information storage area 232 stores information about groups (group information) managed by the server device 20. As shown in FIG. 6, the group information stores a group ID (e.g., "grp1") that uniquely identifies the group, a device ID (e.g., "MFP1") that indicates a device belonging to the group, and a user ID (e.g., "yokomitsu, hiroaki") that indicates a user belonging to the group.

1.2.3 Terminal Device
The functional configuration of the terminal device 30 will be described with reference to Fig. 7. As shown in Fig. 7, the terminal device 30 is configured to include a control unit 300, a display unit 320, an input unit 330, a NW communication unit 340, and a storage unit 350.

The control unit 300 is a functional unit for controlling the entire terminal device 30. The control unit 300 realizes various functions by reading and executing various programs stored in the memory unit 350, and is composed of one or more computing devices (e.g., CPUs).

By executing the program, the control unit 300 functions as a timeline data acquisition unit 302, a posting processing unit 304, and a notification display control unit 306.

The timeline data acquisition unit 302 acquires timeline data by sending a request to acquire timeline data to the server device 20 along with the authentication information stored as authentication information 356. The timeline data acquisition unit 302 also stores the acquired timeline data in the timeline data storage area 352. If the timeline data includes image data, the timeline data acquisition unit 302 sends a request to acquire the image data to the server device 20. When the timeline data acquisition unit 302 acquires image data from the server device 20, it stores the image data in the image data storage area 354.

The posting processor 304 posts a message by sending a request to the server device 20 to post a message for the received image along with the authentication information stored as authentication information 356. In this embodiment, the message to be posted is a message related to a specific received image, and the posting processor 304 prompts the user to select a received image related to the message. The posting processor 304 reads timeline data from the timeline data storage area 352, in which image data corresponding to the received image selected by the user is stored, and acquires the destination group ID and image ID. The posting processor 304 includes in the posting request the message and title entered by the user, the user ID of the user (poster) using the terminal device 30, the group ID (destination group ID) and image ID acquired from the timeline data, etc. This allows the posting processor 304 to send a request to the server device 20 to post a message to users in the group to which the image communication device 10 that received the received image belongs.

The notification display control unit 306 displays a pop-up screen or toast containing the specified notification content on the display unit 320.

The display unit 320 is a functional unit for displaying various types of information and is configured, for example, by an LCD. The input unit 330 is a functional unit for accepting operations by the user of the terminal device 30. The input unit 330 is configured, for example, by a touch panel. The method for detecting input on the touch panel may be any common detection method, such as a resistive film method, an infrared method, an electromagnetic induction method, or a capacitive method. The input unit 330 may also be a hardware key, or may be configured by a touch panel and hardware keys.

The NW communication unit 340 is a functional unit that connects to, for example, a LAN and sends and receives information to and from other devices.

The storage unit 350 is a functional unit that stores various programs and data necessary for the operation of the terminal device 30. The storage unit 350 is configured, for example, by a semiconductor memory such as an SSD.

The storage unit 350 has a timeline data storage area 352 for storing timeline data and an image data storage area 354 for storing image data, and also stores authentication information 356.

Authentication information 356 stores the user ID, password, etc. required for authentication when using the services provided by server device 20. The user ID is an ID used to identify the user who uses the services provided by server device 20.

[1.3 Processing flow]
The main processing flow of this embodiment will be described. First, the processing of the terminal device 30 will be described with reference to Fig. 8. Note that when the terminal device 30 transmits a request or information to the server device 20, the authentication information stored as the authentication information 356 is also transmitted, and authentication is performed by the server device 20. In the description of the processing flow, for simplicity, a description of authentication will be omitted.

1.3.1 Terminal Device
The processing of the terminal device 30 will be described with reference to Fig. 8. First, the control unit 300 (timeline data acquisition unit 302) acquires timeline data from the server device 20 (step S102).

Next, when a new fax is received, the control unit 300 (notification display control unit 306) displays a notification of new data (received image) (step S104). For example, the control unit 300 determines that a new fax has been received if the latest timeline data acquired in step S102 includes timeline data with a message type of "received message." At this time, as a notification of new data (received notification), the control unit 300 (notification display control unit 306) displays a toast or pop-up screen indicating that a fax has been received. Note that the control unit 300 may notify the user that a fax has been received by displaying a badge on the icon selected by the user to display the timeline.

Next, the control unit 300 executes a timeline display process (step S106). The timeline display process is a process for displaying a timeline based on the timeline data. The timeline display process will be described with reference to Figure 9.

The control unit 300 reads the timeline data stored in the timeline data storage area 352 (step S132).

Next, the control unit 300 displays the timeline on the display unit 320 by chronologically arranging the read timeline data and displaying the messages stored in each piece of timeline data (step S134). The control unit 300 may also display information included in the timeline data along with the messages. For example, the control unit 300 may display the poster's user ID, date and time, title, etc., or, if the editing status is "edited," may display the words "edited." Furthermore, the control unit 300 may obtain user information (user name and icon indicating the user) corresponding to the user ID from the server device 20 and display the user name and icon indicating the user.

Furthermore, for timeline data whose message type is "received message," the control unit 300 displays information to be displayed on the timeline in addition to the message displayed in step S134. The additional information displayed is a thumbnail image of the received image, a stamp indicating that a reply has been made, and a reply button for accepting a request to reply. This is explained below.

The control unit 300 displays on the display unit 320 a thumbnail image of the received image stored in the timeline data whose message type is "received message" (step S136). The thumbnail image is an image generated based on the received image, and is, for example, an image of one page of the received image (e.g., the first page) reduced in size. The thumbnail image may be generated by the terminal device 30 or by the server device 20.

When a thumbnail image is generated by the terminal device 30, the control unit 300 acquires the received image based on the image data stored in the timeline data. For example, the control unit 300 reads image data corresponding to the image data stored in the timeline data from the image data storage area 354. The control unit 300 generates a thumbnail image by reducing one page of the received image included in the read image data. Note that when a thumbnail image is generated by the server device 20, the control unit 300 requests the thumbnail image from the server device 20 along with information based on the timeline data (for example, the file name and image ID of the image data). The control unit 300 acquires the thumbnail image sent from the server device 20 in response to the thumbnail image request.

The control unit 300 then displays the generated or acquired thumbnail image near the location where the corresponding message was displayed.

Next, if the reply status stored in the timeline data for which the message type is "received message" is "replied," the control unit 300 superimposes a stamp (identification information) on the thumbnail image displayed in step S136 (step S138). If the reply status is "not replied," the control unit 300 displays the thumbnail image without superimposing a stamp (identification information). In this embodiment, identification information is information indicating that a received image has been replied to, for a received image to which a reply has been made. The identification information is displayed superimposed on the received image or a thumbnail image of the received image, or displayed around the received image or a thumbnail image of the received image. The identification information may be any information that allows the user to recognize that the received image has been replied to. The identification information may be, for example, a specified image such as a stamp or icon, or an animation, video, text, or symbol. The identification information may also be a border or the like displayed around the received image or a thumbnail image of the received image.

Next, the control unit 300 displays a reply button near the location where the message stored in the timeline data with the message type "received message" is displayed (step S140).

In addition, the control unit 300 performs processing to omit and display messages stored in timeline data whose message type is a "reply message" on the timelines of users other than the user performing the reply.

First, the control unit 300 identifies messages displayed based on timeline data whose message type is "reply message" from among the messages displayed in step S134 (step S142).

Next, for each identified message, the control unit 300 determines whether the work user ID stored in the timeline data corresponding to the message matches the user ID of the user using the terminal device 30. The control unit 300 acquires the user ID of the user using the terminal device 30, for example, by reading the user ID stored in the authentication information 356. If the work user ID stored in the timeline data does not match the user ID of the user using the terminal device 30, the control unit 300 omits (reduced) the message stored in the timeline data on the timeline (step S144). Omitted display is a display method in which the control unit 300 omits part or all of a message (a display method in which the message is not expanded in its entirety). Note that, when a message that has been omitted is selected by the user, the control unit 300 may expand the message so that the entire message is displayed. Alternatively, instead of omitting the message, the control unit 300 may hide the message. By doing this, the control unit 300 can omit or hide messages related to reply operations on the timeline of users who are not performing reply operations.

Through the above-described processing, the control unit 300 displays a timeline on the display unit 320 based on the timeline data. Here, the control unit 300 displays messages related to reply operations (for example, reply settings such as comments to be added to the reply image, and edited destinations on the sender device) only on the screen (display unit 320) of the user performing the reply operation. The control unit 300 omits messages related to reply operations by other users on the screen (display unit 320) of users who are not performing the reply operation (displaying messages without expanding them entirely). In this way, the control unit 300 does not display the operations performed by other users as common information, and displays only the results of the interaction between the user who performed the reply operation and the system (server device 20), such as the operation status and results. This allows the control unit 300 to prevent the display of information unnecessary for the user, improving user operability.

Returning to FIG. 8 , when a thumbnail image displayed along with the message displayed when a received image is received is selected by the user, for example, by clicking or tapping (step S108; Yes), the control unit 300 displays a preview image of the received image corresponding to the selected thumbnail image (step S110). The control unit 300 displays the preview image as a larger image than the thumbnail image, for example, on a screen separate from the screen displaying the timeline. In this way, the control unit 300 enables the user to confirm detailed information about the image data of the received image. Note that, as explained in step S138, only a stamp is superimposed on the image data; no changes have been made to the image data itself. Therefore, even if a thumbnail image with a stamp superimposed is selected, a preview image of the received image itself is displayed. No identification information, such as a stamp, is displayed in the preview image.

Next, the control unit 300 determines whether the reply button has been selected (step S112). If the reply button has been selected, the control unit 300 determines whether the reply status of the timeline data corresponding to the reply button is "replied" (step S112; Yes → step S114). If the reply status is "replied," the control unit 300 displays a confirmation message (warning) on the display unit 320 indicating that a reply has already been made (step S114; Yes → step S116).

By displaying a warning, for example, if a user performs an operation (in this embodiment, a reply operation) and then the same user attempts to perform the same operation again, or if another user attempts to perform the same operation, the warning is displayed. This prevents erroneous operations such as duplicate operations performed by multiple users. In addition to displaying the warning, the control unit 300 may also ask the user whether or not to continue with the reply operation. If the user chooses not to continue with the reply operation, the control unit 300 skips steps S118 to S126.

The control unit 300 also reads out timeline data in which an image ID similar to the image ID of the timeline data corresponding to the reply button is stored and the message type is "reply work message." The control unit 300 then determines whether the work user ID stored in the read timeline data matches the user ID of the user using the terminal device 30. If they match, no warning is displayed; if they do not match, a warning indicating that another user has replied may be displayed. In this case, if the user using the terminal device 30 performs a reply operation twice, the control unit 300 does not display a warning.

By performing the above-described processing, when a user other than the user who performed the reply operation presses (e.g., clicks or taps) the reply button for a received image whose reply status is "replied," the control unit 300 can display a warning (confirmation message) such as "Another user has already replied. Do you want to reply?". Furthermore, when a user who performed the reply operation presses (e.g., clicks or taps) the reply button for a received image whose reply status is "replied," the control unit 300 can display a warning (confirmation message) such as "You have already replied. Do you want to reply?".

Next, the control unit 300 sends a reply work request to the server device 20, including the image ID stored in the timeline data corresponding to the reply button and the user ID of the user using the terminal device 30 (step S118). By including the image ID in the reply work request, the server device 20 can identify the received image to be replied to.

When the timeline data acquisition unit 302 acquires timeline data whose message type is "reply message," the control unit 300 displays the message included in the acquired timeline data on the timeline (step S120). If the timeline data includes image data (e.g., a reply image), the control unit 300 displays a thumbnail image of the image data.

The control unit 300 accepts a user operation on the reply message and edits the reply data by sending the results of checking the contents of the reply data and the edited contents to the server device 20 (step S122). Furthermore, if the user instructs the control unit 300 to execute a reply, the control unit 300 sends a request to execute a reply (reply execution request) to the server device 20 (step S124).

Next, when the reply is complete, the control unit 300 superimposes a stamp on the thumbnail image of the received image that was the subject of the reply (step S126).

For example, the control unit 300 receives from the server device 20 an execution result including the image ID corresponding to the received image that was the subject of the reply, and information indicating that the reply has been completed, as information that can identify the received image to which the reply has been made. Next, the control unit 300 identifies timeline data in which the image ID received from the server device 20 is stored and whose message type is "received message." The control unit 300 superimposes a stamp on the thumbnail image displayed on the timeline that corresponds to the image data stored in the identified timeline data.

In this way, the control unit 300 immediately adds (reflects) new information indicating that a reply has been made to the received image to the thumbnail image (previous display content) already displayed on the timeline. As a result, it is possible to prevent discrepancies between the display content on the timeline and the content of operations performed by the user.

1.3.2 Server Device
Next, the processing of the server device 20 will be described with reference to Fig. 10. First, the control unit 200 determines whether or not a reply work request has been received from the terminal device 30 (step S152).

When a reply work request is received, the control unit 200 generates reply data (step S152; Yes → step S154). For example, the control unit 200 reads from the timeline data storage area 222 timeline data in which the image ID included in the reply work request is stored and whose message type is "received message" (timeline data in which the message to be displayed when the received image is received is stored). The control unit 200 acquires the sender stored in the read timeline data and generates a reply image based on the image data. The control unit 200 generates reply data that includes the acquired sender and the generated reply image.

Next, the control unit 200 generates timeline data with the message type set to "reply message" and stores it in the timeline data storage area 222 (step S156). Specifically, the control unit 200 generates timeline data with the poster user ID set to the ID indicating the server device 20 and the destination group ID set to the destination group ID stored in the timeline data read in step S154. Furthermore, the control unit 200 includes in the timeline data the image ID included in the reply request as the image ID, the user ID included in the reply request as the work user ID, and a message inquiring of the user who will be working on the reply.

The content of the message may prompt the user to enter a comment to be added to the reply image, or prompt the user to confirm information about the sender of the received image that is the subject of the reply. Furthermore, the timeline data generated in step S156 is transmitted to the terminal device 30 by the timeline data processing unit 202 based on a request from the terminal device 30 to obtain timeline data.

When the control unit 200 receives edited contents of the reply data from the terminal device 30, it reflects the edited contents in the reply data (step S158; Yes → step S160). For example, when the control unit 200 receives a comment to be added to the reply image from the terminal device 30, it adds the comment to the reply image. Furthermore, when the control unit 200 receives sender information from the terminal device 30, it updates the sender information included in the reply data to the information received from the terminal device 30.

Next, if the control unit 200 receives a reply execution request from the terminal device 30, it transmits reply data to the image communication device 10 (step S162; Yes → step S164).

The control unit 200 then receives from the image communication device 10 the result of image transmission based on the reply data, and updates the reply status of the received image that was the subject of the reply. For example, when the control unit 200 receives from the image communication device 10 a result indicating that the image was successfully transmitted, it updates the reply status of the timeline data read in step S154 to "Reply Completed" (step S166). Furthermore, the control unit 200 adds a comment indicating that a reply has been completed (for example, text data such as "Hiroaki has replied") to the message stored in the timeline data read in step S154, and updates the editing status to "Edited" (step S168). Note that the control unit 200 obtains the user who sent the reply, for example, from the working user ID stored in the timeline data in which the image ID corresponding to the received image to which the reply was sent is stored.

In this way, after completing the reply task, the control unit 200 updates the reply status and adds a comment indicating that a reply has been made to the timeline data at the time the received image that was the subject of the reply was received. This allows new information to be added to the timeline data corresponding to previously displayed content. Furthermore, by displaying a timeline based on the added timeline data, the terminal device 30 can indicate in the message displayed when the received image is received that the received image displayed as a thumbnail image in the message has been replied to.

Furthermore, the control unit 200 sends an execution result to the terminal device 30 that sent the reply execution request, including the image ID of the received image that was the subject of the reply and information indicating that the reply has been completed (step S170).

Note that if no reply is received (for example, if a result indicating that transmission of the reply image failed is received from the image communication device 10), the control unit 200 may omit the processing of steps S166 to S170. Alternatively, the control unit 200 may update the reply status of the timeline data read in step S154 to "transmission failed."

The server device 20 may also generate timeline data including a message indicating the status of the reply operation, as appropriate, such as when editing the reply data, when sending the reply data, or when receiving the reply data sending result from the image communication device 10. In this case, the control unit 200 generates timeline data in which the poster user ID is set to an ID indicating the server device 20, the destination group ID is set to the destination group ID stored in the timeline data read in step S154, the image ID is set to the image ID included in the reply operation request, the message type is set to "system message," and the message includes the content of the reply operation status, and stores the data in the timeline data storage area 222.

[1.4 Operation example]
An example of operation of this embodiment will be described with reference to the drawings. In the description of the example of operation, identical messages among messages displayed on the timeline will be assigned the same reference numerals. In the following description, a message displayed based on timeline data whose message type is "received message" will be simply referred to as a "received message." Similarly, a message displayed based on timeline data whose message type is "reply message" or "system message" will be simply referred to as a "reply message" or "system message."

Figure 11 shows an example of a screen display when a notification (reception notification) indicating that a new fax has been received by the image communication device 10 is displayed on the display unit 320. For example, when the so-called standby screen W100 is displayed as in Figure 11(a) or when the so-called home screen W102 is displayed as in Figure 11(b), a toast notification indicating that a fax has been received (that a new fax has been received) is displayed. Note that when the toast is selected, the display may switch to a screen that displays timeline data in timeline format.

Furthermore, on screen W104 of a dedicated app for using services provided by server device 20, a badge may be displayed as a notification indicating that a fax has been received (that a new fax has been received) on an icon selected by the user to display a timeline, as in area B104. By looking at the toast display or badge display displayed on display unit 320, the user can confirm that a new fax has been received, i.e., that the image communication device 10 has received a fax.

Figures 12 and 13 are examples of a display screen when a timeline is displayed on the display unit 320. On a mobile device, a timeline is generally displayed starting with the most recent information, and older information is displayed by scrolling upward. Figure 12(a) is an example of a display screen W130 when a timeline is displayed on the display unit 320 and a received message C100 is displayed on the display unit 320 by the user scrolling upward. The received message C100 includes information E100 about the received image (e.g., fax number, date and time of receipt, number of pages, file name). A thumbnail image F100 of the received image is also displayed.

Figure 12(b) is an example of the display screen W140 when the timeline displayed in Figure 12(a) is scrolled downward. A reply button B100 is displayed in the received message C100. Note that a button B102 for printing the received image may also be displayed in the received message C100.

When the reply button B100 is selected, a reply message C110 is displayed on the timeline as the most recent message. Figure 12 (b) is an example of the display screen W140 when the reply message C110 displays a message prompting the user to confirm the fax number of the sender of the received image (the destination of the reply image). The reply message C110 displays a button B110 indicating that the sender's fax number is correct and a button B112 for correcting the fax number. By selecting button B110 or button B112, the user can indicate to the server device 20 that the fax number is correct or that it needs to be corrected. When button B112 is selected, for example, a reply message for entering a fax number is displayed on the timeline.

Figure 13(a) shows an example of a display screen W150 when, after the fax number has been confirmed, a reply message C120 prompting the user to enter a comment to be added to the reply image is displayed on the timeline. The user enters the number of a pre-prepared template or a free comment in the input field T120 and selects button B120 to send the selected number or entered comment information to the server device 20 as a comment to be included in the reply image. In this way, the user can edit and check the reply data by operating the reply message.

Figure 13 (b) shows an example of a display screen W160 in which a reply message C130 prompting the user to check the reply image is displayed on the timeline after the comment information to be added to the reply image has been sent. The reply message C130 displays a thumbnail image F130 of the reply image, and also displays a button B130 for sending a reply execution request. By selecting button B130, the user can send the reply image to the sender of the received image.

The reply message C130 may also display an area E130 containing buttons for modifying the recipient's name, telephone number (fax number), and text (comments to be added to the reply image). When the user selects a button included in E130, a reply message for modifying the reply data is displayed on the timeline.

Figure 13 (c) is an example of the display screen W170 when system message C140 and system message C150 are displayed on the timeline after button B130 is selected. For example, system message C140 is a message indicating that a fax transmission will begin based on a reply operation. System message C150 is a message indicating that the fax transmission has been completed.

As shown in Figures 12(a) to 13(c), the most recent messages are displayed one after another on the timeline, with the received message C100 being a past message (past display content). Therefore, by scrolling the display screen W170 shown in Figure 13(c) upward, the received message C100 is displayed on the display unit 320. Figure 13(d) shows an example of the display screen W180 when the display screen W170 shown in Figure 13(c) is scrolled upward to display the received message C100, including a thumbnail image F100 of the received image to be replied to. The received message C100 is an older display content than the system message C150, but once the reply is complete, a stamp (completed stamp) is superimposed on the thumbnail image F100 of the received image to be replied to, which is included in the received message C100, which is past display content. In this way, even if the display content is past, information about the reply operation is added (reflected), as shown in the thumbnail image F100 shown in Figure 13(d).

Figure 14 shows an example of a timeline display. Figure 14(a) shows a timeline L100 displayed on the terminal device 30 used by the user who performed the reply operation. L100 displays the received message C100, reply operation messages C110, C120, C130, and system messages C140, C150.

Figures 14(b) and 14(c) are examples of timelines L110 and L120 displayed on a terminal device 30 used by another user who belongs to the same group as the user who sent the reply, i.e., a user who has not performed the reply operation. The terminal device 30 of a user who is not performing the reply operation may display a timeline like timeline L110 that does not display the reply operation messages C110, C120, and C130. Alternatively, the terminal device 30 of a user who is not performing the reply operation may display a timeline like timeline L120 that omits the reply operation messages C110, C120, and C130. By displaying such a timeline, users who have not performed the reply operation can know only that a fax has been received and the status and results of other users' operations, making it easier to understand the status of the received image.

Figure 15 shows an example of the display screen W190 when a warning W192 is displayed when the reply button is selected for a received image that has already been replied to. By checking the warning W192, the user can avoid performing a duplicate reply operation.

As shown in FIG. 15, a button B192 indicating to perform a reply operation and a button B194 indicating to cancel the reply operation may be displayed together with the warning W192. If the user selects B192, a reply operation message is displayed on the timeline. At this time, if a reply operation is performed in accordance with the user's operation on the reply operation message, information indicating the number of times the reply operation has been performed and information about the user who performed the reply operation may be displayed on the received image to which the reply has been made, along with identification information such as a stamp (done stamp). If the user selects B194, the reply is canceled and the reply operation message is not displayed on the timeline.

Figure 16 shows another example of the display of a received message that includes a received image to which a reply has been made. Figure 16(a) is an example of a timeline L130 that includes a received message C102 that includes a received image to which a reply has been made. The received message C102 may include a comment E102 indicating that a reply has been made (e.g., "hiroaki has replied"). Note that when the reply is completed, the message is edited to add a comment indicating that a reply has been made, and therefore the word "edited" is displayed along with the message, as shown in A102.

Figure 16 (b) is an example of a timeline L140 that includes a received message C104 that includes a received image for which a reply has been made. Instead of superimposing a stamp (Done stamp) on the received image for which a reply has been made, as in the case of the received message C104, a thumbnail image F104 of the reply image (reply data) may be displayed. The thumbnail image of the reply image may be displayed based on the image data stored in the timeline data whose message type is "reply work message."

As such, according to this embodiment, the user can confirm that a fax has been received and perform operations on the received image using a timeline-style UI. Furthermore, a stamp (identification information) is superimposed on the received image for which a reply has been sent, allowing the user to easily check whether or not a reply has been sent.

In addition, the implementation may be modified beyond the scope of the above description, as long as there are no inconsistencies.

For example, in this embodiment, a timeline (working user screen) that displays messages sent and received one-to-one between the server device 20 and users in chronological order may be displayed, separate from the timeline (shared screen) that is viewed by all users belonging to the group. In this case, the terminal device 30 may display the reply work message on the working user screen, but not on the shared screen. As a result, the reply work message is no longer displayed on the shared screen, and users who have not performed a reply operation can only know the results of the interaction between the user who performed the reply operation and the system (server device 20), such as the receipt of a fax and the status and results of other users' operations. Because unnecessary information is no longer displayed for users who do not perform a reply operation, the terminal device 30 can improve user operability.

In addition, in this embodiment, the image communication device 10 is described as being equipped with the reception processing unit 106 and the transmission processing unit 108. However, the processing performed by these functional units may be implemented in an information processing device (edge terminal) different from the image communication device 10. The edge terminal cooperates with the image communication device 10 (MFP cooperation) to implement the functions of the image communication device 10 in this embodiment. In this case, the image communication device 10 transmits received images and information related to the received images to the edge terminal. The edge terminal transmits received data, including the received images and information related to the received images received from the image communication device 10, to the server device 20 along with authentication information stored in the edge terminal. Furthermore, when the edge terminal receives transmission data from the server device 20, it controls the image communication device 10 to transmit a reply image to the transmission source device. Note that the edge terminal may also implement the functions of the image communication device 10. For example, the edge terminal may have the functionality of the FAX control unit 102, thereby enabling the edge terminal to receive and send FAXes from and to the transmission source device.

Furthermore, in the timeline display process, rather than displaying messages related to all received images, only messages related to received images received only from a sender device (destination) preset by the user may be displayed. For example, in step S132, the control unit 300 extracts timeline data from the timeline data storage area 222 that stores the fax number of the sender device specified by the user and has a message type of "received message." The control unit 300 then reads from the timeline data storage area 222 timeline data that stores the same image ID as the image ID stored in the extracted timeline data. The control unit 300 then executes the processing from step S134 onwards. In this way, rather than displaying messages related to all received images, only messages related to received images received from a sender device preset are displayed, making it easier for the user to check the received images.

Furthermore, in the timeline display process, when the reply status stored in timeline data for which the message type is "received message" is "replied," a stamp (identification information) is superimposed on the thumbnail image as described above, but other processing may also be performed. For example, the control unit 300 may display the message in a specified text color, or the background of the message in a specified color. Furthermore, the control unit 300 may color the message or thumbnail image by superimposing a semi-transparent color on it, or may display specified identification information (e.g., an icon) superimposed on or around the thumbnail image. In other words, the control unit 300 may display identification information that visually indicates that a reply has been made for messages or thumbnail images based on timeline data for which the reply status is "replied."

In addition, in step S104, the notification display control unit 306 was described as displaying a notification of a new fax based on the timeline data, but the notification of a new fax may be provided by other methods. For example, after generating the timeline data, the received data processing unit 204 transmits information indicating that a new fax has been received to a terminal device 30 used by a user who belongs to the same group as the image communication device 10 that received the received image. The notification display control unit 306 displays a notification of a new fax when it receives information indicating that a new fax has been received.

Furthermore, when the control unit 200 makes an update, such as adding an entry, to the timeline data, it may send information indicating that the timeline data has been updated to terminal devices 30 used by users belonging to a group identified by the destination group ID stored in the timeline data. Upon receiving the information indicating that the timeline data has been updated, the terminal devices 30 re-acquire the timeline data updated by the server device 20 and reflect it on the timeline. In this way, the timeline can be kept up to date.

In addition, even if the above description is not followed, the order of the steps may be changed or some steps may be omitted, as long as there are no contradictions. For example, in the timeline display process, the control unit 300 may display a reply button (step S140) and then display thumbnail images and superimposed stamps (steps S136 and S138).

According to this embodiment, a timeline based on timeline data obtained from the server device is displayed on the terminal device 30. Because a group is specified as the posting destination, the timeline data is obtained by the terminal devices 30 used by users belonging to the group, enabling information sharing. Furthermore, when a reply operation is performed on a received image, information indicating that a reply operation has been performed is added (reflected) on the timeline. Therefore, as long as the user is connected to the server device, they can share images received by the image communication device and the details of the operation via the terminal device, regardless of their location.

In addition, in this embodiment, when a reply operation is performed, the operation performed on the received image is added to the timeline display content on the terminal device 30 where the reply operation was performed. This allows the user's operation to be displayed in real time and prevents discrepancies between the already displayed timeline display content and the operation performed by the user. The server device also adds (updates) information to the timeline data storing the received image to which the reply was made. For example, the server device updates the reply status to "replied" or adds a comment indicating that the message has been replied to. By re-obtaining and displaying the timeline data, the user can prevent discrepancies between the content displayed on the timeline and the operation performed by the user. In this way, it is possible to perform operations on the display content (received image) without compromising operability, while maintaining the convenience of a service that displays messages entered between multiple user terminals in real time in chronological order. By joining a group, users can obtain information about image communication devices and receive reception notifications via their terminal devices. This prevents situations where reception notifications are only sent to pre-defined users. When you confirm a reception notification and want to reply to the received data (received image), you can easily reply by selecting the reply button displayed on the timeline. In addition, by sharing timeline data with multiple users, multiple users can share the status of their response to the received image.

Furthermore, when a user selects a notification (reception notification) that a new fax has been received and checks the timeline, they first search for a thumbnail image of the received data (received image) on the timeline and check the received image. However, when a timeline is shared with other users, the timing at which the user checks the received image will generally differ from that of the other users. Because the actions (operations) performed by other users up until the user checked the received image are displayed on the timeline, it may be difficult for the user to understand the sequence of events. As a result, it may be difficult for the user to determine whether a reply has been made to the received image. In contrast, in this embodiment, identification information such as a stamp is displayed on thumbnail images corresponding to received images to which a reply has been made, or a comment indicating that a reply has been made is added to the message, allowing the user to easily identify received images to which a reply has been made at a glance.

Furthermore, according to this embodiment, the terminal device displays messages based on timeline data whose message type is a "reply message" to users who perform a reply operation, and omits displaying the messages to users who do not perform a reply operation. By omitting and displaying information that is unnecessary for users who do not perform a reply operation, the terminal device can improve user operability.

Furthermore, according to this embodiment, if a reply operation is performed on a received image to which a reply has already been made, a warning is displayed. This allows the user to prevent erroneous operations such as sending duplicate replies.

[2. Second embodiment]
Next, a second embodiment will be described. In the second embodiment, in addition to adding a comment indicating that a reply has been completed as described in the first embodiment, a comment indicating that a reply is currently being completed (operation in progress) is added and displayed on the terminal device. In the second embodiment, FIG. 10 of the first embodiment is replaced with FIG. 17, and the same functional units and processes are denoted by the same reference numerals, and their description will be omitted.

The processing flow in the server device 20 of this embodiment will be described with reference to FIG. 17. In this embodiment, after processing step S156, the control unit 200 adds a comment indicating that a reply operation is in progress (for example, text data such as "yokomitsu is replying") to the message stored in the timeline data read in step S154 (step S252). At this time, the control unit 200 updates the editing status of the timeline data read in step S154 to "edited."

Furthermore, after processing step S168, the control unit 200 deletes the comment added in step S252 (step S254).

By the above-described process, a comment indicating that a reply is in progress is added to the message in the timeline data storing the received image to be replied to, from the time the reply request is received until the reply execution request is received, i.e., until the reply operation is completed. The terminal device 30 acquires and displays the timeline data to which the comment has been added, thereby indicating that a reply operation is being performed.

In addition, in this embodiment, in step S114, the control unit 300 of the terminal device 30 also determines whether the timeline data corresponding to the reply button includes a comment indicating that a reply is being made to the message. If the message includes a comment indicating that a reply is being made, the control unit 300 displays a confirmation message (warning) indicating that a reply is being made on the display unit 320.

At this time, as in the first embodiment, the control unit 300 may determine whether or not a reply operation has been performed by another user based on the timeline data, and switch whether or not to display a warning.

Furthermore, in step S134, if the message includes a comment indicating that a reply is being made, the control unit 300 may highlight the comment. Methods for highlighting the comment include making the character size of the comment larger than the character size of the message, using a different color for the character of the comment than the color of the character of a normal message, drawing a line around the comment, or changing the background color of the comment.

An example of the operation of this embodiment will be described with reference to Figure 18. Figure 18(a) is an example of a timeline L200 displayed on the display unit 320 in this embodiment. The timeline L200 displays a received message C200.

Received message C200 displays a thumbnail image F200 of the received image corresponding to that received message. If a reply operation is being performed on the received image corresponding to received message C200, a comment indicating that a reply is being made is displayed, as shown in area E200. In addition, since the editing status of the timeline data corresponding to received message C200 is "edited," a display indicating that the data has been edited is displayed, as shown in area A200.

Figure 18(b) is an example of a display screen W200 that is displayed on the display unit 320 when the reply button B200 displayed on the received message C200 shown in Figure 18(a) is selected. In this embodiment, if an attempt is made to reply to a received image that has already been replied to, a warning W202 is displayed. The user can choose whether or not to continue with the reply operation.

According to this embodiment, when a user is working on a reply, the terminal device 30 displays a message indicating that the reply is in progress. Depending on the content of the reply, it may take some time to complete the reply, but even in this case, the user working on the reply can prevent other users from accidentally completing the same task.

3. Third embodiment
The third embodiment is an embodiment in which timeline data is linked to image data and then a timeline is displayed. In the third embodiment, Fig. 8 of the first embodiment is replaced with Fig. 19, and the same functional units and processes are assigned the same reference numerals and descriptions thereof will be omitted.

The processing flow of the terminal device 30 in this embodiment will be described with reference to FIG. 19. In this embodiment, after the processing of step S106, the control unit 300 selects one piece of timeline data from the timeline data displayed as a timeline, and extracts timeline data that has the same image ID as the selected timeline data (step S302). In this way, the timeline data corresponding to one received image is linked.

Next, the control unit 300 displays the same identification mark for the message corresponding to the timeline data extracted in step S302 (step S304). The identification mark is, for example, a strip-shaped rectangle of a predetermined width that is displayed next to the message. In this case, the control unit 300 displays a strip-shaped rectangle with the same background color as the same identification mark next to the message corresponding to the extracted timeline data. In this way, the control unit 300 links the message to one received image and displays it in the same color.

When the control unit 300 repeatedly executes the processing of step S304, it displays an identification display that is different from the identification display that has already been displayed. For example, when the control unit 300 executes step S304 for the first time, it displays a strip-shaped rectangle with a red background. When the control unit 300 executes step S304 for the second time, it displays a strip-shaped rectangle with a green background. In this way, the same identification display is displayed for messages based on timeline data in which the same image ID is stored, but different identification displays are displayed for messages based on timeline data in which different image IDs are stored.

Note that the identification display does not have to be a strip-shaped rectangle, but can be the background color of the message, the text color of the message, an icon added to the message, etc.

Next, the control unit 300 displays in full only the messages based on timeline data whose message type is "received message" (messages displayed when the received image is received) and the messages based on timeline data with the most recent date and time (latest messages) from the extracted timeline data, and omits (reduced) the display of other messages (step S306). By doing this, the control unit 300 omits the display of all messages for the same received image (one piece of data) for each received image, except for the message at the time the received image was received (displayed at the time of reception) and the most recent message.

Next, the control unit 300 displays (places) a message display switch button for each message corresponding to the extracted timeline data, allowing the display to be switched to the previous or next message (step S308). The previous message for a given message is the message with the same identification display that has a date and time that precedes and is closest to the date and time of the given message. Similarly, the next message for a given message is the message with the same identification display that has a date and time that follows and is closest to the date and time of the given message. In other words, the message display switch button is a button that switches the display to the next or previous message in chronological order among messages related to the same received image.

Next, the control unit 300 determines whether all of the timeline data displayed as a timeline has been extracted (step S310). If not all of the timeline data has been extracted, the control unit 300 returns to step S302 and performs steps S302 to S308 on the unextracted timeline data (step S310; No → step S302).

Furthermore, when a message display switching button is selected, the control unit 300 switches the display to the corresponding message (step S312; Yes → step S314). The control unit 300 switches the message display by, for example, displaying the previous or next corresponding message at the position where the message display switching button selected by the user is displayed.

Note that the control unit 300 may execute either step S306 or step S308, or neither.

An example of operation in this embodiment will be explained with reference to the figures. Note that in the explanation of the example of operation, identical messages displayed on the timeline will be assigned the same reference numerals.

Figure 20 shows an example of a timeline L300 in which identification marks are displayed for messages based on timeline data. Here, the same image ID is stored in the timeline data corresponding to message C300, message C320, message C340, message C350, and message C360. In this case, strip-shaped rectangles R300, R320, R340, R350, and R360, each with the same background color, are displayed next to these messages.

On the other hand, the same image ID is stored in the timeline data corresponding to message C310 and message C330. However, the image IDs corresponding to message C310 and message C330 are different from the image ID corresponding to message C300. In this case, strip-shaped rectangles R310 and R330 with the same background color are displayed next to message C310 and message C330. However, rectangles R310 and R330 have a different background color from rectangle R300.

Furthermore, the image ID corresponding to message C370 is different from that of message C300 and message C310. In this case, message C370 is displayed as a strip-shaped rectangle R370 with a background color different from that of rectangle R300 and rectangle R310.

In this way, messages corresponding to the same image ID will be displayed with the same identification mark, and messages corresponding to different image IDs will be displayed with different identification marks. This makes it easy for users to check the series of tasks and statuses performed on the same received image, even when scrolling the timeline.

Figure 21 shows an example of a timeline L310 in which, among messages with the same identification, messages other than those based on timeline data with a message type of "received message" and those based on timeline data with the most recent date and time are displayed in abbreviated form.

For example, message C300, message C320, message C340, message C350, and message C360 are displayed with the same identification. Of these, messages C320, C340, and C350 other than message C300 based on timeline data with a message type of "received message" and message C360 based on timeline data with the most recent date and time are displayed in an abbreviated form. Similarly, of messages C310 and C330 with the same identification, message C310 other than message C330 based on timeline data with the most recent date and time is displayed in an abbreviated form. Furthermore, message C370 with a message type of "received message" is displayed without abbreviation.

In this way, all messages are displayed aside from those indicating the receipt of a fax and the latest status of the received image, allowing the user to easily grasp the latest status of each received image.

Figure 22(a) is an example of a timeline L320 when message display switching buttons are displayed. For example, message C300 displays button B300 for switching the display to the previous message and button B302 for switching the display to the next message. Note that if there is no previous or next message, the message display switching buttons may be made unselectable or hidden. For example, like button B300 in Figure 22(a), if there is no previous message, button B300 may be made unselectable.

Figure 22 (b) is an example of the timeline L330 when button B302 in Figure 22 (a) is selected. In this case, message C320, the next message, is displayed in the position where message C300 was displayed. Message C320 also displays button B320 for switching the display to the previous message and button B322 for switching the display to the next message. When button B320 is selected, message C300, the previous message, is displayed in the position where message C320 was displayed. When button B322 is selected, the next message (e.g., message C340) is displayed in the position where message C320 was displayed.

In this way, the user can switch between messages with the same identification, i.e., messages related to the same received image. As a result, the user can easily grasp the status of each received image.

According to this embodiment, the timeline data associated with the same received image is linked and then displayed as a timeline, allowing the user to easily understand the situation regarding the same received image.

Generally, the display area on a mobile device screen is limited, and messages are displayed in chronological order, making it difficult to understand the connection between the previous and next messages for the same information. For example, as shown in Figure 20, messages related to operations on different received images may be displayed in chronological order. Furthermore, for example, when a user checks message C350 in Figure 20, they can see that reply data is being created (edited), but it is difficult to identify which of the multiple fax data (received images) the message relates to.

In this embodiment, messages related to the same received image are marked with the same identification, making it easier for the user to check the status and series of tasks (operations) for the data (received image) even when scrolling through the screen of the mobile terminal (terminal device). Furthermore, among messages marked with the same identification, messages other than the message when the received image was received and the message based on the most recent timeline data are omitted, making it easier for the user to check the status of a single piece of data (received image). Furthermore, by selecting a button to switch the display to the previous or next message, the user can switch the display to messages that are chronologically related, making it easier to see the connection between the previous and next messages in the information related to the received image.

4. Fourth embodiment
The fourth embodiment is an embodiment in which a contact (notification) is made when there is a received image for which a reply operation has not been performed and for which a reply has not yet been made.

In this embodiment, for example, the notification display control unit 306 periodically extracts timeline data from the timeline data stored in the timeline data storage area 352, the timeline data having a date and time within a predetermined period (e.g., one week) and a reply status of "no reply." The timeline data extracted in this manner is timeline data that includes received images for which no reply operation has been performed within the predetermined period. When one or more timeline data are extracted, the notification display control unit 306 displays a notification indicating that there are received images for which no reply operation has been performed. Note that the notification display control unit 306 may also display a message stored in the extracted timeline data.

Figure 23 is a diagram showing an example of operation in this embodiment. Figure 23 is an example of a display screen W400 when a notification W402 indicating that there is a received image for which a reply operation has not been performed is displayed on the display unit 320. The notification W402 includes, for example, the contents of the message stored in the timeline data extracted by the notification display control unit 306 (for example, information related to the received image, such as the fax number and the date and time of reception).

According to this embodiment, even if unreplied data (received images) is left unattended, the user will receive a notification (notification) from the system (e.g., terminal device 30) via a toast display or the like. By checking the notification, the user can prevent missing out on responding to unreplied received images.

5. Fifth embodiment
The fifth embodiment is an embodiment in which the status of received images is displayed by a method other than the timeline.

In this embodiment, in step S106, the terminal device 30 performs the following process instead of the timeline display process.

The control unit 300 reads out only timeline data whose message type is "received message" and displays the messages in table format on the display unit 320. Next, the control unit 300 displays thumbnail images based on the image data of the read timeline data. If the reply status of the read timeline data is "replied," the control unit 300 superimposes a stamp on the thumbnail image. The control unit 300 also displays a reply button.

When the reply button is selected, the control unit 300 displays a warning if the reply status of the timeline data corresponding to the reply button is "Replied." Furthermore, when a reply operation is performed, the control unit 300 may, for example, display a reply operation message on the display unit 320 and accept an operation to edit the reply data or a request to execute a reply.

Figure 24 is a diagram showing an example of operation in this embodiment. Figure 24 is an example of a display screen W500 when timeline data is displayed in table format on the display unit 320. A message E500 and a thumbnail image F500 of a received image are displayed on the display screen W500. Note that a stamp is superimposed on the thumbnail image F502 of a received image corresponding to timeline data whose reply status is "Replied".

Also displayed on the display screen W500 is a button B500 for replying to the received image. When the user selects button B500, a screen for accepting an operation to reply to the corresponding received image is displayed on the display unit 320.

In this embodiment, the timeline data is described as being displayed in a table format, but it may also be displayed in a format other than a table. It may also be possible to switch between a screen that displays the timeline data in timeline format and a screen that displays it in table format.

According to this embodiment, received images and the status of the received images can be displayed in a summarized form. Therefore, even if a terminal device displays a large amount of timeline data and the displayed timeline is long, the user can easily grasp the received images and the status of the received images.

6. Modifications
The present invention is not limited to the above-described embodiments, and various modifications are possible. In other words, embodiments obtained by combining technical means that are appropriately modified within the scope of the gist of the present invention are also included in the technical scope of the present invention.

Furthermore, although the above-mentioned embodiments are described separately in some parts for convenience of explanation, they may of course be combined to the extent technically possible. For example, the third embodiment may be combined with the fifth embodiment. In this way, the user can check the messages related to each received image in timeline format and view an overview of the status of each received image in table format.

In addition, the programs that run on each device in the embodiments are programs that control the CPU and other devices (programs that cause a computer to function) so as to realize the functions of the above-mentioned embodiments. Information handled by these devices is temporarily stored in temporary storage devices (e.g., RAM) during processing, and is then stored in various storage devices such as ROMs (Read Only Memory) and HDDs, from which it is read, modified, and written by the CPU as needed.

Here, the recording medium for storing the program may be any of semiconductor media (e.g., ROM, non-volatile memory cards, etc.), optical recording media/magneto-optical recording media (e.g., DVD (Digital Versatile Disc), MO (Magneto Optical Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray Disc), etc.), and magnetic recording media (e.g., magnetic tape, flexible disks, etc.). Furthermore, not only are the functions of the above-described embodiments realized by executing the loaded program, but the functions of the present invention may also be realized by processing in cooperation with an operating system or other application programs, etc., based on the program's instructions.

Furthermore, when distributing the program on the market, it can be stored on a portable recording medium and distributed, or transferred to a server computer connected via a network such as the Internet. In this case, the storage device of the server computer is of course also included in the present invention.

1 System 10 Image communication device 100 Control unit 102 FAX control unit 104 Image processing unit 106 Reception processing unit 108 Transmission processing unit 120 Image reading unit 130 Image forming unit 140 Display unit 150 Input unit 160 Storage unit 162 Image data storage area 164 Authentication information 170 Line communication unit 180 NW communication unit 20 Server device 200 Control unit 202 Timeline data processing unit 204 Received data processing unit 206 Authentication processing unit 210 NW communication unit 220 Storage unit 222 Timeline data storage area 224 Image data storage area 226 Authentication information storage area 228 User information storage area 230 Device information storage area 232 Group information storage area 30 Terminal device 300 Control unit 302 Timeline data acquisition unit 304 Posting processing unit 306 Notification information control unit 320 Display unit 330 Input unit 340 NW communication unit 350 Storage unit 352 Timeline data storage unit 354 Image data storage area 356 Authentication information

Claims (8)

An image communication apparatus capable of communicating with a terminal apparatus via a server apparatus that provides a service for managing a group made up of an image communication apparatus and one or more users who use the image communication apparatus,
a control unit for transmitting and receiving images to and from other devices by performing facsimile communication;
The control unit
When the image is received from the other device, the server device transmits received data including the image and information about the image to the server device;
In response to receiving reply data in response to the received data from the server device, transmitting a reply image included in the reply data to the other device;
transmitting a transmission result of the reply image to the server device so that a reply status based on the transmission result of the reply image can be displayed on the terminal device so as to be shareable among one or more users belonging to the group;
Image communication equipment. the reply data is transmitted from the server device when the server device receives a reply request for the received data from the terminal device.
2. The image communication device according to claim 1. The transmission result is used by the server device to update the reply status.
3. The image communication apparatus according to claim 1. If the transmission result indicates that the reply image has been successfully transmitted, the transmission result is used by the server device to update the reply status to "returned."
4. The image communication apparatus according to claim 3. If the transmission result indicates that the transmission of the reply image has failed, the transmission result is used by the server device to update the reply status to "transmission failed."
4. The image communication apparatus according to claim 3. 1. A method for controlling an image communication apparatus capable of communicating with a terminal device via an image communication apparatus that transmits and receives images to and from other apparatuses by performing facsimile communication, and a server apparatus that provides a service for managing a group made up of one or more users who use the image communication apparatus, comprising:
When the image is received from the other device, the server device transmits received data including the image and information about the image to the server device;
In response to receiving reply data in response to the received data from the server device, transmitting a reply image included in the reply data to the other device;
transmitting a transmission result of the reply image to the server device so that a reply status based on the transmission result of the reply image can be displayed on the terminal device so as to be shareable among one or more users belonging to the group;
A method for controlling an image communication device. an image communication device that transmits and receives images to and from other devices by performing facsimile communication;
a server device that provides a service for managing a group made up of one or more users who use the image communication device;
a terminal device capable of communicating with the image communication device via the server device;
A communication system comprising:
The image communication device
When the image is received from the other device, the server device transmits received data including the image and information about the image to the server device;
In response to receiving reply data in response to the received data from the server device, transmitting a reply image included in the reply data to the other device;
transmitting a transmission result of the reply image to the server device so that a reply status based on the transmission result of the reply image can be displayed on the terminal device in a manner that can be shared among one or more users belonging to the group;
The terminal device
In response to receiving from the server device a transmission result of the reply image for which execution of a reply has been requested by another user belonging to the same group as the user using the terminal device, a reply status based on the transmission result is displayed.
Communication system. an image communication device that transmits and receives images to and from other devices by performing facsimile communication;
A control method for a communication system including a server device that provides a service for managing a group made up of one or more users who use the image communication device, and a terminal device that can communicate with the image communication device via the server device,
The image communication device
When the image is received from the other device, the server device transmits received data including the image and information about the image to the server device;
In response to receiving reply data in response to the received data from the server device, transmitting a reply image included in the reply data to the other device;
transmitting a transmission result of the reply image to the server device so that a reply status based on the transmission result of the reply image can be displayed on the terminal device in a manner that can be shared among one or more users belonging to the group;
The terminal device
In response to receiving from the server device a transmission result of the reply image for which execution of a reply has been requested by another user belonging to the same group as the user using the terminal device, a reply status based on the transmission result is displayed.
A method for controlling a communication system.