JP7782233B2 - Information processing system, imaging device, method, and program - Google Patents

Description

The present invention relates to an information processing system, an imaging device, a method, and a program.

Conventionally, imaging devices capable of transferring captured images to external devices are known. However, such imaging devices sometimes fail to transfer images. For example, Patent Document 1 describes a device that, if it detects that the battery output voltage is below a predetermined voltage while sending a file, saves any unsent image data and then turns off the power, and then transmits the saved, unsent image data when the power is turned back on.

However, there are various reasons why an imaging device may be unable to transfer an image, and it would be desirable to be able to retransfer the image regardless of the reason.

The present invention aims to transfer images captured by an imaging device that have not yet been transferred.

An information processing system according to one embodiment of the present invention is an information processing system including an imaging device with communication capabilities and a server that executes jobs of a workflow application linked to the imaging device , wherein the imaging device is equipped with an image status management unit that manages information indicating that images captured by the imaging device have not been transferred , and, if execution of the job is completed successfully, deletes the images that are the subject of the job from a list of untransferred images, and, if execution of the job is not completed successfully, keeps the images that are the subject of the job in the list of untransferred images ; and an image transfer control unit that controls the retransfer of untransferred images to the server in response to one of a plurality of retransfer triggers.

According to the present invention, it is possible to transfer images captured by an imaging device that have not yet been transferred.

FIG. 1 is a diagram for explaining an overview of a service according to an embodiment of the present invention. 1 is a diagram illustrating an overall configuration according to an embodiment of the present invention. 1 is a hardware configuration diagram of an information processing device (server) according to an embodiment of the present invention. FIG. 1 is a diagram illustrating a hardware configuration of a device (omnidirectional imaging device) according to an embodiment of the present invention. FIG. 2 is a functional block diagram of a server and a device according to an embodiment of the present invention. FIG. 10 is a sequence diagram of a process for associating a device with an application according to an embodiment of the present invention. FIG. 2 is a sequence diagram of a process related to a basic operation according to an embodiment of the present invention. FIG. 10 is a sequence diagram of a management process for a transferred/non-transferred state of an image according to an embodiment of the present invention. FIG. 10 is a sequence diagram of a retransfer process when a retransfer trigger occurs according to an embodiment of the present invention. FIG. 10 is a sequence diagram of processing relating to a photography priority mode according to an embodiment of the present invention. 10 is an example of a screen according to an embodiment of the present invention.

Embodiments of the present disclosure will be described below with reference to the drawings.

FIG. 1 is a diagram illustrating an overview of a service according to one embodiment of the present invention. A user (a customer user, A1 in FIG. 1) uses a device 20 to access cloud services provided by servers 11-15 (hereinafter also referred to as server 10). A device partner (B in FIG. 1) registers a workflow (WF) application to the tenant (a group of devices using cloud services) to which the user belongs. A workflow (WF) application developer (C (cloud service partner) in FIG. 1) uses a WF application development tool to develop a workflow (WF) application that allows QR Code (registered trademark) integration settings and sets the type of data that can be input (e.g., still images or videos). A2 is the customer's administrator, and D is the device vendor. The following description uses a cloud service provided by server 10 as an example of a service provided to device 20 via a network. However, the service used by device 20 is not limited to this; it may be any service that utilizes functions provided via a network. For example, by installing server 10 within the same network as device 20, server 10 may provide on-premise services to device 20.

<System Configuration>
2 is a diagram showing the overall configuration of an embodiment of the present invention. A system (also referred to as an information processing system) 1 includes a server 10 and a device 20. The server 10 and the device 20 are connected to each other so as to be able to communicate with each other via a communication network N. Each of these will be described below.

<<Server>>
A server (an example of an information processing device) 10 performs processing to link an application provided by a cloud service with a device 20 .

The devices described in the examples represent only one of several computing environments for implementing the embodiments disclosed herein. In one embodiment, server 10 includes multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communications link, including a network, shared memory, etc., and to perform the processes disclosed herein.

<<Devices>>
The device 20 is an imaging device with a communication function. The device 20 is a device used by a user who uses a cloud service. For example, the device 20 is a device (e.g., an omnidirectional imaging device) that has an imaging function but does not have a display function.

Note that device 20 is not limited to a spherical imaging device, as long as it is a device with a communication function. Device 20 may be, for example, a PJ (Projector), an IWB (Interactive White Board: an electronic whiteboard with a blackboard function that allows for mutual communication), an output device such as digital signage, a HUD (Head Up Display) device, industrial machinery, an imaging device, a sound collection device, medical equipment, a network home appliance, an automobile (Connected Car), a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or a desktop PC.

<Hardware configuration>
FIG. 3 is a hardware configuration diagram of an information processing device (server) 10 according to an embodiment of the present invention.

As shown in Figure 3, the server 10 is constructed by a computer and includes a CPU 1001, ROM 1002, RAM 1003, HD 1004, HDD (Hard Disk Drive) controller 1005, display 1006, external device connection I/F (Interface) 1007, network I/F 1008, data bus 1009, keyboard 1010, pointing device 1011, DVD-RW (Digital Versatile Disk Rewritable) drive 1013, and media I/F 1015.

Of these, CPU 1001 controls the overall operation of server 10. ROM 1002 stores programs used to drive CPU 1001, such as IPL. RAM 1003 is used as a work area for CPU 1001. HD 1004 stores various data, such as programs. HDD controller 1005 controls the reading and writing of various data from HD 1004 under the control of CPU 1001. Display 1006 displays various information, such as a cursor, menu, window, text, or image. External device connection I/F 1007 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. Network I/F 1008 is an interface for data communication using communication network N. Bus line 1009 is an address bus, data bus, etc. for electrically connecting each component, such as CPU 1001, shown in Figure 3.

The keyboard 1010 is a type of input device equipped with multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 1011 is a type of input device for selecting and executing various instructions, selecting a processing target, moving the cursor, etc. The DVD-RW drive 1013 controls the reading and writing of various data from the DVD-RW 1012, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R or the like. The media I/F 1015 controls the reading and writing (storing) of data from the recording medium 1014, such as a flash memory.

Figure 4 is a hardware configuration diagram of a device (in the case of an omnidirectional imaging device) 20 according to one embodiment of the present invention.

In the following, omnidirectional imaging device 20 is referred to as an omnidirectional (all-round) omnidirectional imaging device using two image sensors, but any number of image sensors greater than or equal to two may be used. Furthermore, it does not necessarily have to be a device dedicated to omnidirectional imaging; a regular digital camera, smartphone, or the like may be equipped with an omnidirectional imaging unit as an add-on, so that it has substantially the same functionality as omnidirectional imaging device 20.

As shown in FIG. 4, the omnidirectional imaging device 20 is composed of an imaging unit 2001, an image processing unit 2004, an imaging control unit 2005, a microphone 2006, a sound processing unit 2007, a CPU (Central Processing Unit) 2009, a ROM (Read Only Memory) 2010, an SRAM (Static Random Access Memory) 2011, a DRAM (Dynamic Random Access Memory) 2012, an operation unit 2013, an external device connection I/F 2014, a communication unit 2015, an antenna 2015a, an electronic compass, a gyro sensor, an acceleration sensor, and a concave terminal for a Micro USB.

Of these, imaging unit 2001 is equipped with wide-angle lenses (so-called fisheye lenses) 2002a and 2002b, each with a field angle of 180° or more for forming a hemispherical image, and two imaging elements 2003a and 2003b provided to correspond to each wide-angle lens. Imaging elements 2003a and 2003b each include an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor that converts the optical image captured by fisheye lenses 2002a and 2002b into image data in the form of an electrical signal for output, a timing generation circuit that generates horizontal and vertical synchronization signals and pixel clocks for the image sensors, and a group of registers that set various commands and parameters required for the operation of the imaging elements.

The imaging elements 2003a and 2003b of the imaging unit 2001 are each connected to the image processing unit 2004 via a parallel I/F bus. On the other hand, the imaging elements 2003a and 2003b of the imaging unit 2001 are connected to the imaging control unit 2005 via a serial I/F bus (such as an I2C bus). The image processing unit 2004, imaging control unit 2005, and sound processing unit 2007 are connected to the CPU 2009 via a bus 2008. Also connected to the bus 2008 are a ROM 2010, an SRAM 2011, a DRAM 2012, an operation unit 2013, an external device connection I/F (Interface) 2014, a communication unit 2015, and an acceleration/direction sensor 2016.

The image processing unit 2004 takes in the image data output from the image sensors 2003a and 2003b via a parallel I/F bus, performs predetermined processing on each piece of image data, and then synthesizes the image data to create equirectangular projection image data.

The imaging control unit 2005 generally sets commands and the like in the registers of the imaging elements 2003a and 2003b using the I2C bus, with the imaging control unit 2005 acting as the master device and the imaging elements 2003a and 2003b acting as slave devices. Necessary commands and the like are received from the CPU 2009. The imaging control unit 2005 also uses the I2C bus to retrieve status data and the like from the registers of the imaging elements 2003a and 2003b and send it to the CPU 2009.

In addition, the imaging control unit 2005 instructs the image sensors 2003a and 2003b to output image data when the shutter button on the operation unit 2013 is pressed. Some omnidirectional imaging devices 20 have a preview display function or a video display function on a display (for example, a smartphone display). In this case, the image data is output continuously from the image sensors 2003a and 2003b at a predetermined frame rate (frames/minute).

In addition, as described below, the imaging control unit 2005 also functions as a synchronization control means that cooperates with the CPU 2009 to synchronize the output timing of image data from the image sensors 2003a and 2003b. Note that in this embodiment, the omnidirectional imaging device 20 is not provided with a display, but a display unit may be provided.

The microphone 2006 converts sound into sound (signal) data. The sound processing unit 2007 receives the sound data output from the microphone 2006 via the I/F bus and performs predetermined processing on the sound data.

The CPU 2009 controls the overall operation of the omnidirectional imaging device 20 and executes necessary processing. The ROM 2010 stores various programs for the CPU 2009. The SRAM 2011 and DRAM 2012 are work memories that store programs executed by the CPU 2009 and data currently being processed. In particular, the DRAM 2012 stores image data currently being processed by the image processing unit 2004 and data of processed equirectangular projection images.

The operation unit 2013 is a general term for operation buttons such as the shutter button. The user operates the operation unit 2013 to input various shooting modes and shooting conditions.

The external device connection I/F 2014 is an interface for connecting various external devices. In this case, external devices include, for example, a USB (Universal Serial Bus) memory or a PC (Personal Computer). The equirectangular projection image data stored in the DRAM 2012 is recorded on external media via this network I/F 2014, or, if necessary, transmitted via the network I/F 2014 to an external terminal (device) such as a smartphone.

The communication unit 2015 communicates with an external terminal (device) such as a smartphone using short-range wireless communication technology such as Wi-Fi, NFC (Near Field Communication), or Bluetooth (registered trademark) via an antenna 2015a provided in the omnidirectional imaging device 20. This communication unit 2015 can also transmit data of the equirectangular projection image to an external terminal (device) such as a smartphone.

The acceleration and orientation sensor 2016 calculates the orientation of the omnidirectional imaging device 20 from the Earth's magnetism and outputs orientation information. This orientation information is an example of related information (metadata) conforming to Exif standards and is used for image processing such as image correction of captured images. The related information also includes data such as the date and time the image was captured and the data size of the image data. The acceleration and orientation sensor 2016 detects angle changes (roll angle, pitch angle, and yaw angle) associated with the movement of the omnidirectional imaging device 20. The angle changes are an example of related information (metadata) conforming to Exif standards and are used for image processing such as image correction of captured images. The acceleration and orientation sensor 2016 detects acceleration in three axial directions. The omnidirectional imaging device 20 calculates the attitude (angle relative to the direction of gravity) of its own device (the omnidirectional imaging device 20) based on the acceleration detected by the acceleration and orientation sensor 2016. Providing the acceleration and orientation sensor 2016 in the omnidirectional imaging device 20 improves the accuracy of image correction.

<Function block>
FIG. 5 is a functional block diagram of the server 10 and the device 20 according to an embodiment of the present invention.

<<Server>>
The server 10 includes an application setting unit 101, a device registration unit 102, a collaboration unit 103, a storage unit 104, and a WF application 105. The server 10 also functions as the application setting unit 101, the device registration unit 102, the collaboration unit 103, and the WF application 105 by executing a program.

The application setting unit 101 configures the WF application 105. For example, the application setting unit 101 configures an account for logging in to an online storage service.

The device registration unit 102 provides services that connect the device 20 to other devices and cloud services.

The collaboration unit 103 manages the association between the WF application 105 and the device 20, and upon receiving a request from the device 20 to execute a WF (workflow) job, requests the WF application 105 to execute the job.

The memory unit 104 is a storage unit where images are temporarily stored.

The WF (workflow) application 105 is an application that combines functions to realize a workflow. It defines what input is to be processed, how it is to be processed, and how it is to be output.

<<Devices>>
The device 20 includes an image capturing unit 201, an image storage unit 202, an image transfer control unit 203, an image status management unit 204, a communication unit 205, and a setting value storage unit 206. The device 20 also functions as the image capturing unit 201, the image transfer control unit 203, the image status management unit 204, and the communication unit 205 by executing a program.

The photographing unit 201 takes a photograph when the shutter button on the device 20 is pressed.

The image storage unit 202 is a storage unit for saving images captured by the image capture unit 201. Images transferred to the server 10 are deleted.

The image transfer control unit 203 controls the transfer of images. Specifically, the image transfer control unit 203 controls the retransfer of untransferred images to the server 10 in response to one of multiple retransfer triggers.

The image status management unit 204 manages information indicating that images stored in the image storage unit 202 (i.e., images captured by the imaging device 20) have not yet been transferred.

The communication unit 205 transfers images and notifies the completion of image transfer in response to requests from the image transfer control unit 203.

The setting value storage unit 206 is a storage unit that stores the setting values for the operation of the plug-in (specifically, the threshold value for switching the capture priority mode ON/OFF).

<Processing method>
6 is a sequence diagram of a process for associating a device with an application according to an embodiment of the present invention, which will be explained separately in a <WF application registration phase> and a <WF application execution phase>.

<WF application registration phase>
The following describes a process of registering a workflow (WF) application 105 provided by a cloud service in association with the device 20. The workflow (WF) application 105 is an application on the server 10 that performs predetermined processing.

It is assumed that the developer of the WF application 105 (C (cloud service partner) in Figure 1) has developed the WF application 105 using a WF application development tool, setting the QR code integration settings as allowed and the data types that can be input (still images, video, etc.). Furthermore, the device partner (B in Figure 1) registers the WF application 105 in the tenant (a group of devices that use cloud services) to which the user belongs.

In step 11 (S11), the server (application setting unit) 10 generates and distributes a code (e.g., a QR code) containing information about the application provided by the cloud service and information used to register the device.

Specifically, the user logs in to a site (IoT site in Figure 1) where applications are configured using a PC (personal computer), selects the WF application 105 to be linked with the device 20 (device plug-in in Figure 1), and has a QR code issued.

In step 12 (S12), it is assumed that the shutter button on device 20 is pressed.

In step 13 (S13), the device 20 reads the QR code that was distributed by the server (application setting unit) 10 in S11 and displayed on the PC.

In step 14 (S14), device 20 acquires the connection destination of the site for obtaining information for identifying the device, a token (access token) for accessing the site, and information about the application (e.g., application ID (here, "App 1") and input type (here, "still image")) contained in the QR code.

In step 15 (S15), the device 20 requests the server (device registration unit) 10 to register the device 20. Specifically, the device 20 notifies the server 10 of the information used for device registration obtained by reading the code in S14 (more specifically, the device 20's serial number is sent to the connection destination using an access token).

In step 16 (S16), the server (device registration unit) 10 notifies the device 20 of information for identifying the device (device ID (here, "Device A")) and the certificate.

In step 17 (S17), the device 20 requests the server (collaboration unit) 10 to associate the device 20 with the application and register it. Specifically, the device 20 notifies the server (collaboration unit) 10 of the application ID (i.e., "App 1"), input type (i.e., "Still Image"), and device ID (i.e., "Device A") along with the certificate of S16. The server (collaboration unit) 10 then associates and registers the application ID, input type, and device ID.

<WF application execution phase>
The process of executing a workflow (WF) application will now be described.

In step 21 (S21), it is assumed that the shutter button on device 20 is pressed.

In step 22 (S22), the device 20 obtains a uniform resource locator (URL). Specifically, the device 20 obtains the URL from the server (device registration unit) 10.

In step 23 (S23), device 20 uploads the image taken in S21 to the URL in S22.

In step 24 (S24), the device 20 notifies the server (device registration unit) 10 of the image capture. Specifically, the device 20 notifies the server (device registration unit) 10 of the device ID (i.e., "Device A") and URL.

In step 25 (S25), the server (device registration unit) 10 sends the device ID and URL obtained in S24 to the server (collaboration unit) 10.

In step 26 (S26), the server (collaboration unit) 10 downloads the image uploaded to the URL obtained in S25.

In step 27 (S27), the server (collaboration unit) 10 causes the workflow (WF) application to execute the job. Specifically, the server (collaboration unit) 10 notifies the application linked to the device ID (i.e., "App 1") of the input type (i.e., "still image").

In step 28 (S28), the WF application 105 uploads the image to an online storage service. Note that uploading and saving images to an online storage service is an example of a service provided via a network.

Figure 7 is a sequence diagram of processing related to basic operations in one embodiment of the present invention.

In step 101 (S101), when the shutter button on the device 20 is pressed, the image capturing unit 201 of the device 20 captures an image.

In step 102 (S102), the image capturing unit 201 of the device 20 stores the image captured in S101 in the image storage unit 202 of the device 20.

In step 103 (S103), the image capturing unit 201 of the device 20 notifies the image transfer control unit 203 of the device 20 that an image has been captured. Specifically, the image capturing unit 201 notifies the image transfer control unit 203 of the image ID.

In step 104 (S104), in response to the notification in S103, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the file name.

In step 105 (S105), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to obtain the URL to which the image will be uploaded.

In step 106 (S106), the communication unit 205 of the device 20 obtains the URL requested in S105.

Here, we will explain the URL to which images are uploaded. The communication unit 205 of the device 20 acquires the URL by specifying the image file name. In other words, a URL is acquired for each image file to be uploaded. The URL to which images are uploaded is a temporary area, and becomes an invalid URL after a certain period of time has passed, and is deleted if an image has been uploaded.

In step 107 (S107), the communication unit 205 of the device 20 uploads the image to the memory unit 104 of the server 10 (URL of S106).

In step 108 (S108), when the upload in S107 is complete, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to execute a WF (workflow) job. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the URL.

In step 109 (S109), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to execute a WF (workflow) job. Specifically, the communication unit 205 notifies the device registration unit 102 of the URL and device ID.

In step 110 (S110), the device registration unit 102 of the server 10 requests the collaboration unit 103 of the server 10 to execute a WF (workflow) job. Specifically, the device registration unit 102 notifies the collaboration unit 103 of the URL and device ID.

In step 111 (S111), the collaboration unit 103 of the server 10 downloads an image from the URL that is a parameter of the request to execute the WF (workflow) job.

In step 112 (S112), the collaboration unit 103 of the server 10 requests the WF application 105 to execute the job. Specifically, the collaboration unit 103 notifies the WF application 105 of the app ID and the image.

The collaboration unit 103 of the server 10 manages which devices 20 are linked to which WF (workflow) applications through the application linking registration performed in the <WF application registration phase> of Figure 6, and executes the WF (workflow) application linked to the device 20 that has requested the execution of a WF (workflow) job.

In step 113 (S113), the WF application 105 uploads the image from the device 20 to the online storage service in accordance with a workflow that uploads the image to the online storage service upon receiving the image from the device 20.

In step 114 (S114), the collaboration unit 103 of the server 10 returns a WF (workflow) job acceptance response indicating whether acceptance of the execution of the WF job was successful. If acceptance was successful, a job ID for identifying the job is notified as a parameter.

In step 115 (S115), the device registration unit 102 of the server 10 notifies the communication unit 205 of the device 20 of the WF job acceptance response.

In step 116 (S116), the communication unit 205 of the device 20 notifies the image transfer control unit 203 of the device 20 of the WF job acceptance response.

In step 117 (S117), the image transfer control unit 203 of the device 20 requests the WF application 105 to obtain the job status at arbitrary intervals in response to the WF job acceptance response, and monitors whether the job is completed successfully. Specifically, the image transfer control unit 203 notifies the WF application 105 of the job ID.

In step 118 (S118), the image transfer control unit 203 of the device 20 acquires the job status. S117 to S118 are repeated until the job is completed.

Figure 8 is a sequence diagram of the process for managing the transferred/untransferred status of images according to one embodiment of the present invention.

In step 201 (S201), when the shutter button on the device 20 is pressed, the image capturing unit 201 of the device 20 captures an image.

In step 202 (S202), the image capturing unit 201 of the device 20 stores the image captured in S201 in the image storage unit 202 of the device 20.

In step 203 (S203), the image capturing unit 201 of the device 20 notifies the image transfer control unit 203 of the device 20 that an image has been captured. Specifically, the image capturing unit 201 notifies the image transfer control unit 203 of the image ID.

In step 204 (S204), the image transfer control unit 203 of the device 20 requests the image status management unit 204 of the device 20 to manage the transferred/untransferred status of the image. Specifically, the image transfer control unit 203 notifies the image status management unit 204 of the image ID.

In step 205 (S205), the image status management unit 204 of the device 20 adds the image ID of S204 to the list of untransferred images.

In step 206 (S206), in response to the notification in S203, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the file name.

In step 207 (S207), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to obtain the URL to which the image will be uploaded.

In step 208 (S208), the communication unit 205 of the device 20 obtains the URL requested in S207.

Here, we will explain the URL to which images are uploaded. The communication unit 205 of the device 20 acquires the URL by specifying the image file name. In other words, a URL is acquired for each image file to be uploaded. The URL to which images are uploaded is a temporary area, and becomes an invalid URL after a certain period of time has passed, and is deleted if an image has been uploaded.

In step 209 (S209), the communication unit 205 of the device 20 uploads the image to the memory unit 104 of the server 10 (URL of S208).

In step 210 (S210), when the upload in S209 is complete, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to execute a WF (workflow) job. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the URL.

In step 211 (S211), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to execute a WF (workflow) job. Specifically, the communication unit 205 notifies the device registration unit 102 of the URL and device ID.

In step 212 (S212), the device registration unit 102 of the server 10 requests the collaboration unit 103 of the server 10 to execute a WF (workflow) job. Specifically, the device registration unit 102 notifies the collaboration unit 103 of the URL and device ID.

In step 213 (S213), the collaboration unit 103 of the server 10 downloads the image from the URL that is a parameter of the request to execute the WF (workflow) job.

In step 214 (S214), the collaboration unit 103 of the server 10 requests the WF application 105 to execute the job. Specifically, the collaboration unit 103 notifies the WF application 105 of the app ID and the image.

The collaboration unit 103 of the server 10 manages which devices 20 are linked to which WF (workflow) applications through the application linking registration performed in the <WF application registration phase> of Figure 6, and executes the WF (workflow) application linked to the device 20 that has requested the execution of a WF (workflow) job.

In step 215 (S215), the WF application 105 uploads the image from the device 20 to the online storage service in accordance with a workflow that uploads the image to the online storage service upon receiving the image from the device 20.

In step 216 (S216), the collaboration unit 103 of the server 10 returns a WF (workflow) job acceptance response indicating whether acceptance of the execution of the WF job was successful. If acceptance was successful, a job ID for identifying the job is notified as a parameter.

In step 217 (S217), the device registration unit 102 of the server 10 notifies the communication unit 205 of the device 20 of the WF job acceptance response.

In step 218 (S218), the communication unit 205 of the device 20 notifies the image transfer control unit 203 of the device 20 of the WF job acceptance response.

In step 219 (S219), the image transfer control unit 203 of the device 20 requests the WF application 105 to obtain the job status at arbitrary intervals in response to the WF job acceptance response, and monitors whether the job is completed successfully. Specifically, the image transfer control unit 203 notifies the WF application 105 of the job ID.

In step 220 (S220), the image transfer control unit 203 of the device 20 acquires the job status. S219 to S220 are repeated until the job is completed.

In step 221 (S221), when execution of the WF (workflow) job is completed successfully, the image transfer control unit 203 of the device 20 requests the image status management unit 204 to delete the images that are the subject of the job from the list of untransferred images.

In step 222 (S222), the image status management unit 204 of the device 20 deletes the image from the list of untransferred images in response to the request of S221. Note that if the execution of the WF (workflow) job does not end normally due to a network error, failure, an error or failure on the cloud service side, etc., the image is not deleted from the list of untransferred images, and the untransferred status is maintained.

Figure 9 is a sequence diagram of the retransfer process when a retransfer trigger occurs in one embodiment of the present invention.

Let's assume that a retransfer trigger occurs in step 301 (S301).

Here, we will explain the triggers for retransfer. For example, retransfer is triggered the next time an image is taken (i.e., when an image is taken after an untransferred image has occurred), the next time the plug-in is launched (i.e., when the plug-in is launched after an untransferred image has occurred), when a network connection is detected (i.e., when a connection between the network and device 20 is detected after an untransferred image has occurred), when the imaging device 20 is operated (i.e., when the imaging device 20 is operated after an untransferred image has occurred), when the remaining capacity of the image storage unit 202 of the imaging device 20 falls below a threshold (i.e., when the remaining capacity of the image storage unit 202 falls below a threshold after an untransferred image has occurred), or when a transfer request is received from the server 10 (i.e., when a transfer request is received from the server 10 after an untransferred image has occurred).

In step 302 (S302), the image transfer control unit 203 of the device 20 requests the image status management unit 204 to obtain a list of images that have not yet been transferred.

In step 303 (S303), the image transfer control unit 203 of the device 20 obtains a list of images that have not yet been transferred.

Steps 304 to 320 are then repeated for each untransferred image, and the untransferred images are transferred sequentially.

In step 304 (S304), the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the device 20 of the file name.

In step 305 (S305), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to obtain the URL to which the image will be uploaded.

In step 306 (S306), the communication unit 205 of the device 20 obtains the URL requested in S305.

Here, we will explain the URL to which image files are uploaded. The communication unit 205 of the device 20 acquires the URL by specifying the image file name. In other words, a URL is acquired for each image file to be uploaded. The URL to which images are uploaded is a temporary area, and becomes an invalid URL after a certain period of time has passed, and is deleted if an image has been uploaded.

In step 307 (S307), the communication unit 205 of the device 20 uploads the image to the memory unit 104 of the server 10 (URL of S306).

In step 308 (S308), when the upload in S107 is complete, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to execute a WF (workflow) job. Specifically, the image transfer control unit 203 notifies the communication unit 205 of the URL.

In step 309 (S309), the communication unit 205 of the device 20 requests the device registration unit 102 of the server 10 to execute a WF (workflow) job. Specifically, the communication unit 205 notifies the device registration unit 102 of the device 20 of the URL and device ID.

In step 310 (S310), the device registration unit 102 of the server 10 requests the collaboration unit 103 of the server 10 to execute a WF (workflow) job. Specifically, the device registration unit 102 notifies the collaboration unit 103 of the URL and device ID.

In step 311 (S311), the collaboration unit 103 of the server 10 downloads an image from the URL that is a parameter of the request to execute the WF (workflow) job.

In step 312 (S312), the collaboration unit 103 of the server 10 requests the WF application 105 to execute the job. Specifically, the collaboration unit 103 notifies the WF application 105 of the app ID and the image.

The collaboration unit 103 of the server 10 manages which devices 20 are linked to which WF (workflow) applications through the application linking registration performed in the <WF application registration phase> of Figure 6, and executes the WF (workflow) application linked to the device 20 that has requested the execution of a WF (workflow) job.

In step 313 (S313), the WF application 105 uploads the image from the device 20 to the online storage service in accordance with a workflow that uploads the image to the online storage service upon receiving the image from the device 20.

In step 314 (S314), the collaboration unit 103 of the server 10 returns a WF (workflow) job acceptance response indicating whether acceptance of the execution of the WF job was successful. If acceptance was successful, a job ID for identifying the job is notified as a parameter.

In step 315 (S315), the device registration unit 102 of the server 10 notifies the communication unit 205 of the device 20 of the WF job acceptance response.

In step 316 (S316), the communication unit 205 of the device 20 notifies the image transfer control unit 203 of the device 20 of the WF job acceptance response.

In step 317 (S317), the image transfer control unit 203 of the device 20 requests the WF application 105 to obtain the job status at arbitrary intervals in response to the WF job acceptance response, and monitors whether the job is completed successfully. Specifically, the image transfer control unit 203 notifies the WF application 105 of the job ID.

In step 318 (S318), the image transfer control unit 203 of the device 20 acquires the job status. S317 to S318 are repeated until the job is completed.

In step 319 (S319), when execution of the WF (workflow) job is completed successfully, the image transfer control unit 203 of the device 20 requests the image status management unit 204 to delete the images that are the subject of the job from the list of untransferred images.

In step 318 (S318), the image transfer control unit 203 of the device 20 deletes the image from the list of untransferred images in response to the request of S319.

Figure 10 is a sequence diagram of processing related to the capture priority mode (a mode in which images are not transferred after capture) in one embodiment of the present invention.

In step 401 (S401), when the shutter button on the device 20 is pressed, the image capturing unit 201 of the device 20 captures an image.

In step 402 (S402), the image capturing unit 201 of the device 20 stores the image captured in S401 in the image storage unit 202 of the device 20.

In step 403 (S403), the image capturing unit 201 of the device 20 notifies the image transfer control unit 203 of the device 20 that an image has been captured. Specifically, the image capturing unit 201 notifies the image transfer control unit 203 of the image ID.

In step 404 (S404), the image transfer control unit 203 of the device 20 requests the image status management unit 204 of the device 20 to manage the transferred/untransferred status of the image. Specifically, the image transfer control unit 203 notifies the image status management unit 204 of the image ID.

In step 405 (S405), the image status management unit 204 of the device 20 adds the image ID of S404 to the list of untransferred images.

In step 406 (S406), the image transfer control unit 203 of the device 20 checks the remaining battery power of the device 20.

In step 407 (S407), the image transfer control unit 203 of the device 20 requests the setting value storage unit 206 to acquire the threshold setting value for the capture priority mode (note that the device 20 operates in capture priority mode when the remaining battery charge falls below the threshold, and transfers untransferred images when the remaining battery charge exceeds the threshold due to charging).

In step 408 (S408), the image transfer control unit 203 of the device 20 acquires the threshold setting value for the capture priority mode.

In step 409 (S409), the image transfer control unit 203 of the device 20 compares the remaining battery level referenced in S406 with the threshold setting for the capture priority mode obtained in S408.

In step 410 (S410), if the comparison in S409 shows that the mode is not the image capture priority mode, the image transfer control unit 203 of the device 20 requests the communication unit 205 of the device 20 to transfer the image (the operation thereafter is as described in FIG. 8). On the other hand, if the comparison in S409 shows that the mode is the image capture priority mode, the image is not transferred and the untransferred state is maintained.

<Screen example>
Fig. 11 shows an example of a screen according to an embodiment of the present invention. Fig. 11 shows a setting screen for a plug-in that captures an image and executes a workflow. Settings are made from an application on a smartphone, tablet, or the like that is connected to the device 20 wirelessly (via LAN, Bluetooth, or the like). The value set by the application (threshold setting value) is saved in a setting value storage unit 206 provided in the device 20.

The user can set the threshold for capture-priority mode (i.e., the conditions for switching to capture-priority mode) on the screen shown in Figure 11. The device 20 operates in capture-priority mode when the remaining battery power falls below the threshold, and transfers any untransferred images when the remaining battery power exceeds the threshold due to charging.

<Effects>
Thus, in one embodiment of the present invention, the imaging device can retransfer images that could not be transferred for some reason at a timing desired by the user (for example, in response to the occurrence of one of multiple retransfer triggers (i.e., when the cause of the image not being able to be transferred is resolved)).

The functions of the above-described embodiments can be realized by one or more processing circuits. In this specification, the term "processing circuit" includes processors programmed to perform functions by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to perform the above-described functions.

Note that the present invention is not limited to the configurations described in the above embodiments, and may be combined with other elements. These aspects may be modified without departing from the spirit of the present invention, and may be determined appropriately depending on the application form.

1 System 10 Server 20 Device 101 Application setting unit 102 Device registration unit 103 Collaboration unit 104 Storage unit 105 WF application 201 Photography unit 202 Image storage unit 203 Image transfer control unit 204 Image status management unit 205 Communication unit 206 Setting value storage unit

Japanese Patent Application Laid-Open No. 2005-020452

Claims (7)

An information processing system including an imaging device having a communication function and a server that executes a job of a workflow application linked to the imaging device ,
The imaging device is
managing information indicating that the image captured by the imaging device has not yet been transferred ;
an image status management unit that deletes the image that is the target of the job from a list of untransferred images when the execution of the job is completed normally, and that keeps the image that is the target of the job in the list of untransferred images when the execution of the job is not completed normally;
an image transfer control unit that controls the retransfer of untransferred images to the server in response to one of a plurality of retransfer triggers. The information processing system of claim 1, wherein the trigger for the retransfer is the next time an image is captured, the next time a plug-in is started, when a network connection is detected, when the imaging device is operated, when the remaining capacity of the image storage unit of the imaging device falls below a threshold, or when a transfer request is received from the server. The information processing system of claim 1 or 2, wherein the imaging device operates in a capture-priority mode in which images are not transferred after capture when the remaining battery charge of the imaging device falls below a threshold. The information processing system of claim 3, wherein the conditions for transitioning to the capture-priority mode are set by the user. managing information indicating that an image captured by the imaging device has not yet been transferred ;
an image status management unit that, when the server has normally completed execution of a job of a workflow application linked to the imaging device, deletes the image that is the target of the job from a list of untransferred images, and, when the server has not normally completed execution of the job, keeps the image that is the target of the job in the list of untransferred images ;
and an image transfer control unit that controls the retransfer of untransferred images in response to one of a plurality of retransfer triggers. A method performed by an imaging device, comprising:
managing information indicating that the image captured by the imaging device has not yet been transferred ;
a step of deleting an image that is the target of a job from a list of untransferred images when the server has normally completed execution of a job of a workflow application linked to the imaging device, and retaining the image that is the target of the job in the list of untransferred images when the server has not normally completed execution of the job ;
and controlling the retransmission of untransmitted images in response to any of a plurality of triggers for retransmission. an imaging device; managing information indicating that an image captured by the imaging device has not yet been transferred ;
an image status management unit that, when the server has normally completed execution of a job of a workflow application linked to the imaging device, deletes the image that is the subject of the job from a list of untransferred images, and, when the server has not normally completed execution of the job, keeps the image that is the subject of the job in the list of untransferred images ;
A program for causing the image transfer control unit to function as an image transfer control unit that controls the retransfer of untransferred images in response to one of a plurality of retransfer triggers.