JP7604859B2 - IMAGE MANAGEMENT SYSTEM, IMAGE REGISTRATION MANAGEMENT DEVICE, IMAGE REGISTRATION MANAGEMENT METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to an image management system, an image registration management device, an image registration management method, and a program.

In recent years, there have been systems that acquire dynamic location information of people and objects and display it as a mark on an electronic map. Such systems are used to manage the locations of people and objects, but in order to make it easier to intuitively understand which room a person or object is in, there is already known technology that registers floor map images in association with actual coordinates based on latitude and longitude, etc., and displays their locations on any floor map image.

Furthermore, a technology is known that combines map supplementary information, which is composed of registration point position information indicating the reference for where a map image based on map image data corresponds to an actual geographical location, a scale value that is the scale at which the map image is displayed on the display screen 15a of the display 15, and an azimuth angle deviation indicating the deviation between the reference direction of the map image and the actual north direction, with map image data to obtain map data MD that can be used for navigation and display the vehicle's current position in real time (see, for example, Patent Document 1).

However, in conventional technology, when displaying device location information of managed devices placed in a specific space and an overhead image obtained by looking down on the specific space on an electronic map, there was a problem in that in order to correctly grasp the device location information of the managed devices, the user had to register an overhead image that matched the aspect ratio of the electronic map.

In order to solve the above-mentioned problems, the invention according to claim 1 provides an image management system having an image registration management device that registers and manages aerial images obtained by looking down on a predetermined space, and a registered image display system that communicates with the image registration management device to assist in the registration of the aerial images and display the aerial images as an electronic map, the image management system having a reception means that receives input of first position information indicating the positions of any two points on the aerial image, and a display control means that displays the aerial image on a display means, the display control means being configured to receive the first position information received by the reception means, and based on second location information for displaying a specified electronic map corresponding to the first location information, obtains a corrected overhead image in which the aspect ratio of the overhead image is corrected to match the aspect ratio of an image related to the specified electronic map, using image identification information included in a display request for device location information of a managed device arranged in the specified space as a search key, acquires the corrected overhead image and the device location information, creates an electronic map including the overhead image and the device location information of the managed device, and displays the corrected overhead image together with the device location information on the display means.

As described above, the present invention has the effect of enabling a user to correctly grasp the device location information of managed devices placed in a specified space without the user having to register an overhead image that matches the aspect ratio of the electronic map.

1 is a diagram illustrating an overall configuration of an example of an image communication system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a floor map image. FIG. 13 is a diagram showing an example of a case where a predetermined angle exists between two coordinate systems. FIG. 13 is a diagram showing an example of a floor map image with a corrected aspect ratio. FIG. 13 is a diagram showing another example of a case where a predetermined angle exists between two coordinate systems. 11A to 11C are diagrams illustrating specific examples of aspect ratio correction and image rotation processing. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a managed device according to the embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a position information providing device according to the present embodiment. FIG. 1 is a diagram illustrating an example of a functional configuration of an image communication system according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an example of a functional configuration of an image communication system according to an embodiment of the present invention. 1A is a conceptual diagram showing an example of a floor map management table according to the embodiment, and FIG. 1B is a conceptual diagram showing an example of a target device management table according to the embodiment. 11 is a conceptual diagram showing an example of an actual coordinate information management table according to the embodiment. FIG. FIG. 11 is a sequence diagram showing an example of a registration process of a floor map image according to the present embodiment. FIG. 11 is a sequence diagram showing an example of a registration process of a floor map image according to the present embodiment. FIG. 11 is a sequence diagram showing an example of a registration process of a floor map image according to the present embodiment. FIG. 11 is a sequence diagram showing an example of a registration process of position information according to the embodiment. FIG. 11 is a sequence diagram showing an example of a display process of a floor map image according to the present embodiment. 13A is an example of a screen display when uploading a floor map image, and FIG. 13B is an example of a screen display when registering a current location. 13A is another example of a screen display when registering a current location, and FIG. 13B is an example of a screen display when registering another current location. 13A is an example of another screen display when registering another current location, and FIG. 13B is an example of a screen display when inputting and registering a floor map name. 13 is an example of a screen display showing completion of registration of a floor map. (A) is an example of a screen display after aspect ratio correction and rotation processing, (B) is an example of a screen display in which any floor can be selected from multiple floors, and (C) is an example of a screen display in which managed devices installed on any floor are displayed on a floor map. (A) is an example of a screen display after aspect ratio correction and rotation processing, (B) is an example of a screen display in which any floor can be selected from multiple floors, and (C) is an example of a floor map screen display of the selected floor. FIG. 11 is a sequence diagram showing a modified example of the registration process of the floor map image according to the embodiment.

Below, an embodiment of the present invention will be described with reference to the drawings. Note that if there are any overlapping parts in the description of the drawings, the description will be omitted.

[Embodiment]
First, the present embodiment will be described with reference to FIGS.

[Overall configuration of image communication system]
FIG. 1 is a diagram showing an overall configuration of an example of an image communication system according to the present embodiment. As shown in FIG. 1, the image communication system 1 has various devices including an image registration management device 3, an image registration support device 5, an image display device 7, a management target device 8, and a location information providing device 9. In the image communication system 1, an image management system 2 is constructed as a system including the image registration management device 3, the image registration support device 5, and the image display device 7. In addition, in the image communication system 1, a registered image display system 4 is constructed as a system including the image registration support device 5 and the image display device 7. In addition, the image registration management device 3, the image registration support device 5, the image display device 7, and the management target device 8 are each connected to each other via a communication network 100. Here, the communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), or the like. The communication network 100 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution), etc. Furthermore, the position information providing device 9 is communicably connected to the image management system 2 via the above-mentioned communication network 100 by wireless communication.

<Image registration management device>
The image registration management device 3 performs the registration of a corrected floor map in which the aspect ratio of an overhead image (floor map) obtained by overlooking a specific space is corrected, and the registration of location information transmitted by the management target device 8. The image registration management device 3 also has a function as a Google Maps display server (Google is a registered trademark), and controls the display of the corrected floor map on an electronic map for the image display device 7. Furthermore, the image registration management device 3 also performs the coordinate conversion between floor map image coordinates and actual coordinates performed by the image registration support device 5 described later, i.e., the correction of the aspect ratio of the floor map image, the rotation processing of the floor map image, and the creation function of the corrected floor map.

<Image registration support device>
The image registration support device 5 has a dedicated program such as an electronic map registration application, and provides a user interface (UI) that displays a floor map image to be displayed on the electronic map on a screen and allows a user to input arbitrary coordinates. The image registration support device 5 also has a function of receiving and managing location information provided by a location information providing device such as a GPS (Global Positioning System). Furthermore, the image registration support device 5 has a function of correcting the aspect ratio of the floor map image, rotating the floor map image, and transmitting corrected floor map image data related to the corrected floor map in communication with the image registration management device 3. By having such functions, the image registration support device 5 has a function of supporting the image registration management device 3. The image registration support device 5 can also function as a registration notification device that notifies the registration of a floor map image managed by the above-mentioned image registration management device 3.

<Image display device>
The image display device 7 functions as a display device controlled by the above-mentioned image registration management device 3 for displaying floor map images and the like. The image display device 7 also constitutes a registered image display system 4 together with the image registration support device 5. In this way, when the image display device 7 is configured separately from the image registration support device 5, it is sufficient that the image display device 7 has the function of simply serving as a display device. On the other hand, the image display device 7 may be configured integrally with the image registration support device 5, in which case the display means of the image display device 7 serves the function of the display means of the image registration support device 5.

<Devices to be managed>
The managed devices 8 include devices such as an image forming device (MFP: Multi-Function Peripheral), an image forming device using an inkjet method, an industrial printing machine, a commercial printing machine, etc., and provide location information of these devices to the image registration management device 3. Furthermore, the managed devices 8 include various communication devices (devices) such as an interactive white board (IWB: Interactive White Board) that connects to a personal computer (PC) to communicate, a video conference system (UCS: Unified Communication System), a projector, etc. Furthermore, the managed devices 8 may be home appliances such as a refrigerator, a washing machine, and an air conditioner.

<Location information providing device>
The location information providing device 9 is a device that provides location information indicating the location of the managed device 8, and is, for example, a Global Positioning System (GPS), Wi-Fi (registered trademark), or a GPS service provided by the Japan Aerospace Exploration Agency (JAXA). By utilizing IMES (Indoor Messaging System), a domestically developed "indoor positioning technology" that was invented by the central figure, outdoor or indoor position information is provided to the image registration support device 5 via wireless communication.

●About terminology●
The floor map image of the floor map in this embodiment refers to, for example, an overhead image obtained by looking down on a specific space. The specific space includes a specific indoor floor, basement, and rooftop. Furthermore, the specific space also includes outdoor facilities, parks, sports fields, etc. In this embodiment, for convenience of explanation, the term "floor map image" is used.

The term "aspect ratio" refers to the ratio of the length to the width of a screen or image. It generally refers to the ratio of the length of the long side to the short side, and is expressed as, for example, 4:3 or 16:9. In this embodiment, the term "aspect ratio" will be used hereafter to describe the aspect ratio.

[Aspect ratio correction]
Next, a correction process for correcting an image having a certain aspect ratio to an image having another aspect ratio will be described. Fig. 2 is a diagram showing an example of a floor map image. As shown in Fig. 2, a floor map (hereinafter referred to as floor map image M) having a rectangular shape with a height of pv [pixels] and a width of ph [pixels] and an image size of ph x pv [pixels/dot], in which Room A, Room B, and Room C are arranged, shows two arbitrary points A (one point in Room A) and B (one point in Room C).

FIG. 3 is a diagram showing an example of a case where a predetermined angle exists between two coordinate systems. In FIG. 3, a rotation process for rotating the floor map image M shown in FIG. 2 by a predetermined angle will be described. First, there is an image coordinate system (xy coordinate system), and the floor map image M is placed in this image coordinate system. With respect to this xy image coordinate system, an actual coordinate system (EN coordinate system) based on position information obtained from a position information providing device 9 such as a GPS (Global Positioning System) exists in a state rotated clockwise by a predetermined angle θ. In this case, the distance d (actual distance between two points) of the line segment AB connecting two points A and B, and the angle between the N axis and the line segment AB, i.e., the azimuth angle α, can be obtained, for example, by the following method of "calculating the distance between two points and calculating the azimuth angle". Specifically, if point A is given 139.388164 east longitude, 35.457939 north latitude, and point B is given 139.389123 east longitude, 35.458390 north latitude, refer to "Calculating the distance between two points and the azimuth angle" and you will get the following:
・Actual distance between two points A and B dAB = 100 [m]
・Azimuth angle α＝59.999350°
can be calculated.

Furthermore, the coordinates of any point can be found using, for example, "Vincenty's formula" based on the coordinates of a point in the real coordinate system (E-N coordinate system), the actual distance to this point, and the azimuth angle to this point. Here, the real coordinate system (E-N coordinate system) refers to the coordinate system in which a specified electronic map is displayed. The image management system 2 rotates the floor map image M by a specified angle to match this real coordinate system (E-N coordinate system), and displays it on the display means as a corrected floor map image M'.

Next, the angle β (azimuth angle) between the y-axis of the image coordinate system and the line segment AB can be calculated using the following (Equation 1).

For example, if you substitute the following specific values,
β=|tan ^-1 (-268/1000)|+90°
= 106.669684°
The angle β is expressed as an azimuth angle in the image coordinate system, and in the example of FIG. 3, it is the angle rotated clockwise from the image coordinate system toward the real coordinate system. The value of x = 1000 is the absolute value of the difference between the x component of the image coordinate of the current location number 1 and the x component of the image coordinate of the current location number 2 managed in the real coordinate information management table described later, and is, for example, a value obtained as |1120 - 120| = 1000. Similarly, the value of y = 286 is the absolute value of the difference between the y component of the image coordinate of the current location number 1 and the y component of the image coordinate of the current location number 2 managed in the real coordinate information management table described later, and is, for example, a value obtained as |450 - 182| = 268. As a result, the angle θ between the y axis of the image coordinate system and the N axis of the real coordinate system is
θ=β−α=(106.669684°−59.999350°)
= 46.670334°
In the example of Fig. 3, the angle θ is an angle (tilt) rotated counterclockwise from the real coordinate system toward the image coordinate system.

Next, the actual distance x' in the image coordinate system can be calculated using the following (Equation 2) using trigonometric functions for points A, B, C, L, M, N, and J and angle θ in Figure 3.

When specific values are substituted into the above (Equation 2), for example,
x'＝100×cos(59.999350°)×sin(46.670334°)
+100×sin(59.999350°)×cos(46.670334°)
≒ 99.4517
When calculating the length (distance) of x', the absolute value of each term on the right side of (Equation 2) may be calculated. Similarly, the actual distance y' in the image coordinate system can be calculated using the following (Equation 3) using trigonometric functions for points A, B, C, L, M, N, and J and angle θ in FIG.

When specific values are substituted into the above (Equation 3), for example,
y'＝100×cos(59.999350°)×cos(46.670334°)
-100×sin(59.999350°)×sin(46.670334°)
≒ -28.6854
When calculating the length (distance) of y', the absolute value of each term on the right side of (Equation 3) may be calculated.

From these results, the aspect ratio λ of the image is
λ＝x/y
= 1000/268
= 3.731343
The aspect ratio λ' in real coordinates is calculated from the absolute values of each value as follows:
λ´＝x´/y´
= 99.4517/28.6854
= 3.466980
Here, if λ<λ', the floor map image M is compressed in the y-axis direction, if λ>λ', the floor map image M is compressed in the x-axis direction, and if λ=λ', compression of the floor map image M is unnecessary. In the examples shown in Figs. 2 and 3 and the above-mentioned calculation results, since λ<λ', the image of the floor map image M is compressed in the x-axis direction. In other words,
1350×3.466980/3.731343
= 1254
Then, a process is performed to reduce the image of floor map image M from the size in the x-axis direction (ph): 1250p to 1254p given by (ph)'. As a result, the floor map image M' shown in Figure 4 is obtained. Figure 4 is a diagram showing an example of a floor map image with a corrected aspect ratio. Note that instead of compressing the image in the x-axis direction, it may be expanded in the y-axis direction. The distance equivalent to one pixel may be set to 1.076252. In that case,
830×3.731343/3.466980
= 893
will increase the image size along the y axis from 830p to 893p.

Fig. 5 is a diagram showing another example in which a predetermined angle exists between two coordinate systems. In Fig. 5, first, the azimuth angle of point P in an image coordinate system (xy coordinate system) with point A as the origin is calculated. As shown in Fig. 5, the azimuth angle of point P in an image coordinate system (xy coordinate system) with point A as the origin is represented by an angle γ between the y axis and a line segment AP. In addition, the image coordinates of point A before the aspect ratio change are (120, 380) for the following reasons.
basis:
380
= [y size of image 830] - [y coordinate of image coordinates corresponding to current location number 1 in the actual coordinate information management table: 450]
Therefore, in the image coordinate system with point A as the origin, the coordinates of point P are (-120, 380).

The image coordinates of point P after reduction in the x-axis direction due to the change in aspect ratio are calculated.
x-axis coordinate:
xp = -120 x 3.466980/3.731343
= -111
The y-axis coordinate remains unchanged, yp = 450.
Next, the azimuth angle of point P in the image coordinate system with point A as the origin is calculated. The angle γ is expressed as follows:
γ＝tan ^-1 (-111/380)
= 16.283362°
It can be found that:
Next, the azimuth angle of point P in the real coordinate system (EN coordinate system) with point A as the origin is calculated. As shown in Fig. 5, the azimuth angle of point P in the real coordinate system (EN coordinate system) with point A as the origin is expressed as the angle δ between the N axis and the line segment AP. As shown in Fig. 5, angle δ is the sum of angles θ and γ, so
δ＝θ+γ
= 46.670334° + 6.283362°
= 62.953696°
It can be found that:

Furthermore, the distance dAP between points A and P in the real coordinate system (EN coordinate system) is calculated.
Actual distance xp = [x-axis coordinate of point P before aspect ratio change] × x´/x
Since x' and x are the actual distance and pixel distance before the aspect ratio is changed, the actual distance xp uses the x-axis coordinate of point P before the aspect ratio is changed, which is 120. Alternatively, if the x-axis coordinate of point P after the aspect ratio is changed, which is 111, it will be necessary to further multiply by λ'/λ, but the result will be the same. In this way, the distance dAP can be found by the following (Equation 4).

Substituting the actual distance of yp = yp * y'/y (the y-axis coordinate: 450 is used as is because it is not scaled down in the y-axis direction) into the above (Equation 4),
dAP=sqrt((120×x´/x) ² +(380×y´/y) ² )
= sqrt((120×99.4517/1000) ² +(350×28.6854/268) ² )
= 42.3880 [m]
It can be found that:

When the actual coordinates of point P are calculated based on the above-mentioned actual coordinates of point A, the calculated azimuth angles γ and δ of point P, and the distance dAP between point A and point P, the actual coordinates of point P are calculated as follows:
East longitude 139.387748
Latitude 35.458113
These values of east longitude and north latitude are registered and managed as information on actual coordinates 1 corresponding to floor identification information: F001 in the floor map management table described later. Similarly, the actual coordinates of point Q are
East longitude 139.388460
Latitude 35.457563N
It can be found that:

The coordinates of any point can be calculated using the above-mentioned "Vincenty's formula" for the above-mentioned longitude and latitude. These values of longitude and latitude are registered and managed in the actual coordinate 2 of the floor map management table described later. Note that the above-mentioned angles and calculation results are only examples, and are not limited to these values. Furthermore, the actual coordinate values of points P and Q are values based on the above-mentioned actual coordinates of point A, the calculated azimuth angles γ and δ of point P, and an example of the distance between points A and P, but are not limited to these values. Points P and Q are treated as an example of third position information indicating the position of any one of two points given to the corrected overhead image corrected to the aspect ratio of the image related to the electronic map. In other words, points P and Q are given as reference information that serves as a reference for registering the corrected overhead image by the image correction unit described later.

Figure 6 shows a specific example of aspect ratio correction and image rotation processing. Figure 6 (A) is an example of a floor map image shown in an image coordinate system, showing the above-mentioned two arbitrary points A and B being placed. Figure 6 (B) is an example of a floor map image after the aspect ratio has been corrected and displayed on an electronic map shown in a real coordinate system. And Figure 6 (C) is an example of an electronic map on an electronic map shown in a real coordinate system, in which the floor map image after the aspect ratio has been corrected and rotated to a specified angle, and the device position information of the managed device 8 arranged in a specified space that forms the floor map image are displayed as marks.

It should be noted that instead of the above-mentioned aspect ratio correction method, other methods can be used to correct the aspect ratio. For example, when two arbitrary points A (xa, ya) and B (xb, yb) are taken on the floor map image, the aspect ratio of the floor map can be corrected using the coordinate values (pixel values) and actual coordinate values of the two points A and B. Specifically,
・Aspect ratio based on coordinates on floor map image = a
・Aspect ratio in real coordinates = a'
For example,
・If a <a': The image is compressed horizontally (x-axis direction). ・If a >a': The image is compressed vertically (y-axis direction) to correct the aspect ratio of the floor map.

[Hardware configuration]
<Hardware configuration of image registration management device, image registration support device, and image display device>
Fig. 7 is a diagram showing an example of the hardware configuration of a computer according to this embodiment. The image registration management device 3 is constructed by a computer, and has hardware resources including a CPU 301, a ROM 302, a RAM 303, a HD (Hard Disk) 304, a HDD controller 305, a display 306, an external device connection I/F 308, a network I/F 309, a bus line 310, a keyboard 311, a pointing device 312, a DVD-RW (Digital Versatile Disk Rewritable) drive 314, a media I/F 316, an audio input/output I/F 317, a microphone 318, a speaker 319, and a short-range communication circuit 320, as shown in Fig. 7.

Of these, the CPU 301 controls the operation of the entire distribution terminal 30. The ROM 302 stores programs used to drive the CPU 301, such as IPL. The RAM 303 is used as a work area for the CPU 301. The HD 304 stores various data such as programs. The HDD controller 305 controls the reading or writing of various data from the HD 304 according to the control of the CPU 301. The display 306 displays various information such as a cursor, menu, window, character, or image. The display 306 is an example of a display unit. The display 306 may be a touch panel display equipped with an input means. The external device connection I/F 308 is an interface for connecting various external devices. In this case, the external device is, for example, a USB memory or a printer. The network I/F 309 is an interface for data communication using the communication network 100. Bus line 310 is an address bus or a data bus for electrically connecting each component such as CPU 301 shown in FIG. 12.

The keyboard 311 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 312 is a type of input means for selecting or executing various instructions, selecting a processing target, moving a cursor, etc. The input means may be not only the keyboard 311 and the pointing device 312, but also a touch panel or a voice input device, etc. The DVD-RW drive 314 controls the reading or writing of various data from the DVD-RW 313, which is an example of a removable recording medium. It is not limited to DVD-RW, but may also be DVD-R or Blu-ray (registered trademark) Disc. The media I/F 316 controls the reading or writing (storage) of data from the recording medium 315, such as a flash memory. The microphone 318 is a type of built-in sound collection means for inputting sound (sound signal). The speaker 319 is a type of sound output means for outputting sound (sound signal). The audio input/output I/F 317 is a circuit that processes the input and output of audio signals between the microphone 318 and the speaker 319 under the control of the CPU 301.

The image registration support device 5 is constructed using a computer, and as shown in FIG. 7, the hardware resources from the CPU 501 to the speaker 519 have the same configuration as the hardware resources from the CPU 301 to the speaker 319 of the image registration management device 3, so detailed explanations are omitted.

The image display device 7 is constructed by a computer, and as shown in FIG. 3, the hardware resources from the CPU 701 to the speaker 719 have the same configuration as the hardware resources from the CPU 301 to the speaker 319 of the image registration management device 3, so detailed explanations are omitted. In this case, the sound input/output I/F 717, microphone 718, and speaker 719 do not need to have those configurations.

The above programs may be recorded in a computer-readable recording medium as an installable or executable file, or may be distributed by downloading via a network. Examples of recording media include CD-Rs (Compact Disc Recordable), DVDs (Digital Versatile Disks), Blu-ray (registered trademark) Discs, SD cards, and USB memories. The recording media may be provided domestically or internationally as a program product. For example, the image registration management device 3 realizes the image registration management method according to the present invention by executing the program according to the present invention.

<Hardware configuration of the device to be managed>
Fig. 8 is a diagram showing an example of the hardware configuration of a managed device according to this embodiment. The managed device 8 is, for example, an MFP, and is provided with hardware resources including a controller 810, a short-range communication circuit 820, an engine control unit 830, an operation panel 840, and a network I/F 850 as shown in Fig. 8.

Of these, the controller 810 has a CPU 801, which is the main part of the computer, a system memory (MEM-P) 802, a north bridge (NB) 803, a south bridge (SB) 804, an ASIC (Application Specific Integrated Circuit) 805, a local memory (MEM-C) 806, which is a storage unit, an HDD controller 807, and an HD 808, which is also a storage unit, and is configured such that the NB 803 and the ASIC 805 are connected by an AGP (Accelerated Graphics Port) bus 821.

Of these, the CPU 801 is a control unit that performs overall control of the managed device 8. The NB 803 is a bridge that connects the CPU 801 with the MEM-P 802, SB 804, and AGP bus 821, and has a memory controller that controls reading and writing to the MEM-P 802, a PCI (Peripheral Component Interconnect) master, and an AGP target.

The MEM-P 802 consists of a ROM 802a, which is memory for storing programs and data that realize the various functions of the controller 810, and a RAM 802b, which is used for expanding programs and data, and as a drawing memory during memory printing. The programs stored in the RAM 802b may be provided by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

SB804 is a bridge for connecting NB803 with PCI devices and peripheral devices. ASIC805 is an IC (Integrated Circuit) for image processing applications that has hardware elements for image processing, and plays the role of a bridge connecting AGP bus821, PCI bus822, HDD808, and MEM-C806. This ASIC805 includes a PCI target and AGP master, an arbiter that arbitrates the drive timing of each signal according to a predetermined priority in ASIC805, a memory controller that controls MEM-C806, multiple DMACs (Direct Memory Access Controllers) that rotate image data using hardware logic, and a PCI unit that transfers data between scanner unit831 and printer unit832 via PCI bus822. In addition, a USB (Universal Serial Bus) interface or an IEEE 1394 (Institute of Electrical and Electronics Engineers 1394) interface may be connected to the ASIC 805.

The MEM-C806 is a local memory used as an image buffer for copying and a code buffer. The HD808 is a storage for storing image data, font data used during printing, and forms. The HD808 controls the reading and writing of data from and to the HD808 under the control of the CPU801. The AGP bus821 is a bus interface for a graphics accelerator card proposed to speed up graphic processing, and by directly accessing the MEM-P802 with high throughput, the graphics accelerator card can be made faster.

The short-range communication circuit 820 also includes a short-range communication circuit 820a. The short-range communication circuit 820 is a communication circuit such as NFC or Bluetooth (registered trademark).

The engine control unit 830 is further composed of a scanner unit 831 and a printer unit 832. The operation panel 840 is equipped with a panel display unit 80a such as a touch panel that displays the current setting values and selection screens and receives input from the operator, and an operation panel 840b including a numeric keypad that receives setting values for image formation conditions such as density setting conditions and a start key that receives a copy start instruction. The controller 810 controls the entire managed device 8, and controls, for example, drawing, communication, and input from the operation panel 840. The scanner unit 831 or the printer unit 832 includes an image processing unit such as error diffusion and gamma (γ) conversion.

The managed device 8 can sequentially switch between the document box function, copy function, printer function, and facsimile function using the application switching key on the operation panel 840. When the document box function is selected, the device enters document box mode, when the copy function is selected, the device enters copy mode, when the printer function is selected, the device enters printer mode, and when the facsimile mode is selected, the device enters facsimile mode.

The network I/F 850 is an interface for data communication using the communication network 100. The short-range communication circuit 820 and the network I/F 850 are electrically connected to the ASIC 805 via the PCI bus 822. Note that the managed device 8 is not limited to the MFP described above, as long as it is configured to provide location information of the managed device itself.

<Hardware configuration of location information providing device>
Fig. 9 is a diagram showing an example of the hardware configuration of the position information providing device according to this embodiment. The position information providing device 9 is constructed by a computer, and has hardware resources including a CPU 901, a ROM 902, a RAM 903, and a communication I/F 904 as shown in Fig. 9. The CPU 901, the ROM 902, the RAM 903, and the communication I/F 904 are connected to each other via a bus line 910. The CPU 901, the ROM 902, the RAM 903, and the communication I/F 904 have the same hardware configuration as the CPU, ROM, RAM, and various buses in the image registration management device, the image registration support device, and the image display device described above, so detailed description will be omitted.

[Functional configuration of image communication system]
Next, the functional configuration of this embodiment will be described with reference to Fig. 10 to Fig. 13. Fig. 10 and Fig. 11 are diagrams showing an example of the functional configuration of an image communication system according to this embodiment.

<Functional configuration of the image registration management device>
10, the image registration management device 3 has a transmission/reception unit 31, an acquisition unit 32, a display control unit 34, an image correction unit 36, a creation unit 37, a registration unit 38, and a storage/readout unit 39. Each of these functional units is a function or means realized when any of the hardware resources shown in Fig. 7 operates in response to an instruction from the CPU 301 in accordance with a program for the image registration management device 3 that is loaded from at least one of the ROM 302 and the HD 304 to the RAM 303. The image registration management device 3 also has a storage unit 3000 constructed from at least one of the ROM 302 and the HD 304 shown in Fig. 7.

●Floor map management table●
FIG. 12A is a conceptual diagram showing an example of a floor map management table according to this embodiment. In the storage unit 3000, a floor map management DB 3001 configured by a floor map management table as shown in FIG. 12A is constructed. In this floor map management table, for example, tenant identification information identifying a tenant who is an owner of the floor map is used as a tab, and building identification information, floor identification information, floor name, actual coordinate 1, actual coordinate 2, and image file path are associated and stored and managed for each image identification information divided by each tab. The tenant identification information to be tabbed is, for example, T001. In this embodiment, the floor map management DB 3001 functions or has a means to function as an example of a floor map management means.

Of these, image identification information is given as information including multiple English letters, codes, symbols, numbers, etc., and is a number, symbol, or code with a predetermined number of digits for identifying an image. This image identification information is given in a number corresponding to the floors in a building, and is managed, for example, as IID001, IID002, etc.

The building identification information is identification information for identifying the building (building, company premises, apartment, condominium, factory, etc.) managed by the tenant, and is given and managed, for example, as B001, B002, etc.

The floor identification information is identification information for identifying each floor of a building, and is given, for example, F001, F002, etc. The floor name is the name given to each floor of a building, and is given and managed as names such as Ebina 1st Floor, Ebina 5th Floor, etc.

Actual coordinate 1 is location information such as longitude and latitude provided by the location information providing device 9. Actual coordinate 1 is given and managed, for example, as 139.388100 east longitude and 35.457957 north latitude. Like actual coordinate 1, actual coordinate 2 is also location information such as longitude and latitude provided by the location information providing device 9, and is given and managed, for example, as 139.388177 east longitude and 35.457877 north latitude.

The image file path is a path indicating the file registration destination of the floor map image as an overhead image, and is managed in a format such as "path"/...file1. Alternatively, the image file path may be managed in a specified format such as a URL.

●Target device management table●
Fig. 12(B) is a conceptual diagram showing an example of a target device management table according to this embodiment. A target device management DB 3002 configured by a target device management table as shown in Fig. 12(B) is constructed in the storage unit 3000. In this target device management table, the actual coordinates and image identification information of the managed device are associated with each piece of identification information of the managed device, and are stored and managed. In this embodiment, the target device management DB 3002 functions or has a means of functioning as an example of a target device management means.

The identification information of the managed device is identification information for identifying the managed device 8 such as the MFP described above, and is given and managed, for example, as TID002, TID002, etc.

The actual coordinates (location information) of the managed device indicate the actual location information in a specific space in which the managed device 8 is located, and are called actual coordinates in this embodiment. These actual coordinates are given and managed using historical information such as, for example, 139.388180 east longitude and 35.457950 north latitude.

<<Functional configuration of the image registration management device>>
Next, a detailed description will be given of each functional configuration of the image registration management device 3. The transmission/reception unit 31 of the image registration management device 3 shown in Fig. 10 is mainly realized by the processing of the CPU 301 with respect to the external device connection I/F 308 and the network I/F 309 shown in Fig. 7, and transmits and receives various data (or information) between the image registration assistance device 5, the image display device 7, and the management target device 8 via the communication network 100. In this embodiment, the transmission/reception unit 31 has means that function or function as an example of a transmission means and a reception means.

The acquisition unit 33 is mainly realized by the processing of the CPU 301 shown in FIG. 7, and acquires corrected floor map image data (corrected overhead image data related to the corrected overhead image) related to the corrected floor map image from the floor map management table (floor map management DB 3001, see FIG. 12 (A)). In this embodiment, the acquisition unit 33 functions or has a means to function as an example of an acquisition means.

The display control unit 34 is mainly realized by the processing of the CPU 301 on the display 306 shown in FIG. 7, and controls the display of the corrected floor map image. The display control unit 34 also functions as a web server, and in that case, controls the display of the corrected floor map image on the display 706 of the image registration assistance device 5 or the image display device 7. In this embodiment, the display control unit 34 functions or has means to function as an example of a display control means.

The image correction unit 36 is mainly realized by the processing of the CPU 301 shown in FIG. 7, and performs processing such as correcting the aspect ratio of the floor map image and rotating the floor map image between the image coordinate system and the real coordinate system. In this embodiment, the image correction unit 36 functions or has a means for functioning as an example of an image correction means.

The creation unit 37 is mainly realized by the processing of the CPU 301 shown in FIG. 7, and creates a corrected floor map image after the aspect ratio has been corrected. In this embodiment, the creation unit 37 functions or has a means for functioning as an example of a creation means.

The registration unit 38 is mainly realized by the processing of the CPU 301 shown in FIG. 7, and registers the corrected floor map image transmitted by the image registration support device 5 in the floor map management table (floor map management DB 3001, see FIG. 12 (A)). In this embodiment, the registration unit 38 functions or has a means to function as an example of a registration means.

The storage/reading unit 39 is mainly realized by the processing of the CPU 301 shown in FIG. 7, and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000. In this embodiment, the storage/reading unit 39 functions or has means to function as an example of storage/reading means.

<Functional configuration of the image registration support device>
As shown in Fig. 10, the image registration support device 5 has a transmission/reception unit 51, a reception unit 52, an acquisition unit 53, a display control unit 54, an image correction unit 56, a creation unit 57, a registration unit 58, and a storage/readout unit 59. Each of these functional units is a function or means realized by operating any of the hardware resources shown in Fig. 7 by an instruction from the CPU 501 according to a program for the image registration support device 3 expanded from at least one of the ROM 502 and the HD 504 to the RAM 503. The image registration support device 5 also has a storage unit 5000 constructed by at least one of the ROM 502 and the HD 504 shown in Fig. 7. The storage unit 5000 also stores a floor map image registration program that starts a floor map image registration application executed by the image registration support device 5.

●Actual coordinate information management table●
FIG. 13 is a conceptual diagram showing an example of a real coordinate information management table according to this embodiment. In the storage unit 5000, a real coordinate information management DB 5001 configured by the real coordinate information management table shown in FIG. 13 is constructed. In this real coordinate information management table, the image coordinates [pixel] of the floor map image and the real coordinates (location information) provided by the location information providing device 9 are associated with each current location number (for example, 1, 2, etc.) for identifying the current location, and are stored and managed. The image coordinates indicate the coordinate values in the image of each of the above-mentioned points A and B. Specifically, the image coordinates corresponding to the current location number 1 are (120, 450), and the image coordinates corresponding to the current location number 2 are given as (1120, 164). These two coordinate values become the coordinates of points A and B on the coordinate axes when the lower left corner of the floor map image is the origin, the side with a length ph is the x-axis, and the side with a length pv is the y-axis. Furthermore, the actual coordinates (location information) are given and managed using historical information such as, for example, 139.388164 east longitude and 35.457939 north latitude. In this embodiment, the actual coordinate information management DB 5001 functions or has a means for functioning as an example of an actual coordinate information management means. Furthermore, the image coordinates (image coordinates 1) of the current location number 1 and the image coordinates (image coordinates 2) of the current location number 2 described above are an example of first location information. Furthermore, the actual coordinates (location information 1) of the current location number 1 and the actual coordinates (location information 2) of the current location 2 described above are an example of second location information.

<Functional configuration of the image registration support device>>
Next, each functional configuration of the image registration support device 5 will be described in detail. The transmission/reception unit 51 of the image registration support device 5 shown in Fig. 10 is mainly realized by the processing of the CPU 501 on the external device connection I/F 508 and the network I/F 509 shown in Fig. 7, and transmits and receives various data (or information) between the image registration management device 3, the image display device 7, the management subject device 8, and the position information providing device 9 via the communication network 100. In this embodiment, the transmission/reception unit 51 has a function or a means that functions as an example of a transmission means and a reception means.

The reception unit 52 is mainly realized by the processing of the CPU 501 on the pointing device 512 shown in FIG. 7, and when the user designates an arbitrary point on the floor map displayed on the display 506, the reception unit 52 accepts the designated position as the current location of the user. In this embodiment, the reception unit 52 functions or has a means for functioning as an example of a reception means.

The acquisition unit 53 is mainly realized by the processing of the CPU 501 shown in FIG. 7, and acquires from the storage unit 5000 a floor map image obtained by overlooking a specific space and uploaded by a user. In addition, when a user uploads a floor map image using a camera function, the acquisition unit is realized by the processing of the CPU 501 shown in FIG. 7 for the photographing means provided in the image registration support device 5. The acquisition unit 53 can also directly acquire location information data provided by GPS from the location information providing device 9, data via Wi-Fi (registered trademark), etc. In this embodiment, the acquisition unit 53 functions or has means to function as an example of an acquisition means.

The display control unit 54 is mainly realized by the processing of the CPU 501 on the display 506 shown in FIG. 7, and controls the display of the uploaded floor map image acquired by the acquisition unit 53. In this embodiment, the display control unit 54 functions or has a means for functioning as an example of a display control means.

The image correction unit 56 is mainly realized by the processing of the CPU 501 shown in FIG. 7, and performs processing to correct the aspect ratio of the floor map image, and processing to rotate the floor map image between the image coordinate system and the real coordinate system. In this embodiment, the image correction unit 56 functions or has a means to function as an example of an image correction means.

The creation unit 57 is mainly realized by the processing of the CPU 501 shown in FIG. 7, and creates a corrected floor map image after the aspect ratio has been corrected. In this embodiment, the creation unit 57 functions or has a means for functioning as an example of a creation means.

The registration unit 58 is mainly realized by the processing of the CPU 501 shown in FIG. 7, and registers the actual coordinates of the current location provided by the position information providing device 9 in the actual coordinate information management table (actual coordinate information management DB 5001, see FIG. 13). In this embodiment, the registration unit 58 functions or has a means for functioning as an example of a registration means.

The storage/reading unit 59 is mainly realized by the processing of the CPU 501 shown in FIG. 7, and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000. In this embodiment, the storage/reading unit 59 functions or has means to function as an example of storage/reading means.

<Functional configuration of image display device>
10, the image display device 7 has a transmission/reception unit 71, a reception unit 72, a display control unit 74, and a storage/readout unit 79. Each of these functional units is a function or means realized by operating any of the hardware resources shown in Fig. 7 in response to an instruction from a CPU 701 in accordance with a program for the image display device 7 loaded from at least one of a ROM 702 and a HD 704 to a RAM 703. The image display device 7 also has a storage unit 7000 constructed by the ROM 702 or the HD 704 shown in Fig. 7.

<Functional configuration of image display device>>
Next, a detailed description will be given of each functional configuration of the image display device 7. The transmission/reception unit 71 of the image display device 7 shown in Fig. 10 is mainly realized by the processing of the CPU 701 with respect to the external device connection I/F 708 and the network I/F 709 shown in Fig. 7, and transmits and receives various data (or information) with the image registration management device 3 via the communication network 100. In this embodiment, the transmission/reception unit 71 has means that function or function as an example of a transmission means and a reception means.

The reception unit 72 is mainly realized by the processing of the CPU 701 on the pointing device 712 shown in FIG. 7, and receives any floor selected by the user from the floor information on the electronic map displayed on the display 706. In this embodiment, the reception unit 72 functions or has a means to function as an example of a reception means.

The display control unit 74 is mainly realized by the processing of the CPU 701 on the display 706 shown in FIG. 7, and controls the display of the corrected floor map image transmitted by the image registration management device 3 and the location information of the managed device 8. In this embodiment, the display control unit 74 functions or has a means of functioning as an example of a display control means.

The storage/reading unit 79 is mainly realized by the processing of the CPU 701 shown in FIG. 2, and stores various data (or information) in the storage unit 7000 and reads various data (or information) from the storage unit 7000. In this embodiment, the storage/reading unit 79 functions or has means to function as an example of storage/reading means.

<Functional configuration of managed device>
As shown in Fig. 11, the managed device 8 has a transmission/reception unit 81, an acquisition unit 83, and a storage/readout unit 89. Each of these functional units is a function or means realized by any of the hardware resources shown in Fig. 8 operating in response to an instruction from a CPU 801 in accordance with a program for the managed device 8 loaded from at least one of a ROM 802a and a HD 808 to a RAM 802b. The managed device 8 also has a storage unit 8000 constructed from at least one of a ROM 802a and a HD 808 shown in Fig. 8.

<<Functional configuration of managed devices>>
Next, a detailed description will be given of each functional configuration of the managed device 8. The transmission/reception unit 81 of the managed device 8 shown in Fig. 11 is mainly realized by the processing of the CPU 801 on the short-range communication circuit 820 or the network I/F 850 shown in Fig. 8, and transmits and receives various data (or information) to and from the image registration management device 3 via the communication network 100. In this embodiment, the transmission/reception unit 81 has means that function or function as an example of a transmitting means and a receiving means.

The acquisition unit 83 acquires location information based on each format provided by the location information providing device 9. For example, if the location information is provided (transmitted) from a GPS, information in the NMEA format (a collection of sentences) is acquired. When acquiring location information through communication with Wi-Fi, information including a probe request and a connection request on which the location information is based is acquired. Furthermore, if the indoor location information is provided (transmitted) from an IMES, a signal modulated with BPSK (Binary Phase Shift Keying) at a frequency of approximately 1.58 GHz is acquired. In this manner, the acquisition unit 83 includes a communication I/F 904 that can support the specifications of various types of location information provided by the location information providing device 9. In this embodiment, the acquisition unit 83 functions or has a means for functioning as an example of an acquisition means.

The storage/reading unit 89 is mainly realized by the processing of the CPU 801 shown in FIG. 8, and stores various data (or information) in the storage unit 8000 and reads various data (or information) from the storage unit 8000. In this embodiment, the storage/reading unit 89 functions or has means to function as an example of storage/reading means.

<Functional configuration of location information providing device>
11, the location information providing device 9 has a communication unit 91, a location information management unit 98, and a storage/readout unit 99. Each of these functional units is a function or means realized by any of the hardware resources shown in FIG. 9 operating in response to an instruction from a CPU 901 in accordance with a program for the location information providing device 9 loaded from a ROM 902 to a RAM 903. The location information providing device 9 also has a storage unit 9000 constructed by the ROM 902 shown in FIG. 9.

<<Functional configuration of the location information providing device>>
Next, each functional configuration of the position information providing device 9 will be described in detail. A communication unit 91 of the position information providing device 9 shown in Fig. 11 is mainly realized by processing of the CPU 901 for the communication I/F 904 shown in Fig. 9, and transmits and receives various data (or information) to and from the image registration assistance device 5 via the communication network 100. In this embodiment, the communication unit 91 functions or has a means that functions as an example of a communication means.

The location information management unit 98 is mainly realized by the processing of the CPU 901 shown in FIG. 9, and manages various types of location information such as the above-mentioned GPS, Wi-Fi, IMES, etc. in the storage unit 9000, and in response to a request from the image registration support device 5, reads the corresponding location information from the storage unit 9000 and provides it to the image registration support device 5. In this embodiment, the location information management unit 98 functions or has means that function as an example of location information management means.

The storage/reading unit 99 is mainly realized by the processing of the CPU 901 shown in FIG. 9, and stores various data (or information) in the storage unit 9000 and reads various data (or information) from the storage unit 9000. In this embodiment, the storage/reading unit 99 functions or has means to function as an example of storage/reading means.

[Processing or Operation of the Embodiment]
Next, each process or operation in the image communication system 1 according to this embodiment will be described with reference to Figs. 14 to 24. Figs. 14 and 15 are sequence diagrams showing an example of a registration process of a floor map image according to this embodiment. Fig. 19 (A) is an example of a screen display when uploading a floor map image, and (B) is an example of a screen display when registering a current location. Fig. 20 (A) is another example of a screen display when registering a current location, and (B) is an example of a screen display when registering another current location. Fig. 21 (A) is another example of a screen display when registering another current location, and (B) is an example of a screen display when inputting and registering a floor map name.

<Acquisition of actual coordinates>
First, a user who uses the image registration support device 5 starts, for example, a floor map image registration application that runs on the image registration support device 5. Next, the user makes a floor map image registration screen request by inputting his/her own user identification information to the image registration support device 5. As a result, the reception unit 52 of the image registration support device 5 receives the floor map image registration screen request from the user (step S101).

Next, the display control unit 54 cooperates with the storage/reading unit 59 to read, for example, floor map image registration screen data stored in the storage unit 5000, and displays the floor map image registration screen 5102 on the display 506, as shown in FIG. 19 (A) (step S102).

When the floor map image registration screen 5102 is displayed, the user uploads floor map image data. At this time, both the menu button 5112 for uploading from a folder and the menu button 5113 for taking a picture and uploading are displayed on the floor map image registration screen 5102, so the user can operate (press, tap, etc.) any button to upload floor map image data. In response to the user's operation, the acquisition unit 53 of the image registration support device 5 uploads the floor map image data (step S103). Specifically, the acquisition unit 53 works in cooperation with the storage/readout unit 59 to read out an image file (data) of the floor map corresponding to the floor name stored in the storage unit 5000 for the floor map image registration app that is running, and uploads it to the image registration support device 5.

Then, the display control unit 54 displays the current location 1 selection screen 1 on the display 506 of the image registration assistance device 5 (step S104). Specifically, the display control unit 54 displays the current location 1 selection screen 5103 shown in FIG. 19 (B) as an example of a screen display when registering a current location. On the current location 1 selection screen 5103, a floor map is displayed together with the message "Select your current location and press the 'Position' button." At this time, the positioning button 5114 is also displayed at the bottom of the current location 1 selection screen 5103.

Next, the reception unit 52 receives the current location 1 registration (image coordinates) specified by the user, and stores the received coordinates of the current location 1 in the real coordinate information management table (real coordinate information management DB 5001, see FIG. 13) in cooperation with the storage and reading unit 59 (step S105). Specifically, the reception unit 52, in cooperation with the display control unit 54 and the storage and reading unit 59, displays a predetermined mark corresponding to the position of the current location 1 specified by the user on the current location 1 selection screen 5104 shown in FIG. 20 (A), and stores the received coordinates of the current location 1 in the real coordinate information management table (real coordinate information management DB 5001, see FIG. 13) in cooperation with the storage and reading unit 59. Thereafter, the reception unit 52 receives an operation on the positioning button 5114 operated by the user. Note that the current location 1 selection screen 5104 shown in FIG. 20(A) is in a state where a mark indicating the position of the current location 1 has been added to the current location 1 selection screen 5103 shown in FIG. 19(B). The method of adding the mark indicating the position of the current location 1 described above may be, for example, to obtain the mark from the relative relationship between the coordinates of the floor map image displayed on the current location 1 selection screen 5103 and the coordinates on the display 506 on which the current location 1 selection screen 5103 is displayed, which are specified by the user.

After the reception unit 52 receives the positioning button, the transmission/reception unit 51 transmits a request to obtain the actual coordinates of the current location 1 to the location information providing device 9 (step S106). As a result, the communication unit 91 of the location information providing device 9 receives the request to obtain the actual coordinates of the current location 1 transmitted by the image registration support device 5. At this time, the request to obtain the actual coordinates of the current location 1 includes the identification information of the image registration support device 5.

The storage/reading unit 99 of the location information providing device 9, which has received a request to obtain the actual coordinates of the current location 1, cooperates with the location information management unit 98 to read the location information 1 corresponding to the current location 1 stored in the storage unit 9000 (step S107).

After processing of step S107, the communication unit 91 transmits an acquisition response of the actual coordinates of the current location 1 to the image registration support device 5 (step S108). As a result, the transmission/reception unit 51 of the image registration support device 5 receives the acquisition response of the actual coordinates of the current location 1 transmitted by the location information providing device 9. At this time, the acquisition response of the actual coordinates of the current location 1 includes the identification information of the image registration support device 5 and the actual coordinates of the current location 1 (location information 1).

The storage/read unit 59 of the image registration support device 5 stores the received location information 1, which is the actual coordinates of the current location 1, in the actual coordinate information management table (actual coordinate information management DB 5001, see FIG. 13) (step S109). At this time, for example, 139.388164 east longitude and 35.457939 north latitude are registered as the actual coordinates (location information) of the current location 1.

In addition, in the image communication system 1 according to this embodiment, for example, when the processing of steps S106 and S108 described above is executed, other devices may be present between the image registration support device 5 and the location information providing device 9. In other words, each piece of information (data) transmitted and received between the image registration support device 5 and the location information providing device 9 may be configured to be transmitted and received once via another device. The above-described configuration and processing method can also be applied to other processing steps between the image registration support device 5 and the location information providing device 9.

Similarly, the display control unit 54 displays the current location 2 selection screen 5105 shown in FIG. 20(B) on the display 506 of the image registration assistance device 5 (step S114). The current location 2 selection screen 5105 in FIG. 20(B) displays a floor map with a mark added to the position of current location 1, along with a message saying "After moving, select your current location and press the 'Position' button." At this time, the positioning button 5114 is also displayed at the bottom of the current location 1 selection screen 5103.

Next, the reception unit 52 receives the current location 2 registration (image coordinates) specified by the user, and stores the received coordinates of the current location 2 in the real coordinate information management table (real coordinate information management DB 5001, see FIG. 13) in cooperation with the storage and reading unit 59 (step S115). Specifically, the reception unit 52, in cooperation with the display control unit 54 and the storage and reading unit 59, displays a predetermined mark corresponding to the position of the current location 2 specified by the user on the current location 2 selection screen 5106 shown in FIG. 21 (A), and stores the received coordinates of the current location 2 in the real coordinate information management table (real coordinate information management DB 5001, see FIG. 13) in cooperation with the storage and reading unit 59. Thereafter, the reception unit 52 receives an operation on the positioning button 5114 operated by the user. The method of adding a mark indicating the position of the current location 2 described above may be, for example, to determine the relative relationship between the coordinates of the floor map image displayed on the current location 2 selection screen 5105 and the coordinates on the display 506 on which the current location 2 selection screen 5105 specified by the user is displayed.

After the reception unit 52 receives the positioning button, the transmission/reception unit 51 transmits a request to obtain the actual coordinates of the current location 2 to the location information providing device 9 (step S116). As a result, the communication unit 91 of the location information providing device 9 receives the request to obtain the actual coordinates of the current location 2 transmitted by the image registration support device 5. At this time, the request to obtain the actual coordinates of the current location 2 includes the identification information of the image registration support device 5.

The storage/reading unit 99 of the location information providing device 9, which has received a request to obtain the actual coordinates of the current location 2, cooperates with the location information management unit 98 to read the location information 2 corresponding to the current location 2 stored in the storage unit 9000 (step S117).

After processing of step S107, the communication unit 91 transmits an acquisition response of the actual coordinates of the current location 2 to the image registration support device 5 (step S118). As a result, the transmission/reception unit 51 of the image registration support device 5 receives the acquisition response of the actual coordinates of the current location 2 transmitted by the location information providing device 9. At this time, the acquisition response of the actual coordinates of the current location 2 includes the identification information of the image registration support device 5 and the actual coordinates of the current location 2 (location information 2).

The storage/read unit 59 of the image registration support device 5 stores the received location information 2, which is the actual coordinates of the current location 2, in the actual coordinate information management table (actual coordinate information management DB 5001, see FIG. 13) (step S119). At this time, for example, 139.389123 east longitude and 35.458390 north latitude are registered as the actual coordinates (location information) of the current location 2.

<Registering floor map images>
Next, the registration of a floor map image will be described with reference to FIG. 16. FIG. 16 is a sequence diagram showing an example of a registration process of a floor map image according to this embodiment. The display control unit 54 of the image registration support device 5 displays the floor map information input screen 5107 shown in FIG. 21(B) on the display 506 (step S121). In this floor map information input screen 5107, a floor map name input field 5115 and a "Register" button 5116 are displayed together with a message saying "Enter the name of the floor map and press the "Register"button." By displaying this floor map information input screen 5107, the user can enter the name of the floor map and perform the registration process.

<Correction to actual coordinates>
Next, the reception unit 52 receives input of floor map information (name) by the user (step S122).

Then, the image correction unit 56 performs coordinate conversion between the floor map image coordinates and the actual coordinates (step S123). The details of the process of this step S123 are partially repeated from the contents described in FIG. 2 to FIG. 5 above. Here, the floor map image coordinates represent the coordinates shown in the image coordinate system (x-y coordinate system), and the actual coordinates represent the coordinates shown in the actual coordinate system (E-N coordinate system) based on the position information obtained from the position information providing device 9. On this premise, the image correction unit 56 performs coordinate conversion as described in FIG. 2 to FIG. 5. That is, in the coordinates having the origin of one of the positions (point A in this embodiment) represented by the actual coordinates (position information) of the current location 1 and the actual coordinates (position information) of the current location 2 as an example of the second position information, the aspect ratio of the floor map image is corrected so as to match the aspect ratio of the image related to a predetermined electronic map, and the image rotated by a predetermined angle with respect to the coordinate axis in the north direction in the actual coordinate system (E-N coordinate system) is corrected as a corrected overhead image.

Furthermore, the image correction unit 56 provides third position information indicating the position of one of two arbitrary points given to the corrected overhead image as reference information serving as a basis for registering the corrected overhead image.

After each process in step S123, the creation unit 57 creates the floor map image uploaded by the user as a floor map image to be added to the electronic map displayed in the real coordinate system (E-N coordinate system), i.e., a corrected overhead image (step S124). At this time, the aspect ratio of the original floor map image is converted so that the corrected overhead image matches the aspect ratio of the electronic map displayed in the real coordinate system (E-N coordinate system), and the angle is also corrected according to the inclination of the electronic map displayed in the real coordinate system (E-N coordinate system). As described above, in this embodiment, based on the two image coordinates (first position information) of the current location 1 and the current location 2 accepted by the acceptance unit 52 and the two actual coordinates (second position information) for displaying a predetermined electronic map corresponding to the first position information, the image correction unit 56 corrects the aspect ratio of the floor map image so that it matches the aspect ratio of the image of the electronic map. Then, the creation unit 57 creates a corrected floor map image (corrected overhead image) with the aspect ratio corrected. Then, as described below, the display control unit 74 of the image display device 7 displays the corrected floor map image (corrected overhead image) on the display 706 as an electronic map together with device position information of the managed devices 8 arranged in a specified space provided as the floor map image.

Then, the transmitting/receiving unit 51 transmits a request to register the corrected floor map image (corrected overhead image) created in step S124 to the image registration management device 3 (step S125). As a result, the transmitting/receiving unit 31 of the image registration management device 3 receives information related to the request to register the corrected floor map image (corrected overhead image) transmitted by the image registration support device 5. At this time, the information related to the request to register the corrected floor map image (corrected overhead image) includes user identification information, two or more pieces of actual coordinate information, corrected floor map image data, and corrected floor map image information representing the corrected floor map image. Here, the two or more pieces of actual coordinate information may be position information of two specified points determined in advance as specifications by the image registration management device 3, or may be information that sets a specified position in the floor map image and a specified key value.

After receiving a request to register the corrected floor map image (corrected overhead image), the registration unit 38 of the image registration management device 3 associates the corrected floor map image data and the corrected floor map image information with the image file path of the floor map management table (floor map management DB 3001, see Figure 12 (A)) and registers them (step S126).

After the corrected floor map image is registered, the transmission/reception unit 31 transmits a registration response of the corrected floor map image to the image registration support device 5 (step S127). As a result, the transmission/reception unit 51 of the image registration support device 5 receives the registration response of the corrected floor map image transmitted by the image registration management device 3. At this time, the registration response of the corrected floor map image includes image identification information of the corrected floor map image.

After receiving the registration response for the corrected floor map image, the display control unit 54 displays a registration completion screen for the floor map image on the display 506 (step S128).

The processes of steps S123 and S124 described above may be executed by the image registration management device 3. This will be described in detail in a modified example of this embodiment described later.

In addition, in the image communication system 1 according to this embodiment, for example, when the processing of the above-mentioned steps S125 and S127 is executed, other devices may be present between the image registration support device 5 and the image registration management device 3. In other words, each piece of information (data) transmitted and received between the image registration support device 5 and the image registration management device 3 may be configured to be transmitted and received once via another device. The above-mentioned configuration and processing method can also be applied to other processing steps between the image registration support device 5 and the image registration management device 3.

<Location information registration>
Fig. 17 is a sequence diagram showing an example of a registration process of location information according to this embodiment. As shown in Fig. 17, for example, when the location information providing device 9 is a GPS, the acquisition unit 83 of the managed device 8 acquires the current location (location information) of the managed device 8 using a GPS sensor or the like provided in the managed device 8. Also, when the location information providing device 9 is a WiFi, for example, the acquisition unit 82 controls the short-range communication circuit 820 to acquire the current location of the managed device 8 (step S141).

Next, the transmitting/receiving unit 81 transmits the acquired location information to the image registration management device 3 (step S142). As a result, the transmitting/receiving unit 31 of the image registration management device 3 receives the location information of the managed device 8 itself transmitted by the managed device 8. At this time, the location information includes identification information of the managed device 8, the actual coordinates of the managed device indicating the acquired location information, and an image ID corresponding to the number of floors. If the building from which the floor map is to be acquired has multiple floors, this image ID is identification information for identifying which floor the floor map is for. Furthermore, if the location information providing device 9 is Wi-Fi, a Wi-Fi router ID is included.

The storage/read unit 39 of the image registration management device 3 registers the actual coordinates (location information) of the managed device corresponding to the identification information of the managed device 8 in the target device management table (target device management DB 3002, see FIG. 12(B)) (step S143). The managed device 8 and the image registration support device 5 then execute a loop process in which the above-mentioned steps S141-S143 are repeated the number of times corresponding to the number of managed devices 8.

Figure 22 is an example of a screen display indicating completion of floor map registration. As shown in Figure 22, the display control unit 54 displays a floor map image registration completion screen 5108 on the display 506. At this time, the floor map image is displayed on the floor map image registration completion screen 5108 together with the message "Registered." In addition, a "Go to top screen" button 5117 is also displayed at the bottom of the floor map image registration completion screen 5108. When this floor map image registration completion screen 5108 is displayed, the user can operate the "Go to top screen" button 5117 to register the next floor map image (e.g., a floor map image of another floor).

In addition, in the image communication system 1 according to this embodiment, for example, when the processing of step S142 described above is executed, other devices may exist between the managed device 8 and the image registration management device 3. In other words, each piece of information (data) transmitted and received between the managed device 8 and the image registration management device 3 may be configured to be transmitted and received once via another device. The configuration and processing method described above can also be applied to other processing steps between the managed device 8 and the image registration management device 3.

<Display processing of electronic maps>
18 is a sequence diagram showing an example of a display process of a floor map image according to the present embodiment. The transmitting/receiving unit 71 of the image display device 7 transmits a display request for device location information of the managed device 8 to be displayed on the electronic map together with the corrected floor map image (step S151). As a result, the transmitting/receiving unit 31 of the image registration management device 3 receives the display request for device location information of the managed device 8 transmitted by the image display device 7. At this time, the display request for device location information of the managed device 8 includes image identification information of the corrected floor map image received by the image registration support device 5 in step S127 and then transferred to the image display device 7.

Next, the acquisition unit 33 cooperates with the storage/readout unit 39 to search the floor map management table (floor map management DB 3001, see FIG. 12(A)) using the received image identification information as a search key to acquire an image file related to the floor map image indicated by the corresponding image file path (step S152).

In conjunction with the processing of step S152, the acquisition unit 33 extracts a list of floor map images (step S153) and places the corrected floor map image on the electronic map (step S154).

Then, the acquisition unit 33, in cooperation with the storage/readout unit 39, searches the target device management table (target device management DB 3002, see FIG. 12(B)) using the image identification information of the corrected floor map image received in step S151 as a search key to acquire the actual coordinates (location information) of the corresponding managed device (step S155).

After acquiring the actual coordinates (location information) of the managed devices, the creation unit 37 creates a corrected floor map that includes the floor map image and the device location information of the managed devices (step S156).

After creating the corrected floor map, the transmitter/receiver 31 transmits a display response of the location information to the image display device 7 as a response to step S151 (step S157). As a result, the transmitter/receiver 71 of the display device 7 receives the display response of the location information transmitted by the image registration management device 3. At this time, the display response of the location information includes the corrected floor map image data relating to the corrected floor map image.

After receiving the display response of the location information, the display control unit 74 of the image display device 7 causes the display 706 to display the corrected floor map and a predetermined mark indicating the device location information of the managed device 8 on an electronic map as shown in FIG. 23 (step S158). When performing the screen display in step S158, the display control unit 34 of the image registration management device 3 may use, for example, a function as a Google map display server. Specifically, the display control unit 34 may cause the display 706 of the image display device 7 to display the corrected floor map and a predetermined mark indicating the device location information of the managed device 8 on an electronic map.

In addition, in the image communication system 1 according to this embodiment, for example, when the processing of steps S151 and S157 described above is executed, other devices may be present between the image display device 7 and the image registration management device 3. In other words, each piece of information (data) transmitted and received between the image display device 7 and the image registration management device 3 may be configured to be transmitted and received once via another device. The above-described configuration and processing method can also be applied to other processing steps between the image display device 7 and the image registration management device 3.

●Screen display example●
<Display of corrected floor map and marks of managed devices>
Figure 23 (A) is an example of a screen display after aspect ratio correction and rotation processing, (B) is an example of a screen display in which any floor can be selected from multiple floors, and (C) is an example of a screen display in which managed devices installed on any floor are displayed on a floor map.

As shown in FIG. 23(A), the display device 7 of the image display device 7 displays an electronic map 7101 on the display 706. At this time, the corrected floor map and a mark indicating the position of the managed device 8 are displayed on the electronic map 7101. Furthermore, a selection scroll 7102 for selecting a floor is displayed on the electronic map 7101 together with a message informing the user that multiple floor maps exist.

In FIG. 23(B), a selection scroll 7104 for selecting a floor number is displayed on an electronic map 7103. When a user scrolls this selection scroll 7104 to an arbitrary floor number (for example, by scrolling to the second floor) and a predetermined time has passed, the display control unit 74 displays a battery map 7105 as shown in FIG. 23(C) on the display 706. In FIG. 23(C), a floor map of the second floor and a mark indicating the location of a managed device 8 arranged on the second floor are displayed on the electronic map 7105. Furthermore, a selection scroll 7106 is displayed at the bottom of the floor map. In this way, the display control unit 74 of the image display device 7 selectively displays to the user a corrected overhead image of a floor on which a specific managed device 8 is arranged among multiple floors, together with device position information of the specific managed device.

In addition, FIG. 24 (A) is an example of a screen display after aspect ratio correction and rotation processing, (B) is an example of a screen display in which any floor is selectable from multiple floors, and (C) is an example of a floor map screen display of the selected floor.

Figure 24(A) is similar to the contents of Figure 23(A) described above, so the explanation will be omitted. Figure 24(B) shows a state in which the selection scroll 7110 displayed at the bottom of the electronic map 7109 selects the third floor. When the third floor is selected by the user, the display control unit 74 displays the electronic photo 7111 shown in Figure 24(C) on the display 706, and displays the floor map of the third floor. However, since there is no managed device 8 on the third floor, the mark indicating the position of the managed device 8 is not displayed.

[Main Effects of the Embodiments]
As described above, according to this embodiment, the image correction unit 56 and the creation unit 57 of the image management system 2 correct the aspect ratio of the floor map image to match the aspect ratio of the image of the electronic map based on the two image coordinates accepted by the acceptance unit 52 and the two actual coordinates corresponding to the two image coordinates, and create a corrected floor map image (steps S123 and S124). Then, the display control unit 74 displays the corrected floor map image on the display 706 together with the device position information of the managed device 8 arranged in a specified space (step S158). As a result, the image correction unit 56 and the creation unit 57 correct the aspect ratio and angle of the floor map image according to the coordinate information on which the electronic map is displayed, so that the user can correctly grasp the device position information of the managed device arranged in a specified space without having to register an overhead image that matches the aspect ratio of the electronic map.

[Modifications of the embodiment]
In the above embodiments, the correction unit 56 of the image registration support device 5 corrects and converts the coordinates of the corrected floor map image (corrected overhead image) so that the aspect ratio matches that of the electronic map, and then the creation unit 57 creates the corrected floor map image (corrected overhead image). In contrast, in a modified example of the embodiment, a configuration will be described in which the image registration management device 3 corrects and converts the coordinates of the corrected floor map image (corrected overhead image), and then creates the corrected floor map image (corrected overhead image).

Figure 25 is a sequence diagram showing a modified example of the registration process of the floor map image according to this embodiment. The processes of steps S121 and S122 are the same as those of steps S121 and S122 shown in Figure 16, so the description will be omitted. After the process of step S122, the transmission/reception unit 51 transmits a registration request for the corrected floor map image (corrected overhead image) to the image registration management device 3 (step S161). As a result, the transmission/reception unit 31 of the image registration management device 3 receives information related to the registration request for the corrected floor map image (corrected overhead image) transmitted by the image registration support device 5. At this time, the information related to the registration request for the corrected floor map image (corrected overhead image) includes user identification information, pre-correction floor map image data, two or more image coordinate information representing the image coordinates of points A and B, two or more actual coordinate information on the floor map image, and pre-correction floor map image information representing the pre-correction floor map image. Here, the two or more pieces of actual coordinate information may be position information of two specific points determined in advance as specifications by the image registration management device 3, or may be information that is a set of a specific position on the floor map image and a specific key value.

After receiving a request to register the corrected floor map image (corrected overhead image), the image correction unit 36 performs coordinate conversion between the floor map image coordinates and the actual coordinates (step S162). Details of the process of step S162 are omitted here because they are similar to those described in step S123 above. At this time, the image registration management device 3 may store in the memory unit 3000 a table equivalent to the actual coordinate information management table (actual coordinate information management DB 5001, see FIG. 13) provided in the image registration support device 5 as shown in step S123, and perform the same process as step S123.

The creation unit 37 then creates the floor map image uploaded by the user as a floor map image to be added to the electronic map displayed in the real coordinate system (E-N coordinate system), i.e., a corrected overhead image (step S163). At this time, the aspect ratio of the original floor map image is converted so that the corrected overhead image matches the aspect ratio of the electronic map displayed in the real coordinate system (E-N coordinate system), and the angle is also corrected to match the inclination of the electronic map displayed in the real coordinate system (E-N coordinate system).

After the creation unit 37 generates the corrected overhead image, the registration unit 38 associates the corrected floor map image data and the corrected floor map image information with the image file path of the floor map management table (floor map management DB 3001, see FIG. 12 (A)) and registers them (step S164).

After the corrected floor map image is registered, the transmission/reception unit 31 transmits a registration response of the corrected floor map image to the image registration support device 5 (step S165). As a result, the transmission/reception unit 51 of the image registration support device 5 receives the registration response of the corrected floor map image transmitted by the image registration management device 3. At this time, the registration response of the corrected floor map image includes image identification information of the corrected floor map image.

After receiving the registration response for the corrected floor map image, the display control unit 54 displays a registration completion screen for the floor map image on the display 506 (step S166).

[Main Effects of the Modification of the Embodiment]
As described above, according to the modified example of the present embodiment, the image correction unit 36 and the creation unit 37 of the image management system 2 correct the aspect ratio of the floor map image to match the aspect ratio of the image of the electronic map based on the two or more actual coordinates accepted by the acceptance unit 52 and the actual coordinates corresponding to the two or more actual coordinates, and create a corrected floor map image (steps S162 and S163). Then, the display control unit 74 displays the corrected floor map image on the display 706 together with the device position information of the managed device 8 arranged in a specified space (step S158). As a result, the image correction unit 56 and the creation unit 57 correct the aspect ratio and angle of the floor map image according to the coordinate information on which the electronic map is displayed, so that the user can correctly grasp the device position information of the managed device arranged in a specified space without having to register an overhead image that matches the aspect ratio of the electronic map.

Furthermore, in a modified embodiment, the image registration management device 3 of the image management system 2 performs coordinate conversion between floor map image coordinates and actual coordinates and creates a corrected floor map image, so the hardware and software configurations of the image registration support device 5 can be simplified. In other words, it is expected that the image registration support device 5 and the image display device 7 can be integrated and used as a communication terminal equipped with communication and image display functions.

[Supplementary description of the embodiment]
Each function of the above-mentioned embodiment can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a device programmed to execute each function by software, such as a processor implemented by an electronic circuit. This device refers to, for example, a processor, an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), an SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module, etc.

The various tables in the above-described embodiments may be generated by the learning effect of machine learning, and by classifying the data of each associated item by machine learning, it is not necessary to use tables. Here, machine learning is a technology for making a computer acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates algorithms necessary for judgments such as data identification from learning data that is previously loaded, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, and furthermore, the learning method for machine learning may be a combination of these learning methods, and any learning method for machine learning is acceptable.

So far, we have described an image management system, image registration management device, image registration management method, and program according to one embodiment of the present invention, but the present invention is not limited to the above-described embodiment, and can be modified within the scope of what a person skilled in the art can imagine, such as adding, changing, or deleting other embodiments, and any aspect is within the scope of the present invention as long as it achieves the functions and effects of the present invention.

1 Image communication system 2 Image management system 3 Image registration management device 4 Registered image display system 5 Image registration support device 7 Image display device 8 Managed device 9 Location information providing device 34 Display control unit (an example of a display control means)
38 Registration unit (an example of a registration means)
51 Transmitting/receiving unit (an example of a receiving means, an example of a transmitting means)
52 Reception unit (an example of a reception means)
56 Image correction unit (an example of an image correction means)

JP 2010-117291 A

Claims (10)

An image management system including an image registration management device that registers and manages an aerial view image obtained by overlooking a specific space, and a registered image display system that communicates with the image registration management device to support the registration of the aerial view image and display the aerial view image as an electronic map,
a receiving means for receiving an input of first position information indicating positions of two arbitrary points on the overhead image;
A display control means for displaying the overhead image on a display means;
having
The display control means
a corrected overhead image in which the aspect ratio of the overhead image is corrected to match the aspect ratio of an image related to the specified electronic map based on the first location information accepted by the accepting means and second location information for displaying a specified electronic map corresponding to the first location information, the corrected overhead image and the device location information are acquired using image identification information included in a display request for device location information of a managed device arranged in the specified space as a search key, and a display of the overhead image and the device location information of the managed device are included on the electronic map;
displaying the corrected overhead image together with the device position information on the display means;
1. An image management system comprising: The display control means
displaying the corrected overhead image, obtained by rotating the overhead image by a predetermined angle in accordance with the coordinates at which the predetermined electronic map is displayed, together with the device position information, on the display means;
2. The image management system according to claim 1, 3. The image management system according to claim 2, further comprising:
an image correction means for correcting an aspect ratio of the overhead image in a coordinate system having an origin at any one of the positions represented by the first position information so that the aspect ratio of the overhead image coincides with the aspect ratio of an image related to the predetermined electronic map, and correcting an image rotated by the predetermined angle with respect to a coordinate axis in a north direction of the coordinate system as the corrected overhead image;
1. An image management system comprising: The image correction means includes:
third position information indicating one of two arbitrary points given to the corrected overhead image is given as reference information serving as a reference for registering the corrected overhead image;
4. The image management system according to claim 3. 5. The image management system according to claim 1, further comprising:
a receiving means for receiving, as the second location information, latitude information and longitude information corresponding to the two arbitrary points transmitted by a location information providing device that provides predetermined location information;
1. An image management system comprising: 6. The image management system according to claim 1, further comprising:
A registration means for registering the corrected overhead image,
1. An image management system comprising: When the corrected overhead image is registered for a plurality of floors in the predetermined space,
The display control means,
selectively displaying a corrected overhead image of a floor on which a predetermined managed device is located among the plurality of floors together with device position information of the predetermined managed device;
7. The image management system according to claim 6, An image registration and management device that registers and manages an overhead image obtained by overlooking a predetermined space,
A receiving means for receiving the overhead image transmitted by an image registration support device that supports the registration of the overhead image by communicating with the image registration management device;
a display control means for displaying the overhead image as an electronic map on a display means,
The display control means
a corrected overhead image in which the aspect ratio of the overhead image is corrected to match the aspect ratio of an image related to the specified electronic map based on first position information indicating the respective positions of any two points on the overhead image received by the receiving means and second position information for displaying a specified electronic map corresponding to the first position information, the corrected overhead image and the device position information of the managed device are acquired using image identification information included in a display request for device position information of the managed device arranged in the specified space as a search key, and a display of the overhead image and the device position information of the managed device are included on the electronic map;
displaying the corrected overhead image together with the device position information on the display means;
1 is a block diagram showing an image registration management device according to the first embodiment of the present invention; An image registration management method executed by an image registration management device that registers and manages an overhead image obtained by overlooking a predetermined space, comprising:
a receiving step of receiving the overhead image transmitted by an image registration support device that communicates with the image registration management device;
a display control step of displaying the overhead image as an electronic map on a display means,
The display control step includes:
a process of acquiring a corrected overhead image in which an aspect ratio of the overhead image is corrected to match an aspect ratio of an image related to the predetermined electronic map based on first position information indicating the respective positions of any two points on the overhead image received in the receiving step and second position information for displaying a predetermined electronic map corresponding to the first position information, using image identification information included in a display request for device position information of a managed device arranged in the predetermined space as a search key, and creating the overhead image and the device position information of the managed device on an electronic map including the overhead image and the device position information of the managed device;
and displaying the corrected overhead image on the display means together with the device position information.
13. An image registration and management method comprising: A program for causing a computer to function as each of the means described in claim 8.

Images