JP7463582B2 - Sensing system, maintenance terminal device, data distribution method, and image sensor - Google Patents

Description

Embodiments of the present invention relate to a sensing system, a maintenance terminal device, a data distribution method, and an image sensor.

Recently, image sensors have been equipped with a CPU (Central Processing Unit) and memory, and can be thought of as embedded computers with lenses. They also have advanced image processing functions, and can analyze captured image data to calculate, for example, the presence or absence of people, or the number of people.

JP 2017-138922 A

For example, lighting control systems using infrared motion sensors are known, but the use of image sensors has the potential to provide unprecedented services in today's society, which is equipped with a diverse range of social infrastructure. In particular, image sensors that can detect objects using dictionary data are expected to be used in a variety of ways.
Therefore, an object is to provide a sensing system, a maintenance terminal device, a data distribution method, and an image sensor that are easier to use.

According to an embodiment, the sensing system includes an image sensor that detects an object using a dictionary data set prepared for each detection item. The sensing system further includes a management means and a distribution means. The management means manages the occupancy rights of the image sensor. The distribution means distributes dictionary data corresponding to the dictionary data set from a data server of an entity that has been granted the occupancy rights to the image sensor.

FIG. 1 is a schematic diagram illustrating an example of a sensing system including an image sensor according to an embodiment. FIG. 2 is a diagram showing an example of a floor of a building. FIG. 3 is a block diagram illustrating an example of the sensing system shown in FIG. FIG. 4 is a diagram showing an example of the layout of tenants in the building 10. As shown in FIG. FIG. 5 is a functional block diagram showing an example of the maintenance terminal 200 shown in FIG. FIG. 6 is a diagram showing an example of dictionary data organized by service content. FIG. 7 is a diagram showing an example of a GUI window displayed on the monitor 210. As shown in FIG. FIG. 8 is a diagram showing an example of a state during an operation for specifying the timing of downloading dictionary data. FIG. 9 is a diagram showing an example of a state in which the timing of downloading dictionary data has been specified. FIG. 10 is a block diagram showing an example of the image sensor 3. As shown in FIG. FIG. 11 is a diagram showing an example of a data flow between the functional blocks shown in FIG. FIG. 12 is a sequence chart showing an example of a processing procedure between the maintenance terminal 200 and the image sensor 3 for distributing dictionary data. FIG. 13 is a sequence chart showing another example of the processing procedure between the maintenance terminal 200 and the image sensor 3 regarding distribution of dictionary data. FIG. 14 is a functional block diagram showing an example of the maintenance terminal 200 in the second embodiment. FIG. 15 is a functional block diagram showing an example of the image sensor 3 in the second embodiment. FIG. 16 is a diagram showing an example of the dictionary data 32d stored in FIG. FIG. 17 is a diagram for explaining that the dictionary data that can be used is limited depending on the service content. FIG. 18 is a diagram for explaining that the dictionary data that can be used is limited depending on the service content. FIG. 19 is a block diagram showing an example of a sensing system according to the third embodiment of the present invention. FIG. 20 is a functional block diagram showing an example of the maintenance terminal 200 in the third embodiment. FIG. 21 is a functional block diagram showing an example of the image sensor 3 in the third embodiment. FIG. 22 is a diagram showing a form in which some functions are transferred to the cloud 600. In FIG. FIG. 23 is a diagram showing an example in which the functions according to the embodiment are installed in a BEMS server 5 instead of a maintenance terminal 200.

Image sensors can obtain a wider variety of information than motion sensors, light sensors, or infrared sensors. By using a fisheye lens or an ultra-wide-angle lens, the area that can be captured by a single image sensor can be expanded, and image distortion can be corrected by computation. Image sensors with a function to mask areas in the field of view that you do not want to sense, and with a learning function are also known. Furthermore, it is possible to optimize the detection performance of objects by updating the dictionary data for detecting dictionary data objects. Below, an embodiment of a system equipped with this type of image sensor will be described.

FIG. 1 is a schematic diagram showing an example of a sensing system including an image sensor according to an embodiment. In FIG. 1, lighting equipment 1, air conditioning equipment 2, and image sensor 3 are provided, for example, on each floor of a building 10, and are communicatively connected to a controller 4. The controller 4 on each floor is communicatively connected, for example, to a BEMS (Building Energy Management System) server 5 in a building management center via an in-building network 500. A representative communication protocol for the in-building network 500 is the Building Automation and Control Networking protocol (BACnet (registered trademark)). Other known protocols include DALI (registered trademark), ZigBee (registered trademark), and ECHONET Lite (registered trademark).

The BEMS server 5 can be connected to a cloud computing system (cloud) 600, for example, via a TCP/IP (Transmission Control Protocol/Internet Protocol)-based communication network 300. The cloud 600 includes a database 70 and a server 80, and provides services related to building management, etc.

As shown in FIG. 2, the lighting equipment 1, the air conditioner 2 outlets, and the image sensor 3 are disposed on, for example, the ceiling of each floor. The image sensor 3 captures images captured within its field of view to obtain image data. This image data is processed in the image sensor 3 to generate person information and/or environmental information.

The environmental information is information about the environment of the space (zone) to be imaged, such as the illuminance and temperature of the floor. The person information is information about people in the target space. Examples of person information include the number of people in an area, people's behavior, the amount of activity of people, presence/absence indicating the presence or absence of people, and walking/staying indicating whether a person is walking or staying in one place. It is also possible to calculate the environmental information and person information for each small area (area) obtained by dividing a zone into multiple areas.
In recent years, studies have been conducted on controlling the lighting devices 1 and air conditioners 2 based on this information in human living environments to ensure human comfort, safety, and the like.

Figure 3 is a block diagram showing an example of the sensing system shown in Figure 1. In Figure 3, an air conditioning controller 41 that controls the air conditioning equipment 2, an elevator controller 42 that controls the elevators, a security controller 43 that controls the security system, a disaster prevention controller 44 that controls the disaster prevention system, and a lighting controller 45 that controls the lighting equipment 1 (1-1, 1-2) are connected under the BEMS server 5 via an in-building network 500.

The maintenance terminal 200 and the gateway 7 are also connected to the building network 500 so as to be able to communicate with the BEMS server 5. Furthermore, multiple image sensors 3 (3-1 to 3-n) are housed under the control of the gateway 7.

The image sensors 3-1 to 3-n are connected in a single line by the sensor network 100 (crossover wiring). The sensor network 100 connects the image sensors 3-1 to 3-n and the gateway 7 so that they can communicate with each other, for example, using EtherCAT (registered trademark). The BEMS server 5 and the maintenance terminal 200 can also communicate with the image sensors 3-1 to 3-n via the gateway 7.

Figure 4 shows an example of the layout of tenants in a building 10. For example, assume that three tenants A, B, and C occupy one floor, separated from each other by walls (dotted lines). Each tenant has four image sensors 3, which are connected to a gateway 7 via a sensor network 100.

Here, the solution using the image sensor 3 does not need to be the same for all tenants, and it is rather desirable to be able to flexibly propose different services. In other words, there is a possibility that the services required by each of the tenants A, B, and C are different. For example, if tenant A is used as a store, tenant B as an office, and tenant C as a warehouse, the services required by each of them will be different. As an example, tenant A may require the acquisition of people flow analysis data, tenant B may require detailed control of air conditioning/lighting, and tenant C may require security services.
Also, for example, in a store, the services required may change depending on the time of day, such as obtaining crowd distribution data during the day and crime prevention at night.
With existing technologies, it has been difficult to flexibly respond to such diverse service demands. Below, we will disclose a technology that can resolve such difficulties.

[First embodiment] (a form in which a minimum amount of dictionary data is installed in an image sensor) FIG. 5 is a functional block diagram showing an example of the maintenance terminal 200 shown in FIG. 3. The maintenance terminal 200, which is an example of a data server, is a computer equipped with a processor 250 such as a CPU or an MPU (Micro Processing Unit), a ROM (Read Only Memory) 220, and a RAM (Random Access Memory) 230. The maintenance terminal 200 further includes a storage unit 240 such as a hard disk drive (HDD), an optical media drive 260, a communication unit 270, and a monitor 210 that displays various information.

The ROM 220 stores basic programs such as the Basic Input Output System (BIOS) and Unified Extensible Firmware Interface (UEFI), as well as various setting data. The RAM 230 temporarily stores programs and data loaded from the storage unit 240.

The optical media drive 260 reads digital data recorded on a recording medium such as a CD-ROM 280. The various programs executed by the maintenance terminal 200 are, for example, recorded on a CD-ROM 280 and distributed. The programs stored on the CD-ROM 280 are read by the optical media drive 260 and installed in the storage unit 240.

The communication unit 270 has a function for communicating with the image sensor 3, the BEMS server 5, etc. via the in-building network 500. Various programs executed by the maintenance terminal 200 can be downloaded from a server via the communication unit 270, for example, and installed in the storage unit 240. The latest programs can also be downloaded from the cloud 600 via the communication unit 270, and installed programs can be updated. The maintenance terminal 200 can also download and transmit dictionary data to the image sensor 3 via the in-building network 500 and the gateway 7.

The memory unit 240 stores the program 240a executed by the processor 250, as well as dictionary data 240b and billing data 240c.

The dictionary data 240b is similar to template data, which is prepared according to the detection items (people, objects, men, women, etc.) of the image sensor 3. A chair is detected from the image data acquired by the image sensor 3 by performing template matching processing using dictionary data 240b prepared for detecting, for example, a "chair". A variety of dictionary data 240b is prepared in advance to improve the accuracy and to distinguish between chair types (office chairs, sofas, chairs, etc.) and colors. In addition, dictionary data can be created to detect wheelchairs, strollers, children, carts, white cane users, wandering elderly people, suspicious people, lost children, people who were unable to escape in time, etc. As an example of a method of creation, a machine learning framework such as a support vector machine or a Boltzmann machine can be used.

As shown in FIG. 6, the dictionary data 240b may be a set of multiple dictionary data for each service type (service pack). The basic service detects objects using dictionary data D written to the storage unit 240 by default, and this function can be used using the image sensor immediately after installation. When the service functions are changed or upgraded, the type, number, and combination of dictionary data used also change. For example, for the service pack WWW, the default dictionary data D is combined with dictionary data m1 and m2, and compiled into a dictionary data set suitable for detecting objects. Note that the dictionary data set mainly targets a data group including multiple dictionary data, but is not limited to this and also includes data consisting of one dictionary data (for example, the default dictionary data D).

In general, multiple dictionary data are used in combination for one detection item, rather than just one. Also, one dictionary data is often not used for only one detection item, but is commonly used for multiple detection items.

The price (billing plan) offered to the service subscriber can also be set for each service pack. In addition, the billing plan can also be set according to, for example, the detection accuracy (low accuracy, medium accuracy, high accuracy, etc.) and the number of types of detection items. The amount charged to the entity (e.g., the service subscriber) who downloaded the dictionary data included in the service pack is managed in a database in billing data 240c.

The processor 250 executes an operating system (OS) and various programs. The processor 250 also includes, as processing functions according to the embodiment, a selection unit 250a, a distribution unit 250b, a billing processing unit 250c, a timing designation unit 250d, a period designation unit 250e, and a key information generation unit 250f.

These functional blocks can be understood as processes that are generated when program 240a is loaded into RAM 230 and executed. That is, program 240a causes maintenance terminal 200, which is a computer, to operate as a selection unit 250a, a distribution unit 250b, an accounting processing unit 250c, a timing designation unit 250d, a period designation unit 250e, and a key information generation unit 250f.

The selection unit 250a selects a combination of dictionary data (dictionary data set) corresponding to the requested service for each of the image sensors 3-1 to 3-n. Services are often determined for each tenant in a building. For example, if there are three tenants as shown in FIG. 4, the image sensors can be grouped by tenant (groups A, B, and C) as shown in FIG. 7. In other words, the services can be associated with groups A, B, and C. Therefore, the selection unit 250a selects a dictionary data set for each of groups A, B, and C associated with the tenants, for example.

Of course, the optimal dictionary data set may be selected for each image sensor 3-1 to 3-n. For example, an image sensor near the entrance of tenant A may require different dictionary data than an image sensor located toward the back of the store. In this way, by selecting a dictionary data set according to the installation environment of the image sensor 3, more detailed services can be realized.

The distribution unit 250b reads out the dictionary data included in the dictionary data set selected by the selection unit 250a from the storage unit 240 and distributes it to the target image sensor. The distribution of the dictionary data is performed at the time specified by the timing specification unit 250d.

The timing designation unit 250d accepts a user operation for designating the timing for distributing dictionary data by the distribution unit 250b.
Fig. 7 is a diagram showing an example of a GUI window displayed on the monitor 210. For example, icons corresponding to the tenant layout and image sensor layout in Fig. 4 are displayed. For example, when group A is clicked with the mouse, a calendar and a drop-down list of data distribution times are displayed as shown in Fig. 8. When the user specifies, for example, 23:00 on 4th July 2018, this date and time are set in group A as shown in Fig. 9.

The timing specification unit 250d stores the timing of downloading the dictionary data specified in this manner in the storage unit 240 for each tenant, group, or image sensor. At the specified time, the distribution unit 250b downloads the dictionary data to the target image sensor.

The period designation unit 250e accepts a user operation for designating the operation period of the requested service.
The key information generating unit 250f generates key information for validating the dictionary data distributed by the distributing unit 250b during the operation period specified by the period specifying unit 250e. This key information is distributed to the target image sensor by the distributing unit 250b.

The billing processor 250c manages billing information for billing subscribers of the above-mentioned service in association with the downloading of dictionary data. In other words, the dictionary data downloaded to the image sensor 3 varies depending on the service, and therefore it is necessary to download and update the data each time. The billing processor 250c manages billing information that is generated when a download occurs, when a service starts operating, etc. The billing information is stored, for example, in the storage unit 240 (billing data 240c).
The selection unit 250a may select a dictionary data set based on a billing plan for the requested service.

Figure 10 is a block diagram showing an example of an image sensor 3. The image sensor 3 includes an imaging unit 31, a storage unit 32, a processor 33, and a communication unit 34. These are connected to each other via an internal bus 35.

The imaging unit 31 includes a fisheye lens 31a, an aperture mechanism 31b, an image sensor 31c, and a register 30. The fisheye lens 31a captures the interior of a room as a target space in a field of view, for example, looking down from the ceiling, and forms an image on the image sensor 31c. The image sensor 31c is an image sensor such as a CMOS (complementary metal oxide semiconductor), and generates a video signal at a frame rate of, for example, 30 frames per second. This video signal is digitally encoded and output as image data. The amount of light incident on the image sensor 31c is adjusted by the aperture mechanism 31b.

The register 30 stores camera information 30a. The camera information 30a is information about the imaging unit 31, such as the state of the auto gain control function, the gain value, and the exposure time, or information about the image sensor 3 itself.

The storage unit 32 is a semiconductor memory such as a dynamic random access memory (DRAM), a synchronous dynamic RAM (SDRAM), or an erasable programmable ROM (EPROM), and stores image data 32a acquired by the imaging unit 31, a program 32b for causing the processor 33 to execute various functions related to the embodiment, a sensor ID (IDentification) 32c for uniquely identifying the image sensor, and dictionary data 32d.

At the factory shipping stage, basic dictionary data that can be used only for human detection, for example, may be installed by default. Once the image sensor 3 has been installed at the site and services appropriate to that location are started, the required dictionary data 32d also changes. Therefore, in the embodiment, the functions of the image sensor 3 are changed or upgraded by downloading and updating the dictionary data 32d, for example, from the maintenance terminal 200. In other words, the functions of the image sensor 3 can be customized according to needs, rather than being general-purpose.

The processor 33 loads and executes programs stored in the storage unit 32 to realize various functions described in the embodiment. The processor 33 is, for example, a multi-core CPU and is an LSI (Large Scale Integration) tuned for high-speed image processing. The processor 15 can also be configured with an FPGA (Field Programmable Gate Array) or the like. An MPU is also one type of processor.
The communication unit 34 can be connected to the sensor network 100, and mediates data transmission and reception with communication destinations such as the gateway 7, the maintenance terminal 200, and the BEMS server 5. In particular, the communication unit 34 also functions as an acquisition unit that acquires dictionary data corresponding to a dictionary data set according to a requested service.

The processor 33 includes, as processing functions according to the embodiment, a detection unit 33a, a feature data calculation unit 33b, a learning unit 33c, a discrimination unit 33d, and a judgment unit 33e. The detection unit 33a, the feature data calculation unit 33b, the learning unit 33c, the discrimination unit 33d, and the judgment unit 33e are processes that are generated in the process in which the processor 33 executes arithmetic processing according to the program 32b loaded into the register of the processor 33, for example.

The detection unit 33a detects objects in the field of view using the dictionary data 32d. That is, the detection unit 33a detects objects in the object space from the image data 32a using a dictionary data set prepared for each detection item.

The feature data calculation unit 33b analyzes the image data 32a stored in the memory unit 32 using a predetermined algorithm, and calculates features such as histograms of oriented gradients (HOG) features, contrast, resolution, S/N ratio, color tone, co-occurrence histograms of oriented gradients (Co-HOG) features, and Haar-Like features.

The identification unit 33d identifies the detected object based on the dictionary data.
The determination unit 33e determines what the identified object is.
The learning unit 33c performs learning based on the result of the determination by the determination unit 33e and the image data 32a in order to improve the accuracy rate of the classification result by the classification unit 33d.

Figure 11 is a diagram showing an example of the data flow between the functional blocks of Figure 10. Image data from the imaging unit 31 is stored in the memory unit 32 and then passed to the detection unit 33a and the learning unit 33c. The detection unit 33a detects an object from the image data and passes the result to the identification unit 33d. The identification unit 33d identifies the detected object using dictionary data 32d and passes the identification score to the determination unit 33e. The determination unit 33e determines the detected object based on the identification score and passes the result to the communication unit 34. The communication unit 34 notifies the BEMS server 5, the service contractor, etc. of the determination result. The object determination result is also passed to the learning unit 33c and is fed back to improve the accuracy of the identification process by the identification unit 33d. Next, the operation of the above configuration will be described.

Figure 12 is a sequence chart showing an example of a processing procedure between the maintenance terminal 200 and the image sensor 3 for distributing dictionary data. Figure 12 shows an example of a push-type procedure in which the maintenance terminal 200 takes the initiative in transferring dictionary data to the image sensor 3.

When the distribution time of the dictionary data is set by the timing designation unit 250d of the maintenance terminal 200 (step S10), the maintenance terminal 200 waits for the distribution time to arrive (step S11). When the distribution time arrives (Yes in S11), the maintenance terminal 200 reads the dictionary data for each target image sensor from the storage unit 240 (step S13) and transmits one or more dictionary data to the target image sensor (step S14).

After receiving the dictionary data (step S50), the image sensor 3 returns an Ack message and its own sensor ID to the maintenance terminal 200 (step S51), and stores the downloaded dictionary data in the memory unit 32 (step S52).

The maintenance terminal 200 that has received the Ack message (step S15) writes billing information for the subscriber of the service that uses the image sensor to which the dictionary data has been downloaded, into the billing data 240c. The series of steps S13 to S16 (including steps S50 to S52) are repeated until completion for all target image sensors (loop of steps S12 and S17). When the dictionary data has been downloaded to all target image sensors, the maintenance terminal 200 waits for the dictionary data to be updated (step S18). When the dictionary data has been updated (Yes in step S18), the processing returns to step S11.

Figure 13 is a sequence chart showing another example of the processing procedure between the maintenance terminal 200 and the image sensor 3 involved in distributing dictionary data. Figure 13 shows an example of a pull-type procedure in which the image sensor 3 takes the lead in acquiring dictionary data from the maintenance terminal 200. The procedure that differs from Figure 12 will be mainly explained.

In FIG. 13, the distribution time of the set dictionary data is notified from the maintenance terminal 200 to the image sensor 3 (step S20). The image sensor 3 that has been notified of the distribution time (step S53) stores the distribution time in the memory unit 32 (S54) and waits for the arrival of the distribution time (step S55). When the distribution time arrives (Yes in S55), the image sensor 3 transmits a dictionary data acquisition request together with its own sensor ID to the maintenance terminal 200 (step S56).

When the maintenance terminal 200 receives the acquisition request (Yes in step S21), it transmits dictionary data for the image sensor of the sensor ID to the image sensor that made the request (steps S13 and S14). The procedures in steps S50 to S52 and steps S15 to S18 are the same as those in FIG. 12.

As described above, in the first embodiment, multiple dictionary data are stored in the maintenance terminal 200, and a dictionary data set that is a set of dictionary data corresponding to the requested service is selected for each image sensor. The user can then set the distribution time of the dictionary data set using a GUI, and when the distribution time arrives, the dictionary data is downloaded from the maintenance terminal 200 to the image sensor in a push or pull procedure.

As a result, the dictionary data can be updated even after installation (fixtures) of the image sensor 3 is complete. This makes it possible to update to the latest functions and flexibly respond to changes in the services required.

In addition, since the user can decide the timing of downloads, data updates can be performed during the night or on holidays, when the building network 500 is relatively quiet, and weekdays when heavy traffic is avoided, improving operational convenience.

As a result, it will be possible to provide a sensing system, maintenance terminal device, data distribution method, and image sensor that are even easier to use.

[Second embodiment] (a form in which a full set of dictionary data is installed in an image sensor)
In the first embodiment, an example is considered in which dictionary data is downloaded to the image sensor and updated each time a service change occurs, etc. In the second embodiment, a form is described in which a full set of dictionary data is installed in the image sensor in advance, and usable dictionary data is managed on the system side.

Figure 14 is a functional block diagram showing an example of a maintenance terminal 200 in the second embodiment. The following mainly describes the parts that differ from Figure 5. The processor 250 has a restriction unit 250g as a processing function related to the second embodiment. The restriction unit 250g can be understood as a process that is generated during the execution of the program 240a. In other words, the program 240a causes the maintenance terminal 200, which is a computer, to operate as the restriction unit 250g.

Figure 15 is a functional block diagram showing an example of an image sensor 3 in the second embodiment. The following mainly describes the parts that are different from Figure 10. The memory unit 32 of the image sensor 3 stores key data 32e. The image sensor 3 also downloads a full set of dictionary data 32d as shown in Figure 16 from the maintenance terminal 200 and stores it in the memory unit 32.

Figure 16 is a diagram showing an example of dictionary data 32d stored in Figure 15. For example, all of the m-series dictionary data (m1 to m5) for "person detection", the C-series dictionary data (C1 to C5) for "chair detection", the O-series dictionary data (O1 to O5) for "object detection", and the a, b, c, d, and e dictionary data (m1 to m5) for "specific person detection" are stored in the memory unit 32 of the image sensor 3 together with the default dictionary data D.

The restriction unit 250g in FIG. 14 restricts the use of the remaining dictionary data so that the dictionary data corresponding to the dictionary data set selected by the selection unit 250a can be used in the corresponding image sensor. To distinguish between usable and unusable dictionary data, for example, key information generated by the key information generation unit 250f may be used. This key information is distributed from the maintenance terminal 200 to the image sensor 3, for example, when downloading the dictionary data set, and is stored in the storage unit 32 as key data 32e.

Figures 17 and 18 are diagrams for explaining that the dictionary data that can be used is restricted depending on the service content. Figure 17 shows that when providing the service pack WWW for human detection, only dictionary data D, m1, and m2 can be used, and other dictionary data is disabled as [Not Available]. The disabled data is in a state where it cannot be used by the program 32b (application) of the image sensor 3 due to the key data 32g.

Figure 18 shows dictionary data for service pack XXX, where the keys for dictionary data D, m1, and m2 as well as dictionary data O1 and O2 have been unlocked. This allows the system to detect not only people but also objects.

In this way, in the second embodiment, a full set of dictionary data is installed in advance in the image sensor 3, and only dictionary data that is permitted for use is enabled, while the remaining dictionary data is disabled (not available). By doing this, when the service content changes, new key information is generated and only the key information is distributed to the target image sensor. Therefore, it is not necessary to distribute dictionary data, which has a relatively large amount of data, each time, and it is possible to ease the strain on communication resources.

[Third embodiment] (embodiment in which occupancy conditions can be set for an image sensor)
In the third embodiment, a form will be described in which a right (occupancy right) to exclusively use the image sensor 3 for a certain period of time is given to a third party, making it possible to develop a new service.

Figure 19 is a block diagram showing an example of a sensing system according to a third embodiment of the present invention. A server provided by a service provider (hereinafter referred to as a vendor server 400) is connected to an in-building network 500 via a gateway 8.

Figure 20 is a functional block diagram showing an example of the maintenance terminal 200 in the third embodiment. The processor 250 includes an occupancy rights management unit 250h. The occupancy rights management unit 250h is understood as a process generated during the execution of the program 240a, and the program 240a causes the maintenance terminal 200, which is a computer, to operate as the occupancy rights management unit 250h.

The occupancy rights management unit 250h manages the occupancy rights of the image sensors. For example, a certain period of time is set using a method similar to that described in Figures 7 to 9, and the service provider having the vendor server 400 is given the authority to occupy all or part of the image sensors 3-1 to 3-n for that period of time.

The image sensor 3 that has been granted the exclusive right is then given the opportunity to download private dictionary data from the vendor server 400 and provide a unique service based on the dedicated dictionary data. The billing processing unit 250c then performs billing processing for the exclusive right, and stores the billing data for the service provider in the memory unit 200 of the maintenance terminal 200 (billing data 240c).

Alternatively, as described in the second embodiment, an exclusive right may be set for an image sensor that stores a full set of dictionary data, and the usable dictionary data may be distinguished from the unusable dictionary data by the same method as in the second embodiment using the key data 32e shown in the storage unit 32 in FIG. 21. It is also possible to set the expiration date of the exclusive right by giving the key data 32e itself an expiration date.

The distribution unit 250b of the maintenance terminal 200 may execute a process to distribute the dictionary data set from the vendor server 400 of the service provider that has been granted the right of occupancy to the image sensor that has been granted occupancy.

As described above, in the third embodiment, the right to occupy the image sensor 3 is given to a third-party service provider, and a fee can be charged for the right to occupy. In this way, it becomes possible to provide an even more diverse range of services.

It should be noted that the present invention is not limited to the above-described embodiment.
For example, the term "full set" in the second embodiment does not necessarily mean all dictionary data. In short, a large amount of dictionary data may be collectively transferred to the image sensor within the range permitted by the system requirements. By appropriately setting the transfer timing, it is possible to minimize the strain on resources such as transmission channel capacity.

In the first embodiment, the image sensors 3 are grouped by tenant, and the timing of downloading dictionary data is managed on a group-by-group basis, but this is just one example. It is of course possible to set the timing of downloading dictionary data individually for each image sensor 3. In addition, it is possible to manage the image sensors 3 in any group.

Dictionary data is not necessarily downloaded from the maintenance terminal 200 to all image sensors. For example, when updating dictionary data, only one image sensor per group (or tenant) may communicate with the maintenance terminal 200, and the acquired dictionary data may be distributed as image data within the group, for example in a relay format.

Furthermore, at least a part of the functions of the sensing system of the embodiment may be configured by a cloud server device. In other words, at least a part of the processing performed by the sensing system of the embodiment may be executed by cloud computing.

Figure 22 is a diagram showing a form in which some functions are transferred to the cloud 600. The maintenance terminal 200 can communicate with the cloud 600 via a firewall 6 connected to the building network 500, and some or all of the functions shown in Figures 5, 14, and 20 can be transferred to the cloud 600.

This cloud computing may include at least one of SaaS (Software as a Service), which provides applications (software) as a service, PaaS (Platform as a Service), which provides a platform for running applications as a service, and IaaS (Infrastructure as a Service), which provides resources such as server devices, central processing units, and storage as a service (public cloud). For example, this cloud computing may include remote operation via the Internet by the cloud service provider layer (PaaS). Furthermore, the maintenance terminal 200 or the BEMS server 5 may be a cloud server.

Figure 23 is a diagram showing an example in which the functions according to the first embodiment are installed in the BEMS server 5 instead of the maintenance terminal 200. It goes without saying that the effects of the present invention can be obtained in this way as well. The same applies to the second and third embodiments. Furthermore, some or all of the functions shown in Figure 23 can be transferred to the cloud 600.

Other functions and effects of the embodiment of the present invention are listed below.
- Functional restrictions and schedules for the full-function dictionary of the image sensor shall be instructed to the image sensor from the higher-level system.

- When updating the image sensor dictionary, the dictionary to be downloaded and the schedule can be set for each image sensor or group.

- The host system will have a function for distributing dictionaries to image sensors. It will have a function and a setting screen that allows you to set the dictionary to be distributed for each image sensor or group and the distribution schedule.

- To update the dictionary of the image sensor, the user can create a dictionary at will and distribute it directly to the image sensor using a maintenance PC terminal, etc.

- For image sensors with the above functions, a system will be in place to charge users according to the dictionary functions they use (both full-function software restrictions and optional function additions/updates). In addition, a system will be in place that allows users to occupy a specific group of image sensors for a fee.

- Regarding billing, there is a table that indicates which dictionary functions are assigned to each image sensor or group in association with a subscriber ID (or tenant ID), and this table is updated when new dictionary data is sent, and the billing table associated with the subscriber ID (or tenant ID) is updated. Alternatively, billing is performed when a response is received from the image sensor indicating that the update has been completed without any problems. The upper system has a mechanism for monitoring such billing timing.

Note that although the processor 33 and memory unit 32 are described here as being singular, two or more processors and memories may be provided. As an example, a case in which a first and second processor are provided will be described. The first processor is a dedicated image recognition processor capable of realizing high-speed image recognition, and has the functions of the detection unit 33a, feature data calculation unit 33b, learning unit 33c, identification unit 33d, and judgment unit 33e in FIG. 10, and can access a high-speed memory unit such as a DRAM that can read and write at high speed. The second processor is a general-purpose processor, and has the functions of saving, managing, and communicating the image processing results recognized by the first processor, and can access a large-capacity memory unit such as a flash memory.

In this case, dictionary data 32d is stored in a large-capacity storage unit accessible to the second processor, and dictionary data D, m1, m2, O1, and O2 are saved therein. A dictionary data selection unit (not shown) executed by the first processor or the second processor selects dictionary data to be used by the first processor from the dictionary data stored in the large-capacity storage unit, stores the selected dictionary data in the high-speed storage unit, and the first processor executes a specified operation.

By using a processor dedicated to image recognition in this way, it is possible to realize high-speed image processing, and it can be realized even in cases where a high-speed response is required in response to the presence or absence of people and their actions, such as lighting control. Furthermore, even if the detection target is changed due to a change in the service pack, there is no need to rewrite the dictionary data 32d, and the detection target can be changed simply by changing the target selected by the dictionary data selection unit.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the inventions described in the claims and their equivalents. The inventions described in the original claims of this application are appended below.
[1]
An image sensor that detects an object using a dictionary data set prepared for each detection item; and a data server that stores a plurality of dictionary data.
A selection means for selecting a dictionary data set for each of the image sensors according to a requested service;
and a distribution means for distributing dictionary data corresponding to the selected dictionary data set from the data server to the image sensor.
[2]
An image sensor that detects an object using a dictionary data set prepared for each detection item; and a data server that stores a plurality of dictionary data.
a distribution means for distributing a full set of dictionary data from the data server to the image sensor;
A selection means for selecting a dictionary data set for each of the image sensors according to a requested service;
and limiting means for limiting the use of dictionary data corresponding to the selected dictionary data set in the image sensor.
[3]
an image sensor that detects an object using a dictionary data set prepared for each detection item; and a management means for managing the ownership of the image sensor.
and a distribution means for distributing dictionary data corresponding to the dictionary data set from a data server of the entity to which the exclusive right has been granted to the image sensor.
[4]
The sensing system according to any one of appendices [1] to [3], wherein the distribution means transfers the dictionary data from the data server to the image sensor in a push-type procedure.
[5]
The sensing system according to any one of appendices [1] to [3], wherein the distribution means obtains the dictionary data from the data server in a pull-type procedure by the image sensor.
[6]
The method further includes a timing designation means for designating a distribution timing of the dictionary data,
The sensing system according to any one of appendices [1] to [3], wherein the distribution means distributes the dictionary data at the specified timing.
[7]
The sensing system according to claim 1 or 2, further comprising a billing processing means for managing billing information for billing subscribers of the service.
[8]
The sensing system according to claim 7, wherein the selection means selects the dictionary data based on a billing plan for the requested service.
[9]
The image sensor is installed in a building including a plurality of tenants;
The sensing system according to claim 1 or 2, wherein the selection means selects the dictionary data for each tenant.
[10]
a period designation means for designating a period during which the dictionary data is to be available by the restriction means;
a key information generating means for generating key information for validating the dictionary data during the specified period;
The sensing system described in appendix [2], wherein the distribution means distributes the generated key information to a target image sensor.
[11]
a period designation means for designating a valid period of the right of possession;
a key information generating means for generating key information for validating the dictionary data during the validity period;
The sensing system described in appendix [3], wherein the distribution means distributes the generated key information to a target image sensor.
[12]
A maintenance terminal device applicable to a sensing system having a plurality of image sensors, comprising: a storage unit that stores dictionary data used in the image sensors;
A processor that functions based on a program,
The processor,
A selection unit that selects a dictionary data set corresponding to a requested service for each of the image sensors;
a distribution unit that distributes dictionary data corresponding to the selected dictionary data set from the storage unit to the image sensor.
[13]
A maintenance terminal device applicable to a sensing system having a plurality of image sensors, comprising: a storage unit that stores dictionary data used in the image sensors;
A processor that functions based on a program,
The processor,
a distribution unit that distributes a full set of dictionary data to the image sensor;
A selection unit that selects a dictionary data set for each of the image sensors according to a requested service;
a restriction unit that allows dictionary data corresponding to the selected dictionary data set to be used in the image sensor.
[14]
A maintenance terminal device applicable to a sensing system having a plurality of image sensors, comprising: a storage unit that stores dictionary data used in the image sensors;
A processor that functions based on a program,
The processor,
a management unit that manages the ownership of the image sensor;
a distribution unit that distributes a dictionary data set from a data server of the entity that has been granted the occupancy right to the image sensor.
[15]
The maintenance terminal device according to any one of appendices [12] to [14], wherein the distribution unit transfers the dictionary data to the image sensor in a push-type procedure.
[16]
The processor,
The maintenance terminal device according to any one of appendices [12] to [14], further comprising a timing designation unit for designating a distribution timing of the dictionary data, wherein the distribution unit distributes the dictionary data at the designated timing.
[17]
The maintenance terminal device according to any one of appendices [12] and [13] further comprises an accounting processing unit that manages accounting information for charging a subscriber of the service.
[18]
The maintenance terminal device according to claim 17, wherein the selection unit selects the dictionary data based on a billing plan for the requested service.
[19]
When the image sensor is installed in a building including multiple tenants,
The maintenance terminal device according to any one of appendices [12] and [13], wherein the selection unit selects the dictionary data for each tenant.
[20]
a period designation means for designating a period during which the dictionary data is available to the restriction unit;
a key information generating unit that generates key information for validating the dictionary data during the specified period,
The maintenance terminal device according to claim 13, wherein the distribution unit distributes the generated key information to a target image sensor.
[21]
a period designation unit for designating a valid period of the right of possession;
a key information generating unit that generates key information for validating the dictionary data during the validity period,
The maintenance terminal device according to claim 14, wherein the distribution unit distributes the generated key information to a target image sensor.
[22]
A data dissemination method applicable to a sensing system having an image sensor that detects an object using a dictionary data set prepared for each detection item, comprising:
selecting, by a computer, a dictionary data set for each of the image sensors according to a requested service;
and a step of distributing, by a computer, dictionary data corresponding to the selected dictionary data set from a data server storing a plurality of dictionary data to the image sensor.
[23]
A data dissemination method applicable to a sensing system having an image sensor that detects an object using a dictionary data set prepared for each detection item, comprising:
A step in which a computer distributes a full set of dictionary data from a data server storing a plurality of dictionary data to the image sensor;
selecting, by a computer, a dictionary data set for each of the image sensors according to a requested service;
and a computer step of making dictionary data corresponding to the selected dictionary data set available to the image sensor.
[24]
A data dissemination method applicable to a sensing system having an image sensor that detects an object using a dictionary data set prepared for each detection item, comprising:
a computer managing ownership of the image sensor;
and a computer step of distributing dictionary data corresponding to the dictionary data set from the proprietary entity's data server to the image sensor.
[25]
An imaging unit that captures an image of a target space and acquires image data;
a detection unit that detects an object in the object space from the image data using a dictionary data set prepared for each detection item;
an acquisition unit that acquires dictionary data corresponding to a dictionary data set according to a requested service;
and a storage unit that stores the acquired dictionary data.
[26]
An imaging unit that captures an image of a target space and acquires image data;
a storage unit for storing a full set of dictionary data;
a detector configured to detect objects in the object space from the image data using dictionary data corresponding to a dictionary data set selected according to a requested service.
[27]
1. An image sensor comprising:
An imaging unit that captures an image of a target space and acquires image data;
a detection unit that detects an object in the object space from the image data using a dictionary data set prepared for each detection item;
an acquisition unit that acquires dictionary data from a subject that has been granted an occupancy right to the sensor;
and a storage unit that stores the acquired dictionary data.

1 (1-1, 1-2)... Lighting equipment, 2... Air conditioning equipment, 3 (3-1 to 3-n)... Image sensor, 4... Controller, 5... BEMS server, 6... Firewall, 7, 8... Gateway, 10... Building, 15... Processor, 30... Register, 30a... Camera information, 31... Imaging unit, 31a... Fisheye lens, 31b... Aperture mechanism, 31c... Imaging element, 32... Storage unit, 32a... Image data, 32b... Program, 32d... Dictionary data, 32e... Key data, 32g... Key data, 33... Processor, 33a... Detection unit, 33b... Feature data calculation unit, 33c... Learning unit, 33d... Identification unit, 33e... Judgment unit, 34... Communication unit, 35... Internal bus, 41... Air conditioning controller, 42... Elevator controller, 43... Defense Crime controller, 44... disaster prevention controller, 45... lighting controller, 70... database, 80... server, 100... sensor network, 200... maintenance terminal, 210... monitor, 220... ROM, 230... RAM, 240... storage unit, 240a... program, 240b... dictionary data, 240c... billing data, 250... processor, 250a... selection unit, 250b... distribution unit, 250c... billing processing unit, 250d... timing designation unit, 250e... period designation unit, 250f... key information generation unit, 250g... restriction unit, 250h... occupancy rights management unit, 260... optical media drive, 270... communication unit, 300... communication network, 400... vendor server, 500... building network, 600... cloud.

Claims (11)

an image sensor that detects an object using a dictionary data set prepared for each detection item; and a management means for managing the ownership of the image sensor.
and a distribution means for distributing dictionary data corresponding to the dictionary data set from a data server of the entity to which the exclusive right has been granted to the image sensor. The sensing system according to claim 1, wherein the distribution means transfers the dictionary data from the data server to the image sensor in a push-type procedure. The sensing system according to claim 1, wherein the distribution means acquires the dictionary data from the data server in a pull-type procedure using the image sensor. The method further includes a timing designation means for designating a distribution timing of the dictionary data,
The sensing system according to claim 1 , wherein the distribution means distributes the dictionary data at the specified timing. a period designation means for designating a valid period of the right of possession;
a key information generating means for generating key information for validating the dictionary data during the validity period;
The sensing system of claim 1 , wherein the distribution means distributes the generated key information to a target image sensor. A maintenance terminal device applicable to a sensing system having a plurality of image sensors, comprising: a storage unit that stores dictionary data used in the image sensors;
A processor that functions based on a program,
The processor,
a management unit that manages the ownership of the image sensor;
a distribution unit that distributes a dictionary data set from a data server of the entity that has been granted the occupancy right to the image sensor. The maintenance terminal device according to claim 6, wherein the distribution unit transfers the dictionary data to the image sensor using a push-type procedure. The processor,
7. The maintenance terminal device according to claim 6, further comprising a timing designation unit for designating a distribution timing of said dictionary data, said distribution unit distributing said dictionary data at said designated timing. a period designation unit for designating a valid period of the right of possession;
a key information generating unit that generates key information for validating the dictionary data during the validity period,
The maintenance terminal device according to claim 6 , wherein the distributor distributes the generated key information to a target image sensor. A data dissemination method applicable to a sensing system having an image sensor that detects an object using a dictionary data set prepared for each detection item, comprising:
a computer managing ownership of the image sensor;
and a computer step of distributing dictionary data corresponding to the dictionary data set from the proprietary entity's data server to the image sensor. 1. An image sensor comprising:
An imaging unit that captures an image of a target space and acquires image data;
a detection unit that detects an object in the object space from the image data using a dictionary data set prepared for each detection item;
an acquisition unit that acquires dictionary data from a subject that has been granted an occupancy right to the sensor;
and a storage unit that stores the acquired dictionary data.