JP6040804B2 - Object identification system and object identification method - Google Patents

Description

  The present invention relates to a technique for identifying an object in a space.

  With the spread of computers, manuals such as operation manuals and work procedure manuals that have been conventionally created on paper media have been digitized. Electronic manuals are mainly those in which paper manuals are simply converted into electronic documents, but electronic manuals that can perform interactive operations on computers have recently appeared.

  For example, Patent Document 1 describes a device that, when a target device is photographed with a built-in camera, analyzes an image to determine a target object and displays an electronic manual suitable for the device. By using this apparatus, it is possible to obtain an electronic manual by a simple operation without taking the trouble of searching a library using the model name.

In addition, an electronic manual has been devised for explaining operations on devices using AR (Augmented Reality) technology. For example, Non-Patent Document 1 describes a system that detects an object to which a barcode is attached through a camera and displays an operation description of the object in an overlay on a captured image.

  When such a system is used, the user can acquire an operation description corresponding to the device simply by pointing the camera toward the target device. In addition, since the information is presented to the user by an image or a moving image, a large amount of information can be transmitted compared to a manual using a paper medium.

JP2011-158959A

"Business support system by AR", [online], Nippon Steel Solutions, [Search January 23, 2013], Internet <URL: http://www.ipros.jp/advertising/detail/24553/>

  Conventional electronic manuals discriminate objects using image recognition technology. For example, the apparatus described in Patent Literature 1 discriminates an object by pattern-matching an image taken by a user with an image stored in advance.

However, when an object is discriminated using an image, there is a problem that recognition accuracy is lowered depending on the environment. For example, when an electronic manual is used in a car, a clear image cannot be obtained depending on the time zone or place, and there is a high possibility that image recognition will fail. In particular, at night, indoor parking lots, etc., it is often impossible to obtain sufficient brightness for image recognition even if the room light is turned on.
On the other hand, in the system described in Non-Patent Document 1, since the object is discriminated by reading the two-dimensional barcode attached to the apparatus, higher recognition accuracy can be obtained compared to the method of matching images. . However, it is not practical to attach barcodes to any parts referenced in the manual due to visual problems.

  The present invention has been made in consideration of the above-described problems, and an object thereof is to provide an object identification system for accurately identifying an object.

  In order to solve the above-described problem, the object identification system according to the present invention is configured to narrow down candidate objects by executing a process for detecting the position of the object in addition to the object discrimination process based on the image. I took.

  An object identification system according to the present invention includes a mobile terminal equipped with a camera and a management server, and is an object imaged by a camera among a plurality of objects arranged substantially fixed relative to each other. It is the object identification system which identifies.

  The target object is an article that is relatively fixedly arranged, for example, a lever or a button that is arranged in a passenger compartment of an automobile. The size and range of the object can be arbitrary. For example, when considering the interior of an automobile, the object may be an air conditioner, a navigation device, or the like, or a temperature adjustment button, a scale designation button, or the like. Any object may be used as long as the relative position does not change. Further, the object may be movable as long as the approximate position is determined. For example, a winker lever that can move up and down may be used.

  Further, the mobile terminal includes a reference position detection unit that detects that the terminal is at a reference position that is a predetermined position, and a displacement detection unit that detects a displacement of the terminal from the reference position, Have

The reference position detection means is means for detecting that the mobile terminal is at the reference position. The reference position is a point serving as a reference representing the arrangement position of the object. For example, when the object is placed in a car, the reference position is a point in the car. If the positional relationship with the object does not change, the reference position may be provided anywhere.
The reference position may be detected by, for example, a user placing the mobile terminal at the reference position and pressing a button, or by detecting a marker or the like placed at the reference position with a sensor or camera. Also good.
The displacement detection means is means for detecting the amount of displacement of the mobile terminal from the reference position. The amount of displacement can be expressed, for example, by a moving distance from the reference position for each of the X, Y, and Z axes.

  In addition, the management server includes an object storage unit that stores arrangement information that is information representing the arrangement of the object, a displacement detected by the displacement detection unit, the arrangement information, and an image captured by the camera. Object identifying means for identifying the object based on the object.

  The object storage means is means for storing information (placement information) indicating where the object is placed in the space. The arrangement information may be any information that represents the position of the object with reference to the reference position, and is not necessarily the coordinates of the object itself. For example, a point (observation point) for observing the object may be determined, and the coordinates of the observation point may be used as arrangement information, or an elevation angle, an azimuth angle, etc. when the object is observed from the observation point is added. There may be.

The object identifying means is means for identifying the object using the displacement from the reference position of the mobile terminal and the stored arrangement information in addition to the image captured by the camera. Since the arrangement information is information representing the position of the object with reference to the reference position, the object in the vicinity of the portable terminal can be narrowed down by comparing with the displacement from the reference position of the portable terminal.

  As described above, the management server according to the present invention identifies the object by further using the displacement of the mobile terminal in addition to the identification processing based on the image. Since the object is identified without relying only on the image, the object can be identified even in an environment where matching with the image cannot be performed accurately, such as in a dark place.

  The portable terminal may further include a three-axis acceleration sensor, and the displacement detection unit may detect a displacement from the reference position based on the acceleration detected by the acceleration sensor.

  A three-axis acceleration sensor is a sensor that can acquire acceleration in each of three-dimensional axial directions. Since the speed can be obtained from the acquired acceleration and the moving distance can be obtained from the speed, the displacement detection means can calculate the displacement of each of the three axes based on the sensor information.

  The reference position detection means may further include an input acquisition means for acquiring an input operation from a user, and may determine that the portable terminal is at a reference position based on the input operation. .

  Whether the portable terminal is at the reference position may be determined based on a user input operation. For example, the user may indicate that the mobile terminal is at the reference position by pressing a button with the mobile terminal placed at the reference position.

  The reference position detection unit may determine that the portable terminal is at the reference position when a marker representing the reference position is captured by the camera.

  In this way, a marker for detecting the reference position may be attached, and the reference position may be automatically detected using a camera. By doing so, it is not necessary to place the portable terminal once, so that the object identification system according to the present invention can be used even when it is not possible to provide a place for placing the terminal.

  The portable terminal may further include a tilt sensor that detects the tilt of the terminal, and the object identifying unit may further identify the target by further using the tilt detected by the tilt sensor. .

  In addition to the displacement of the mobile terminal, the management server can further specify the direction vector of the mobile terminal by transmitting information related to the tilt of the mobile terminal to the management server. That is, since it becomes possible to know where the camera of the mobile terminal is facing, the objects can be further narrowed down and the identification accuracy of the objects can be improved. The tilt sensor may be any sensor as long as it can detect the tilt. For example, an acceleration sensor or a gyro sensor may be used.

  The object identification system according to the present invention can also be specified as an information providing system that provides information about an object.

For example, the management server may further store related information about the target object such as an instruction manual or work procedure manual of the target object, and transmit information related to the identified target object to the portable terminal. . By doing in this way, the target object identification system which concerns on this invention can be utilized as an information provision system. For example, it is possible to acquire an electronic manual corresponding to a part imaged by a mobile terminal.

  In addition, this invention can be specified as an object identification system containing at least one part of the said means. The present invention can also be specified as an object identification method including at least a part of the above processing. The above processes and means can be freely combined and implemented as long as no technical contradiction occurs.

  ADVANTAGE OF THE INVENTION According to this invention, the target object identification system which identifies the target fixed relatively accurately can be provided.

It is a figure showing the inside of a car. 1 is a system configuration diagram of a mobile terminal according to a first embodiment. It is a system configuration figure of the management server concerning a first embodiment. It is an example of the site | part information which concerns on 1st embodiment. It is a figure showing the state which put the portable terminal in the reference position. It is a processing flowchart concerning a first embodiment. It is an example of the screen displayed on a portable terminal in 1st embodiment. It is an example of the site | part information which concerns on the modification of 1st embodiment. It is an example of the screen displayed on a portable terminal in 2nd embodiment. It is a figure explaining the reference position in 2nd embodiment. It is an example of the screen displayed on a portable terminal in a modification.

(First embodiment)
<System overview>
The electronic manual system according to the first embodiment is a system that provides electronic data (hereinafter referred to as an electronic manual), which is an instruction manual of an automobile, to a user. The mobile terminal 10 carried by the user and the management server 20 Consists of.
The management server 20 is a server in which electronic manuals are stored, and the mobile terminal 10 is a terminal with a built-in camera for acquiring and displaying electronic manuals from the management server 20. A method for acquiring a manual desired by the user from the management server 20 will be described with reference to FIGS.

The mobile terminal 10 transmits to the management server 20 information indicating which part in the vehicle the electronic manual that the user wants, and the management server 20 determines the target part based on the information. A corresponding one is extracted from the electronic manual that is specified and stored, and is transmitted to the mobile terminal 10.
A specific example is given. FIG. 1 is a diagram showing the interior of a car. For example, assume that the user wants to refer to the manual for the shift knob (reference numeral 101). In this case, it is necessary to transmit information for specifying the shift knob from the mobile terminal 10 to the management server 20.

In a conventional electronic manual device, an image is acquired by photographing a target part with a camera, and the part is collated using the image (hereinafter, a photographed image). In the present embodiment, in addition to the captured image, the mobile terminal 10 transmits information representing the position and tilt of the camera when the image is captured to the management server 20. Thereby, the management server can know which direction the camera is looking from which position. That is, if the management server has position information of each part in the vehicle, the part captured by the camera can be estimated by comparing the two, and this can be used together with image recognition. The identification accuracy can be improved.

This will be described more specifically. The mobile terminal 10 according to the present embodiment transmits the following two pieces of information to the management server 20 together with the captured image. These pieces of information are referred to as instruction information.
(1) Displacement amount of portable terminal with respect to reference position at the time of shooting (movement distance in each axis direction)
(2) Mobile device tilt (direction vector) at the time of shooting
The reference position is a reference position in the vehicle. In the case of the example in FIG. 1, the area (reference numeral 102) provided on the accessory case is used as a reference position, and information indicating how much the terminal is displaced with respect to the reference position in each of the X, Y, and Z axes is generated. Then send. In addition, information (direction vector) indicating the attitude of the terminal is generated and transmitted. The timing for generating information is the timing at which the user images the target part.

  As described above, the mobile terminal according to the present embodiment transmits the instruction information including the above (1) and (2) to the management server in addition to the photographed image, and the management server allows the user to Narrow down the indicated part. In the following description, (1) is referred to as camera position information, and (2) is referred to as a camera direction vector. A method for the mobile terminal 10 to acquire each and a method for the management server to narrow down the objects based on the instruction information will be described later.

<System configuration>
Next, system configurations of the mobile terminal 10 and the management server 20 will be described with reference to FIG.
The mobile terminal 10 is typically a mobile information terminal such as a mobile phone or a smartphone. The portable terminal 10 has a CPU, a main storage device, and an auxiliary storage device, and the programs stored in the auxiliary storage device are loaded into the main storage device and executed by the CPU, whereby each unit illustrated in FIG. Functions (CPU, main memory, and auxiliary memory are all not shown).

The communication unit 11 is a communication unit for communicating with the management server 20. It is possible to communicate with the management server 20 via a network using a mobile communication service such as 3G or LTE.
The camera 12 is a means for taking a still image. The user uses the camera 12 to photograph a part in the vehicle for which an electronic manual is desired.
The acceleration sensor 13 is a means for measuring the acceleration applied in the three directions (X, Y, Z axes) and outputting it as sensor information. The acceleration sensor 13 can output both a value including a gravitational acceleration component and a value not including the gravitational acceleration component. When the gravitational acceleration component is not included, the acceleration for each axis is 0 when the mobile terminal is stationary.

The input / output unit 14 is means for accepting an input operation performed by the user and presenting information to the user. Information can be input and output by means such as a touch screen. The input / output unit 14 is an input acquisition unit and an information provision unit in the present invention.
The control unit 15 is means for controlling the operation of the mobile terminal 10. Specifically, instruction information is generated from the information acquired from the acceleration sensor 13 and transmitted to the management server 20 together with the captured image acquired from the camera 12, and the electronic manual transmitted from the management server 20 is received. Processing to be presented to the user through the output unit 14 is executed.
The control unit 15 is a reference position detection unit, a displacement detection unit, and an object identification unit in the present invention.

Next, the system configuration of the management server 20 will be described with reference to FIG. The management server 20 according to the first embodiment stores a plurality of electronic manuals corresponding to each part in the vehicle and information on the position of each part, and when a request is received from the mobile terminal 10, A computer that acquires and transmits the corresponding electronic manual.

  The management server 20 also has a CPU, a main storage device, and an auxiliary storage device, similar to the portable terminal 10, and each unit illustrated in FIG. 3 functions by executing a stored program. The management server 20 may be a single computer or a combination of a plurality of computers.

The communication unit 21 is means for communicating with the mobile terminal 10 via a network.
The storage unit 22 is a means for storing a plurality of electronic manuals and information (location information in the present invention, hereinafter referred to as part information) indicating where the parts corresponding to the electronic manual are in the vehicle.
FIG. 4 is an example of part information stored in the storage unit 22. In the present embodiment, the part information includes information for identifying the part identifier and name, camera position information, camera orientation vector, and electronic manual data.

The camera position information included in the part information is a pre-stored amount of displacement from the reference position of the terminal when the user images the target part with the mobile terminal, and the camera orientation vector included in the part information. Means that the direction vector of the terminal is stored in advance when the user images the target part with the portable terminal. The management server 20 collates the camera position information and camera direction vector included in the received instruction information with the camera position information and camera direction vector stored in the storage unit 22, so that the camera of the mobile terminal 10 is directed. Estimate the location.
Although not shown here, the storage unit 22 separately stores an electronic manual (moving image, electronic document, image, etc.) corresponding to each part and a template image for matching a captured image. Yes. The storage unit 22 is an object storage unit and an object information storage unit in the present invention.

  The control unit 23 is means for performing identification processing of the target part by collating the received instruction information and captured image with information stored in the storage unit 22. Details of the processing will be described later.

<How to generate instruction information>
Next, a method for generating instruction information by the mobile terminal 10 will be described. As described above, the instruction information includes camera position information and a camera orientation vector.

First, a method for obtaining camera position information, that is, a displacement amount of the mobile terminal with respect to a reference position will be described. FIG. 5 is a diagram illustrating a state in which the mobile terminal is placed at the reference position (reference numeral 102 in FIG. 1). In this state, the displacement amount of the mobile terminal is 0 for each axis. Here, the camera position information is (x _camera , y _camera , z _camera ). The time when the mobile terminal 10 is placed at the reference position is t ₀ .
Then, the user moves the mobile terminal to a position where the target part can be imaged, and performs imaging. Let t ₁ be the time of shooting. If the accelerations acquired from time t ₀ to t ₁ are α _x (t), α _y (t), and α _z (t), respectively, the camera position information (x _camera , y _camera , z _camera ) is It can be obtained by Equation 1. The integration interval is from t ₀ to t ₁ , and α _x (t), α _y (t), and α _z (t) are values that do not include the gravitational acceleration component.

Next, a method for obtaining the camera direction vector, that is, the direction vector of the mobile terminal will be described. If the user obtains accelerations acquired at the timing when the target part is imaged by the camera as α _x0 , α _y0 , and α _z0 , respectively, the inclination (θ _x , θ _y , θ _z )
Can be obtained by Equation 2. Α _x0 , α _y0 , and α _z0 are values including a component of gravitational acceleration.

From the inclinations (θ _x , θ _y , θ _z ) obtained as described above, the camera orientation vector R can be obtained by Equation 3.

  The portable terminal 10 transmits the information obtained as described above to the management server 20 as instruction information. Thereby, the management server 20 can acquire the position and orientation vector of the mobile terminal at the time of capturing the target region.

<Process flowchart>
Next, identification processing performed by the electronic manual system according to the present embodiment will be specifically described with reference to FIG. 6 which is a processing flowchart and FIG. 7 which is an example of a screen displayed on the mobile terminal 10. .

Steps S11 to S13 and step S19 are processes performed by the mobile terminal 10, and steps S14 to S18 are processes performed by the management server 20.
In step S11, first, the control unit 15 displays a screen to start searching for an electronic manual, and prompts the user to place the terminal at the reference position and press a button on the screen (FIG. 7). (A)). When the user places the terminal at a reference position (for example, an area denoted by reference numeral 102 in FIG. 1) and presses the button, the control unit 15 starts to acquire acceleration periodically from the acceleration sensor 13.

  In step S12, the control unit 15 activates the camera 12, displays a screen for performing imaging, and prompts the imaging of the target part. FIG. 7B shows an example where the user photographs the wiper lever. When the shooting button is pressed, the control unit 15 acquires an image through the camera 12. And the acquisition of the acceleration from the acceleration sensor 13 is complete | finished, the acquired acceleration is memorize | stored temporarily, and it changes to step S13.

  Step S <b> 13 is a step in which the control unit 15 generates instruction information based on the acceleration acquired from the acceleration sensor 13. First, the amount of displacement from the reference position is acquired using Equation 1. In this example, since the acceleration is a discrete value acquired periodically, the velocity and the displacement are acquired not by integration but by calculating the sum of the periodically acquired acceleration. Thereby, camera position information is generated. Also, the camera orientation vector is generated using Equations 2 and 3. And the produced | generated information is transmitted to the management server 20 through the communication part 11 with a picked-up image.

  In step S <b> 14, the control unit 23 acquires the transmitted instruction information and captured image through the communication unit 21.

Step S15 is a process in which the control unit 23 compares the camera position information included in the acquired instruction information with the camera position information of each part stored in the storage unit 22, and extracts a part having a high degree of similarity. is there. Since the camera position information is a movement distance from the reference position in each of the X, Y, and Z axes, the similarity can be evaluated by taking the square of the distance. For example, when the camera position information included in the instruction information is (x _cam , y _cam , z _cam ) and the camera position information included in the part information is (x _ref , y _ref , z _ref ), the similarity r _{camera_place} can be expressed by Equation 4.
In step S15, r _{camera_place} is calculated for all records of the part information, a predetermined number of records are extracted in descending order of similarity (that is, r _{camera_place} is in ascending order), and the corresponding part is specified.

Step S16 is a process of comparing the camera orientation vector included in the instruction information with the camera orientation vector of each part stored in the storage unit 22 and extracting a part having a high degree of similarity. Since the camera direction vector is the direction vector of the mobile terminal, the similarity can be evaluated by calculating the inner product of the vectors. For example, when the camera direction vector component included in the instruction information is (X _cam , Y _cam , Z _cam ), and the camera direction vector component included in the part information is (X _ref , Y _ref , Z _ref ) The similarity r _{lens_vector} can be expressed by Equation 5.
In step S16, r _{lens_vector} is calculated for all records of the part information,
A predetermined number of records are extracted in descending order of similarity (that is, r _{lens_vector} is in ascending order), and corresponding parts are specified.

  In step S17, images corresponding to the plurality of parts acquired in steps S15 and S16 are respectively acquired from the storage unit 22, and matching with the captured image acquired in step S14 is performed. The matching can be performed by any method such as pattern matching. As a result, the part corresponding to the image having the highest similarity is specified.

In step S <b> 18, an electronic manual corresponding to the part specified in step S <b> 17 is acquired from the storage unit 22 and transmitted to the mobile terminal 10 through the communication unit 21.
In step S19, the portable terminal 10 displays the received electronic manual. The received electronic manual may be displayed on the screen as it is, or may be displayed superimposed on the image obtained through the camera.

  As described above, in the first embodiment, in addition to the matching by the conventionally used image, the matching is performed by further using the displacement of the terminal from the reference position and the inclination of the terminal. Thereby, even in an environment in which it is difficult to specify a site only by an image, the site indicated by the user can be specified.

(Modification of the first embodiment)
In the first embodiment, the camera position information and the camera orientation vector are stored in the management server, and the part is estimated by collating with the instruction information transmitted from the mobile terminal. However, with this method, even when the same part is imaged, the similarity may change depending on the angle of the camera. For example, it is conceivable that the result differs between when the driver photographs the shift knob and when the passenger in the passenger seat photographs the shift knob. In order to prevent this, this modification is an embodiment in which the coordinates of the target part itself are stored in the management server and collation is performed using the coordinates. The system configuration of the mobile terminal 10 and the management server 20 according to this modification is the same as that of the first embodiment except for the points described below.

  In this modification, information about the position of the target part is stored not by the camera position and the camera orientation vector but by the coordinates of the part. FIG. 8 is an example of the part information according to this modification. Moreover, in this modification, the process of step S15-S16 replaces the process shown below.

The line of sight of the camera when the mobile terminal images the target part can be represented by a straight line. In this modification, the straight line is referred to as a camera line of sight.
First, the control unit 23 generates a mathematical expression representing the camera line of sight from the acquired camera position information and camera orientation vector. When the camera position information included in the instruction information is (x _cam , y _cam , z _cam ) and the component of the camera orientation vector is (X _cam , Y _cam , Z _cam ), the camera line of sight is x = x _cam + TX _cam , y = y _cam + tY _cam , z = z _cam + tZ _cam and parameters can be used.

Next, the control unit 23 calculates the shortest distance d _min between the coordinates of each part stored in the part information and the camera line of sight for each stored part. _Set the coordinates of the target part to (X _trgt , Y _trgt
, Z _trgt ), the vector from the coordinates toward the camera line of sight can be expressed as (x _cam + tX _cam −X _trgt , y _cam + tY _cam −Y _trgt , z _cam + tZ _cam −Z _trgt ). That is, the shortest distance d _min can be obtained by obtaining t at which the inner product of the vector and the camera orientation vector becomes 0. Since this method is publicly known, detailed mathematical formulas are omitted.

Then, the control unit 23 extracts a predetermined number of records in descending order of similarity (that is, in ascending order of _dmin ), and specifies a corresponding part. The processes after step S17 are the same as those in the first embodiment. The shortest distance d _min may be calculated using a method other than the exemplified method.

According to this modification, the similarity d _min increases as the camera line of sight is directed to the target part, so that the similarity can be prevented from changing depending on the camera angle at the time of shooting, and the target part can be determined more accurately. It can be performed.

(Second embodiment)
In the first embodiment, it is clearly indicated that the mobile terminal 10 is arranged at the reference position when the user presses the button. On the other hand, the second embodiment is a mode in which a marker representing a reference position is detected through a camera and the terminal itself is automatically recognized as being at the reference position. The system configuration diagrams of the mobile terminal 10 and the management server 20 according to the second embodiment are the same as those in FIGS. 2 and 3.

In the second embodiment, only the process of step S11 is different from the first embodiment.
FIG. 9 is a screen example of the mobile terminal 10 in step S11. In the second embodiment, the screen of FIG. 9 is displayed instead of the screen of FIG. A position guide of a marker (in this example, a logo mark at the center of the handle) is displayed on the screen, and the user moves the mobile terminal 10 to a position where the position guide and the logo mark match. If the control unit 15 determines that the positions match, the subsequent processing is continued. In other words, the position of the mobile terminal (FIG. 10) where the position guide and the logo mark match on the screen is the reference position. Whether or not the position guide matches the logo mark can be determined, for example, by storing an image of the logo mark in advance and performing pattern matching with the image acquired through the camera.

  In the first embodiment, a space for placing the mobile terminal is necessary to detect the reference position. However, as in this embodiment, the space can be secured by detecting the reference position through the camera. Even in such a case, the electronic manual system according to the present invention can be used.

(Modification)
The above embodiment is merely an example, and the present invention can be implemented with appropriate modifications within a range not departing from the gist thereof. For example, in the description of the embodiment, a system including a mobile terminal and a management server is illustrated, but a single device may have all functions.
Moreover, it may be implemented as an information providing system that provides information other than an electronic manual, or may be implemented as a simple object identification system.

In the description of the embodiment, it is assumed that there is one vehicle type. However, since the part information differs depending on the vehicle type, the part information may be stored separately for each vehicle type. In this case, information indicating the vehicle type may be included in the instruction information, and the determination may be made on the management server side.
Further, it is not always necessary to use the camera orientation vector. Even when only the camera position information is used, the object of the present invention to improve the identification accuracy of the target part can be achieved.

In the description of the embodiment, the inclination of the mobile terminal is acquired using the acceleration sensor, but the inclination may be acquired using an inclination sensor such as a gyro sensor. Since the gyro sensor is a sensor that acquires angular velocities, the acceleration is obtained by integrating the angular velocities acquired from the detection of the reference position in step S11 until the shooting is performed in step S12 for each of the X, Y, and Z axes. Similar to the case of using the sensor, the inclination of the mobile terminal can be acquired.

  In the description of the embodiment, the electronic manual corresponding to the site with the highest degree of similarity is transmitted. However, a plurality of candidates may be presented in order from the highest degree of similarity so that the user can select ( FIG. 11 (a)). In addition, when there are a plurality of electronic manuals corresponding to the same part, the user may be allowed to select (FIG. 11B).

DESCRIPTION OF SYMBOLS 10 Portable terminal 11,21 Communication part 12 Camera 13 Acceleration sensor 14 Input / output part 15,23 Control part 22 Storage part

Claims (7)

It consists of a mobile terminal equipped with a camera and a management server.
Among a plurality of objects arranged substantially fixed relative to each other, an object identification system for identifying an object imaged by a camera,
The portable terminal is
The terminal itself, and the reference position determining means for determining that it is in a single reference position set in advance on the plurality of objects,
Displacement detecting means for detecting displacement of the terminal from the reference position;
Have
The management server
An object storage means for storing arrangement information which is information representing a positional relationship of the object with respect to the reference position ;
Object identification means for identifying an object based on the displacement detected by the displacement detection means, the arrangement information, and an image captured by the camera;
An object identification system. The mobile terminal further includes a triaxial acceleration sensor,
The displacement detection means detects a displacement from the reference position based on the acceleration detected by the acceleration sensor;
The object identification system according to claim 1. The reference position determination means further includes an input acquisition means for acquiring an input operation from a user,
Based on the input operation, the portable terminal is determined to be in the reference position,
The object identification system according to claim 1 or 2. The reference position determination unit, when a marker indicating a reference position is captured by the camera, the mobile terminal is determined to be in the reference position,
The object identification system according to claim 1 or 2. The mobile terminal further includes a tilt sensor that detects the tilt of the terminal itself,
The object identifying means identifies the object using the inclination detected by the inclination sensor;
The object identification system according to any one of claims 1 to 4. It consists of a mobile terminal equipped with a camera and a management server.
An information providing system that provides information about an object imaged by a camera among a plurality of objects arranged substantially fixed relative to each other,
The portable terminal is
The terminal itself, and the reference position determining means for determining that it is in a single reference position set in advance on the plurality of objects,
Displacement detecting means for detecting displacement of the terminal from the reference position;
Have
The management server
An object storage means for storing arrangement information which is information representing a positional relationship of the object with respect to the reference position ;
Object information storage means for storing information about the object for each object;
Object identification means for identifying an object based on the displacement detected by the displacement detection means, the arrangement information, and an image captured by the camera;
Information providing means for acquiring information about the identified object and transmitting the information to the mobile terminal;
An information providing system. It consists of a mobile terminal equipped with a camera and a management server.
An object identification method performed by an object identification system for identifying an object imaged by a camera among a plurality of objects arranged relatively fixed to each other,
The mobile terminal is
The terminal itself, and the reference position determination step of determining that it is in a single reference position set in advance on the plurality of objects,
A displacement detecting step of detecting a displacement of the terminal from the reference position;
Run
The management server is
An object storage step for storing arrangement information which is information representing a positional relationship of the object with respect to the reference position ;
An object identification step for identifying an object based on the displacement detected in the displacement detection step , the arrangement information, and an image captured by the camera;
An object identification method is executed.

Images