JP4844150B2 - Information processing apparatus, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing apparatus, an information processing method, and an information processing program capable of searching for desired stored information based on information for search, and in particular, searching for information representing relevance of objects in time and space. The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

  Metadata is data that describes a description of data. Metadata is created to help find the target data from a huge pile of data. The followings have been proposed as conventional techniques for searching for moving image information using this metadata.

  The moving image processing apparatus described in Patent Document 1 automatically outputs an output moving image corresponding to a moving image feature amount or a usage method by cutting out an appropriate region for each frame of the input moving image using accompanying metadata. To create.

  The moving image management apparatus described in Patent Document 2 makes it possible to easily optimize and utilize metadata attached to a moving image even after editing the moving image.

  The moving image search apparatus described in Patent Document 3 extracts an object region from an input video, extracts a shape feature that changes including a change in the shape of a continuous frame of the object region, and stores it in a metadata database in advance. The feature feature metadata specified for the search is compared with the metadata stored in advance in the metadata database, and an image having a similarity is displayed.

  The image processing apparatus described in Patent Document 4 captures a person's face with a photosensor, extracts identification information that is information for identifying the person from image data obtained by imaging, and based on the extracted identification information. The name of the person specified in this step is assigned as image data metadata.

  The image capturing device described in Patent Document 5 captures the iris of a person who captures an image with a photosensor, and assigns data such as a digital watermark created based on the iris to the image captured by the photographer as metadata.

JP2004-172671 JP 2003-259268 A JP 2001-268479 A JP2004-062868 JP2004-023733

  By the way, it is easy to specify an individual based on a still image and add metadata such as name and digital watermark, but it is difficult to extract metadata for moving images. For example, it is difficult to automatically and simultaneously add metadata to a moving image group at the same time as recording of a person or an object to be photographed in the moving image. For this reason, there is a problem that such a moving image cannot be searched using metadata. Depending on the above-mentioned patent documents, metadata cannot be automatically assigned to such a moving image.

  Furthermore, since the metadata is created to help find the target data from the enormous amount of data, it is necessary to specifically represent the data to be explained. Among the specific expressions, especially for metadata describing temporal or spatial relations called directionality, the problem is that it is more difficult to extract metadata because it requires enormous calculations. Exists.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an information processing apparatus, an information processing method, and an information processing program capable of determining explanation information with a small amount of calculation based on sensor information. I will provide a.

An information processing apparatus according to the present invention includes an input unit that inputs sensor information output by one or more sensors that detect, as sensor information, a target object that is at least one of a person, a tangible object, and an intangible object; A determination means for determining explanation information for explaining an object based on input sensor information, and a logical expression representing at least one of a specific positional relationship and a temporal relationship between the objects, the same contents as the logical expression A decision to determine a lightweight logical expression, which is a logical expression with a small amount of calculation, based on the amount of calculation required by the determination means for each operator constituting the modified logical expression. An information processing apparatus comprising: determining means for determining the description information based on the light-weight logical expression determined by the determining means and the sensor information .
According to this configuration, for example , compared with the case where the description information “adjacent” is determined only from the position information, the content “faced” has a directional meaning and is more specific. Typical explanation information can be determined. In addition, according to this configuration, the description information can be determined based on the specific positional relationship of the target object. Therefore, as an example, the description information indicating more specific content having a positional relation meaning is determined. can do. Furthermore, according to this configuration, since it is possible to determine the explanation information after determining a logical expression with a small amount of calculation required for determination, as an example, compared with a case where determination is performed without determining a logical expression with a small amount of calculation. An increase in determination time associated with an increase in the number of target sensor information can be suppressed.

In the above configuration, the determination unit can adopt a configuration in which a light-weight logical expression is determined by transforming a logical expression by a combination of a mathematical law, a distribution rule and a combination rule, and logical negation.
According to this configuration, since the logical expression can be transformed by a logical operation, as an example, a modification rule that transforms into a logical expression with a small amount of calculation is registered in the information processing apparatus in advance, and the modification rule is applied. Compared with the method for determining whether or not it is possible, it is possible to reduce the labor of the administrator resulting from the addition or change of the deformation rule.

In the above-described configuration, the determining unit may employ a configuration in which a light-weight logical expression is determined using a logical rule that is a modification rule determined by Allen's temporal relevance and that reduces a calculation amount.
According to this configuration, in order to determine whether or not to apply the deformation rule by registering in the information processing apparatus in advance various deformation rules that are deformed into different logical expressions with the same logical contents in advance, for example, Compared with the method of transforming logical expressions according to mathematical rules such as distribution rules, combination rules, and exchange rules, which are mathematical properties, not only can the transformation rules be more flexible, but also the expansion and modification of the transformation rules. Is easy.

In the above configuration, the determination means may be a modification rule determined by Egenhofer's spatial relevance, and a light-weight logical expression may be determined using a logical expression modification rule that reduces the amount of calculation.
According to this configuration, in order to determine whether or not to apply the deformation rule by registering in the information processing apparatus in advance various deformation rules that are deformed into different logical expressions with the same logical contents in advance, for example, Compared with the method of transforming logical expressions according to mathematical rules such as distribution rules, combination rules, and exchange rules, which are mathematical properties, not only can the transformation rules be more flexible, but also the expansion and modification of the transformation rules. Is easy.

In the above configuration, the deformation rule determined by Allen's temporal relevance is a deformation rule that deforms from the left side to the right side while satisfying the following logical expression (1). .
(Equation 3)
equal (Ai, Bi) or after (Ai, Bi) or duration (Ai, Bi) or containers (Ai, Bi) or overlays (Ai, Bi)
or overwrapped_by (Ai, Bi) or meet (Ai, Bi) or met_by (Ai, Bi) or start (Ai, Bi) or finishes (Ai, Bi) =
not ( before (Ai, Bi) ) (1)
However, Ai and Bi represent time intervals.
According to this configuration, in order to determine whether or not to apply the deformation rule by registering in the information processing apparatus in advance various deformation rules that are deformed into different logical expressions with the same logical contents in advance, for example, Compared with the method of transforming logical expressions according to mathematical rules such as distribution rules, combination rules, and exchange rules, which are mathematical properties, not only can the transformation rules be more flexible, but also the expansion and modification of the transformation rules. Is easy.

In the above configuration, a configuration characterized in that the deformation rule determined by Egenhofer's spatial relationship is a deformation rule that transforms from the left side to the right side satisfying the following logical expression (2): .
(Equation 4)
(Ar Contain Br) or (Ar Inside Br) or (Ar Equal Br) or (Ar Meet Br) or (Ar Cover Br) or (Ar CoveredBy Br) or (Ar Overlap Br) = not Ar・ (2)
However, Ar and Br represent regions.
According to this configuration, in order to determine whether or not to apply the deformation rule by registering in the information processing apparatus in advance various deformation rules that are deformed into different logical expressions with the same logical contents in advance, for example, Compared with the method of transforming logical expressions according to mathematical rules such as distribution rules, combination rules, and exchange rules, which are mathematical properties, not only can the transformation rules be more flexible, but also the expansion and modification of the transformation rules. Is easy.

The said structure WHEREIN: The structure which further has the sensor information storage means which associates and accumulate | stores the description information determined by the determination means, sensor information, and time information is employable.
According to this configuration, the object is detected as sensor information, and the description information is determined based on the direction information. Therefore, in two or more objects , the spatial between each object and the object is determined. Associations and temporal relationships can be accumulated in association with each other.

In the above-described configuration, the object may have a priority order, and may further include a consistency maintaining unit that maintains the consistency of the contents of the plurality of explanation information stored in the sensor information storage unit based on the priority order.
According to this configuration, it is not necessary to make a precise determination based on a logical expression, so that the configuration of the determination unit can be simplified. In addition, it is possible to flexibly cope with addition or change of explanation information . Further, according to this configuration, since the description information is determined based on a predetermined priority order, for example, compared with the method of determining the description information based on the user's selection, the information is stored in the sensor information storage unit. It is possible to suppress an increase in user management labor accompanying an increase in the number of explanation information .

In the above configuration, it is possible to employ a configuration that further includes an alignment maintaining unit that maintains the alignment of the contents of the plurality of pieces of description information stored in the sensor information storage unit based on the user's selection.
According to this configuration, it is not necessary to make a precise determination based on a logical expression, so that the configuration of the determination unit can be simplified. In addition, it is possible to flexibly cope with addition or change of explanation information . Further, according to this configuration, description information for determining the description information based on the selection of the user, for example, as compared with the method for determining the description information based on the priority predetermined as determined by the determining means And the explanation information recognized by the user by actual observation is less likely to occur.

In the above configuration, the input unit further includes a moving image storage unit that inputs sensor information output by a sensor that detects an object at the time of shooting a moving image as sensor information and stores the moving image and time information in association with each other. The structure which has is employable.
According to this configuration, sensor information can be stored in association with time, and a moving image can be stored in association with time information. Therefore, as an example, by storing in association with time information indicating the shooting time of a moving image, it is possible to specify the shooting time of the moving image and store the moving image and sensor information in association with each other.

The said structure WHEREIN: The structure further provided with the search means to search a moving image based on the description information which the determination means determined can be employ | adopted.
According to this configuration, it searches the moving image based on the description information can be retrieved as an example, a moving image of the object was photographed object based on the description information representing features of the object. As an example, in order to specify a moving image in which the action of an object characterized by “discussion” is photographed, the moving image can be specified using the explanation information “discussion”.

In the above configuration, the search unit can employ a configuration for searching for an object based on the explanation information determined by the determination unit.
According to this configuration, since the object can be searched based on the description information , the object can be specified using the description information that characterizes the object as a clue. As an example, when the “discussion” characterizes the action of the person who is the object in the meeting, the person can be identified using the explanation information “discussion” as a clue.

In the above configuration, the search unit can employ a configuration for searching the explanation information determined by the determination unit based on the object .
According to this configuration, since the description information can be searched based on the target object , it is possible to acquire the description information indicating the state or operation of the target object at a specific time using the target object as a clue. As an example, the projection history of an electronic file that is explanatory information can be acquired using an electronic file that is an object as a clue.

An information processing method according to the present invention includes an input step of inputting sensor information output by one or more sensors that detect, as sensor information, an object to be described that is at least one of a person, a tangible object, and an intangible object; A determination step for determining explanation information for explaining an object based on sensor information input in the input step, and a logical expression representing at least one of a specific positional relationship and temporal relationship between the objects are the same as the logical expression A lightweight logical expression that is a logical expression with a small amount of calculation is determined based on the amount of calculation required for the determination performed for each operator constituting the modified logical expression. a determination step of, an information processing method comprising a determination step, especially that determines description information on the basis of the determined weight logical expression and the sensor information at decision step It is set to.
According to this configuration, for example , compared with the case where the description information “adjacent” is determined only from the position information, the content “faced” has a directional meaning and is more specific. Typical explanation information can be determined. In addition, according to this configuration, the description information can be determined based on the specific positional relationship of the target object. Therefore, as an example, the description information indicating more specific content having a positional relation meaning is determined. can do. Furthermore, according to this configuration, since it is possible to determine the explanation information after determining a logical expression with a small amount of calculation required for determination, as an example, compared with a case where determination is performed without determining a logical expression with a small amount of calculation. An increase in determination time associated with an increase in the number of target sensor information can be suppressed.

An information processing program according to the present invention includes a computer, an input unit that inputs sensor information output by one or more sensors that detect, as sensor information, a target object that is at least one of a person, a tangible object, and an intangible object; A determination unit that determines explanatory information that explains an object based on sensor information input by an input unit, and a logical expression that represents at least one of a specific positional relationship and a temporal relationship between the objects, A lightweight logical expression that is a logical expression with a small amount of calculation based on the amount of calculation required for determination performed by the determination means for each operator that is transformed into other logical expressions that represent the same content and that constitutes the deformed logical expression an information processing program to function as a determining means for determining, determining means, and wherein the determining the description information on the basis of the determined weight logical expression and the sensor information of the determining means To have.
According to this configuration, for example , compared with the case where the description information “adjacent” is determined only from the position information, the content “faced” has a directional meaning and is more specific. Typical explanation information can be determined. In addition, according to this configuration, the description information can be determined based on the specific positional relationship of the target object. Therefore, as an example, the description information indicating more specific content having a positional relation meaning is determined. can do. Furthermore, according to this configuration, since it is possible to determine the explanation information after determining a logical expression with a small amount of calculation required for determination, as an example, compared with a case where determination is performed without determining a logical expression with a small amount of calculation. An increase in determination time associated with an increase in the number of target sensor information can be suppressed.

According to the present invention, it is possible to provide an information processing apparatus, an information processing method, and an information processing program capable of determining explanation information with a small amount of calculation based on sensor information.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments of the invention will be described with reference to the accompanying drawings.
FIG. 1 is a configuration diagram of an information processing apparatus according to an embodiment of the present invention.
The information processing apparatus 100 of the present invention includes a camera management unit 110, a moving image recording unit 120, a moving image database 130, a sensor management unit 140 as an input unit, a determination unit 150 as a determination unit, a determination unit 160 as a determination unit, Deformation rule database 170, metadata database 180, sensor information recording unit 190, sensor information database 200, consistency maintaining unit 210 as a consistency maintaining unit, environment information database 220, search unit 230 as a search unit, input unit 240, and display It consists of a part 250 and the like.

  In the information processing apparatus 100, a search condition is input by the user 300. In addition, the information processing apparatus 100 displays the search result, metadata to be selected, and a message that prompts the user 300 to select the metadata.

  The camera management unit 110 manages video cameras. The camera management unit 110 is connected to the moving image recording unit 120. The camera management unit 110 acquires a moving image and transmits the acquired moving image to the moving image recording unit 120.

Here, a configuration example of the camera management unit 110 will be described with reference to FIG. FIG. 2 is a diagram for explaining a configuration of the camera management unit 110.
The camera management unit 110 manages one or a plurality (n in FIG. 2) of video cameras 111 to 11n.

  The camera management unit 110 acquires moving images from the video cameras 111 to 11n using the video camera. The camera management unit 110 manages camera identification information, which is information for identifying the video cameras 111 to 11n, and transmits the camera identification information and the acquired moving image in association with each other to the moving image recording unit 120.

  The video cameras 111 to 11n are composed of, for example, a CCD camera or the like. The video cameras 111 to 11n are connected to the moving image recording unit 120 through the camera management unit 110. In this embodiment, the video cameras 111 to 11n are fixedly arranged so as to image a predetermined area. The video cameras 111 to 11n transmit a moving image obtained by capturing a predetermined area to the moving image recording unit 120 through the camera management unit 110.

The moving image recording unit 120 is connected to the camera management unit 110 and the moving image database 130. Moving image recording unit 120 receives the moving image and the camera identification information associated with the camera management unit 110. In addition, the moving image recording unit 120 associates the time, the received moving image, and the camera identification information and stores them in the moving image database 130.

Here, returning to FIG. 1, the description of the configuration of the information processing apparatus of the present invention will be continued.
The moving image database 130 is connected to the moving image recording unit 120 and the search unit 230. The moving image database 130 stores time information, moving images, and camera identification information in association with each other by the moving image recording unit 120. The moving image database 130 is referred to by the search unit 230 for accumulated time information, moving images, and camera identification information.

  According to this configuration, sensor information can be stored in association with time, and a moving image can be stored in association with time information. Therefore, as an example, by storing in association with time information indicating the shooting time of a moving image, it is possible to specify the shooting time of the moving image and store the moving image and sensor information in association with each other.

  The sensor management unit 140 manages various sensors. The sensor management unit 140 is connected to the determination unit 150. The sensor management unit 140 acquires an object that is at least one of a person, a tangible object, and an intangible object as sensor information, and transmits the acquired sensor information to the determination unit 150.

Here, a configuration example of the sensor management unit 140 will be described with reference to FIG. FIG. 3 is a diagram for explaining a configuration of the sensor management unit 140.
The sensor management unit 140 includes one or a plurality of file servers, a mail server, a projector, an RFID reader, a microphone, and the like. The sensor management unit 140 acquires, as sensor information, an object that is at least one of a person, a tangible object, and an intangible object using a file server, a mail server, a projector, an RFID reader, a microphone, and the like. In addition, the sensor management unit 140 transmits sensor information to the determination unit 150.

  The file servers 1401 to 140m are composed of, for example, a personal computer. The file servers 1401 to 140m store and manage, for example, electronic files representing the contents of slides displayed at the time of presentation. In particular, the file servers 1401 to 140m function as sensors that detect electronic files that are intangible objects as sensor information. In this embodiment, the object identification information for identifying the electronic file and the modification content of the electronic file are detected as sensor information.

  In particular, when an electronic file is modified, a configuration in which the modified file is detected as a separate object from the unmodified file, and the modification content can be detected as a single object.

  The file servers 1401 to 140m are referred to by the determination unit 150 for electronic files or the like representing the contents of the stored slides. In the file servers 1401 to 140m, the determination unit 150 stores whether or not each slide represented by the referenced electronic file includes a technical term in association with the object identification information.

  The mail servers 1411 to 141m ′ are composed of, for example, a personal computer. The mail servers 1411 to 141m ′ transmit / receive an electronic file attached to an e-mail, for example, according to a protocol such as SMTP or POP.

  In particular, the mail servers 1411 to 141m ″ function as sensors that detect electronic files, which are intangible objects, as sensor information. In this embodiment, object identification information for identifying an electronic file and a transmission destination such as an electronic file are detected as sensor information.

  The projectors 1421 to 142m ″ are configured by, for example, a liquid crystal projector. The liquid crystal projectors 1421 to 142m ″ project the slide represented by the input electronic file or the like onto the screen. In particular, the projectors 1421 to 142m ″ function as sensors that detect projected slides, which are intangible objects, as sensor information. In this embodiment, switching of projected slides and object identification information that is information for identifying the projected slides are detected as sensor information.

  The RFID readers 1431 to 143m ′ ″ are sensors that detect position information of the RFID tag. The RFID readers 1431 to 143m ′ ″ are installed so as to communicate with the RFID tag. In particular, in the present embodiment, it functions as a sensor that detects, as sensor information, desks, chairs, screens, projectors, and the like, which are tangible objects and objects, and persons and attendees of the meeting that are objects. .

  Specifically, a desk, a chair, a screen, a projector, and a person each have an RFID tag. The RFID reader detects three-dimensional position information of the RFID tag and information for identifying the RFID tag emitted from the RFID tag. In this embodiment, information for identifying an RFID tag is regarded as information for identifying a tangible object or person having the RFID tag.

  The microphones 1441 to 144m ′ ″ are composed of, for example, a piezoelectric microphone. The microphones 1441 to 144m ′ ″ are carried by a person and are arranged and carried so that the voice of the person can be collected. In particular, the microphones 1441 to 144m ′ ″ function as sensors that detect a person as an object as sensor information. Specifically, the microphones 1441 to 144m detect information that identifies a person's voice and the collected microphone. In this embodiment, information for identifying a microphone is regarded as information for identifying a person having the microphone.

  The determination unit 150 is connected to the sensor management unit 140, the determination unit 160, the metadata database 180, the sensor information recording unit 190, and the environment information database 220. The determination unit 150 receives sensor information and object identification information from the sensor management unit 140. The determination unit 150 receives environment information from the environment information database 220. Note that what kind of information is referred to as environment information will be described later.

  Next, the determination unit 150 creates a logical expression for determining metadata. Thereafter, the metadata is determined based on the direction information and the logical expression determined based on the received sensor information and, if necessary, the environment information. Finally, the determined metadata and the received sensor information are transmitted to the sensor information recording unit 190.

  Specifically, in this embodiment, intangible and object electronic data used in the meeting, words added to the electronic data, tangible and object desk used in the meeting that is the object, chair, A meeting attendee who is a person and an object such as a projector and a screen is detected based on object identification information that is sensor information received from the sensor management unit 140. Thereafter, a logical expression for determining appropriate metadata is created in consideration of all temporal and spatial relationships between detected objects.

Here, a method of considering the temporal relationship between objects will be described with reference to FIG. FIG. 4 is a diagram for explaining Allen's time interval model (Allen, JF: “Maintaining Knowledge about Temporal Intelligence,” Communications of the ACM, No. 26, pp. 832-843 (1983)). .
The temporal relationship between the time point Ap and the time point Bp, the time point Ap and the time interval Bi, the time interval Ai and the time point Bp, and the time interval Ai and the time interval Bi is 3, 5, 5, and It has been clarified by Allen that it is classified into 11 associations. The present invention analyzes the temporal relationship between objects using Allen's time interval model.

  The temporal relationship between the time point Ap and the time point Bp is all represented by three, equal, before, and after.

equal (Ap, Bp) indicates that the time point Ap is coincident with the time point Bp, before (Ap, Bp) indicates that the time point Ap is before the time point Bp, and after (Ap, Bp) indicates that the time point Ap is It represents that it is after the time point Bp.

  The temporal relationship between the time point Ap and the time interval Bi is all represented by before, after, contains, starts, and finishes.

before (Ap, Bi) indicates that the time Ap is the start time Bi. Indicates that it is before stp . after (Ap, Bi) indicates that the time point Ap is the end time Bi. Indicates that it is later than enp . contains (Ap, Bi) is the start time Bi. stp is before the time point Ap, and the end time Bi. It represents that enp is later than the time point Ap . starts (Ap, Bi) is the start time Bi. stp represents that matches the time Ap. finishes (Ap, Bi) is the end time Bi. enp represents that it matches the time Ap.

  The temporal relationship between the time interval Ai and the time point Bp is all expressed by before, after, during, starts, and finishes.

before (Ai, Bp) is the end time Ai. It represents that enp is before time point Bp . after (Ai, Bp) is the start time Ai. It represents that stp is after time point Bp . duration (Ai, Bp) is the start time Ai. stp is before the time point Bp, and the time point Bp is the end time Ai. Indicates that it is before emp . starts (Ai, Bp) is a time point Bp when the start time Ai. Indicates that it matches stp . finishes (Ai, Bp) is the time Bp is the end time Ai. It represents matching with emp .

  The temporal relationship between the time interval Ai and the time interval Bi is all expressed by equal, before, after, during, contents, overlaps, overlapped_by, meets, met_by, starts, and finishes.

equal (Ai, Bi) is the start time Ai. stp and the start time Bi. stp and the end time Ai. of the section Ai. enp and the end time Bi. It represents that emp matches . before (Ai, Bi) is the end time Ai. enp is the start time Bi. Indicates that it is before stp . after (Ai, Bi) is the end time Bi. emp is the start time Ai. Indicates that it is before stp . duration (Ai, Bi) is the start time Ai. stp is the start time Bi. before the time point stp and the end time Bi. emp is the end time Ai. Indicates that it is before stp . contains (Ai, Bi) is the start time Bi. start time start time of the section Ai of stp Ai. before end time and end time Ai. of section Ai. enp is the end time Bi. Indicates that it is before emp . overlaps (Ai, Bi) is the start time Ai. stp is the start time Bi. stp and before the start time Bi. of the section Bi. stp is the end time Ai. enp and before the end time Ai. enp is the end time Bi. Indicates that it is before emp . overlapped_by (Ai, Bi) is the start time Bi. stp is the start time Ai. stp and before the start time Ai. stp is the end time Bi. enp and before the end time Bi. emp is the end time Ai. Indicates that it is before emp . Meets (Ai, Bi) is the end time Ai. enp is the start time Bi. Indicates that it matches stp . met_by (Ai, Bi) is the end time Bi. emp is the start time Ai. Indicates that it matches stp . starts (Ai, Bi) is the start time Ai. stp is the start time Bi. Indicates that it matches stp . finishes (Ai, Bi) is the end time Ai. enp is the end time Bi. It represents matching with emp.

Next, how to consider the spatial relationship between objects will be described with reference to FIG. Fig. 5 shows Egenhofer's spatial relevance model (Egenhofer, MJ: “Spatial Relations: Models, Inferences, and the Future Futurization Applications”, Proceedings of Advanced Database Symposium, Tokyo, Japan, Decumer 2-4, sepa Egenhofer, MJ, Rashid, A. and Shari, B.M .: “Metric Detais for Natural-Language Spatial Relations”, ACM Transactions on Information Systems, Vol. 16, No. 4, pp. 295-321 (1998) .) Is a diagram for explaining.
The spatial relationship between the region Ar and the region Br, the region Ar and the line segment Bl, the line segment Al and the line segment Bl, the region Ar and the point Bp, the line segment Al and the point Bp, and the point Ap and the point Bp is shown in FIG. It is revealed by Egenhofer that it is classified into 8, 10, 11, 3, 3, and 2 patterns shown in FIG. The present invention analyzes the spatial relationship between objects using Egenhofer's spatial relationship model.

  The spatial relationship between the area Ar and the area Br is expressed by Disjoint, Contain, Inside, Equal, Meet, Cover, Covered_By, and Overlap.

Disjoint (Ar, Br) represents that the region Ar and the region Br have no relationship . Contain (Ar, Br) indicates that the region Ar includes the region Br . Inside (Ar, Br) indicates that the region Ar is in the region Br . Equal (Ar, Br) represents that the region Ar and the region Br coincide with each other . Meet (Ar, Br) represents that the region Ar and the region Br are in contact with each other . Cover (Ar, Br) represents that the region Ar includes the region Br . Covered_By (Ar, Br) represents that the area Ar is included in the area Br . Overlap (Ar, Br) may be tables that overlap each other and the region Ar and the region Br.

Here, an example of metadata determined by the determination unit 150 will be described with reference to FIG. FIG. 6 is a diagram for explaining an example of the definition of metadata determined by the determination unit.
The metadata illustrated in FIG. 6 is an example of metadata determined by the determination unit 150, and is determined based on the arrangement direction among the spatial relationships between objects.

  The “facing each other” in the metadata is a metadata that is determined by the determination unit 150 when the objects A and B, which are persons, are in spatial contact with each other and the arrangement directions (viewing directions) are opposite to each other. It is data.

  Further, in the metadata “sit side by side”, the persons A and B are in spatial contact with each other, the arrangement directions (viewing directions) are in the same direction, and the persons A and B and the chair C are in spatial contact. The metadata is determined by the determination unit 150.

In the metadata “write to each other on the whiteboard”, the person A and the pen P overlap and spatially overlap, the person B and the pen P overlap and spatially overlap, and the person A and the pen Pa , The person B, the pen Pb, and the whiteboard H are in spatial contact, the arrangement directions (viewing directions) of the persons A and B are the same, and the arrangement directions of the persons A and B are the whiteboard. This metadata is determined by the determination unit 150 in the case where the arrangement direction is opposite.

  In addition, in the metadata “facing to the desk side by side”, the persons A and B are spatially in contact, the arrangement directions (viewing directions) are the same direction, and the persons A and B and the desk T are spatial. Metadata that is determined by the determination unit 150 when it is in contact with.

  Further, in the metadata “sitting facing each other”, the persons A and B are spatially in contact, the arrangement directions (viewing directions) are opposite to each other, and the persons A and B and the chair C are spatially separated. The metadata is determined by the determination unit 150 when in contact.

  Further, in the metadata “sitting at a position where it can be written on the whiteboard, facing each other”, the persons A and B are spatially in contact with each other, and the arrangement directions (viewing directions) are opposite to each other. This is metadata that is determined by the determination unit 150 when B and the chair C are in spatial contact with each other, and when the persons A and B, the chair C, and the whiteboard H are in spatial contact with each other.

Next, an example of metadata that is determined by the determination unit 150 and that is determined based on the moving direction will be described.
Among the spatial relationships between objects, “discussion”, which is metadata determined based on the direction of movement, is performed after objects A and B, which are persons, approach a screen displaying technical terms. The metadata is determined by the determination unit 150 when the screen is standing and facing the screen.

  As long as it is “discussion”, it is also possible to adopt a definition added as a requirement that either one of the objects A or B as a person speaks. The presence or absence of a statement at a specific time is determined by the determination unit 150 based on the sound pressure of the sound detected by the microphones 1441 to 143m ′ ″ of the sensor management unit 140.

  Also, a method for determining whether or not the screen displays technical terms at a specific time will be described. The determination unit 150 identifies the display slide displayed on the screen at a specific time based on the information for identifying the display slide detected by the projectors 1421 to 142m ″ of the sensor management unit 140 and the switching of the display slide. . Next, the determination unit 150 determines whether the identified display slide includes technical terms in the content.

  The determination unit 150 analyzes slide data stored in the file servers 1401 to 140m of the sensor management unit 140 in advance, and stores in the file server whether or not technical terms are included, in association with the display slide. Thereafter, it is acquired whether or not the specified display slide has a technical term from the file server 1401 based on the identification information of the specified display slide.

Here, an example of a method for determining whether or not electronic data representing a slide includes technical terms will be described with reference to FIG. FIG. 7 is a diagram for explaining an example of a slide data analysis method.
For example, taking slide data DS created by Microsoft's PowerPoint presentation software as an example, the determination unit 150 receives slide data stored in the file servers 1401 to 140m of the sensor management unit 140. To do.

  Thereafter, the determination unit 150 converts the slide data from a file format used by PowerPoint to a file format of a structured document such as XML in order to facilitate analysis.

  Although not shown, the determination unit 150 includes a technical term accumulation unit 151 that accumulates technical terms. The technical term accumulating unit is composed of, for example, a relational database. The determination unit 150 refers to the technical terms stored in the technical term storage unit 151 and determines whether or not the referred technical terms are included in the content indicated by the file converted into the file format of the structured document.

Next, an example of metadata that is determined by the determination unit 150 and that is determined based on the moving direction will be described.
“Transmission” that is metadata is metadata that is determined by the determination unit 150 when electronic data that is an intangible object is attached to an email and transmitted by the mail servers 1411 to 141m ′.

  Similarly, “reception” that is metadata is metadata that is determined by the determination unit 150 when electronic data that is an intangible object is attached to an e-mail and received by the mail servers 1411 to 141m ′. It is.

  Further, as metadata including directionality in the semantic content, metadata such as “projection”, “pointing”, or “editing” is projected onto a screen or the like when electronic data is projected onto a projection screen or the like. It is also possible to adopt a configuration that is defined as when the person points to the electronic data or when the electronic data is edited.

Furthermore, as metadata that includes directionality as semantic content, metadata such as “reading” is defined as the case where the document layout direction printed on a tangible object paper medium or the like is the same as the human object viewing direction. It is also possible to adopt a configuration that does this. Note that the direction of the paper medium can be acquired based on sensor information using an RFID tag in the same manner as other tangible objects.

  Further, referring now to FIG. 8, new operators “reverse”, “add”, “multi”, “difference”, for describing both relevance between temporal relevance and spatial relevance, And “division”. FIG. 8 is a diagram for explaining a new operator.

  In addition, the data structure used to define the new operators “reverse”, “add”, “multi”, “difference”, and “division” will be described.

  The data structure includes time information such as times t0, t1, t2,..., And position information (x1i, y1i, z1i) based on three-dimensional coordinates at a certain time ti-1 (where i = 1, 2,...). ), Position information (x2i, y2i, z2i) based on three-dimensional coordinates at time ti, and an information ID for identifying the object are represented in parallel for each time t0, t1, t2,. It has a data structure sq.

  This makes it possible to easily and universally describe a description for each object sequence having time, space, and direction, which has been difficult or impossible in the past.

Next, a new operator “reverse” will be described.
“Reverse” expresses the parallel order of the data sets constituting the data structure sq in reverse order.

  Specifically, sq52 can be given as an example in which the operation sequence for raising the pen is described by the data structure sq.

  The operation sequence for lowering the pen can be sq53 as an example described by the data structure sq.

  The sq 53 representing the operation sequence of lowering corresponds to the data sequence of sq 52 describing the operation sequence of raising the pen by the data structure sq in parallel in the reverse order.

  Therefore, sq53 describing the operation sequence of lowering the pen with the data structure sq can be described as sq53: (reverse (seq52)) using sq52 describing the operation sequence of raising the pen with the data structure sq.

  Further, sq101 is a data structure sq that represents a motion sequence in which a certain person object A walks toward the whiteboard H when the whiteboard H exists in the three-dimensional space coordinates (2.0, 0.0, 0.0). It is described by.

  Further, sq102 describes a motion sequence in which a certain person object B walks toward the whiteboard H using a data structure sq.

  Next, the operators “add”, “multi”, “difference”, and “division” will be described.

The operator “add” is a symbol representing an operation for performing a sum of a plurality of sequences.
The operator “multi” is a symbol representing an operation for extracting a common part of a plurality of sequences.

The operator “difference” is a symbol representing an operation for extracting a sequence obtained by excluding a sequence described after the operator from a sequence described before the operator.
The operator “division” is a symbol representing an operation for extracting a sequence corresponding to a sequence described after the operator from a sequence described before the operator. Note that the motion sequence extracted by the operators “difference” and “division” does not have an object as the motion subject.

  Specifically, by using each operator, an operation for the following sequence can be described using the data sq101 and sq102 already described.

seq101 add seq101 can describe that A or B is walking toward the whiteboard H.
Moreover, it can be described that A and B are walking toward the whiteboard H by seq101 multi seq101.

Further, seq101 difference seq101 can describe that the conference participant is walking along a trajectory obtained by excluding the B trajectory from the A trajectory.
seq101 division With seq101, it can be described that the participant of the conference is walking from the trajectory of A to the trajectory corresponding to the trajectory of B.

Although not shown in FIG. 3, the sensor management unit 140 includes a sensor that detects a position indicated by an instruction device used by a presenter or the like to indicate a screen. Therefore, the determination unit 150, the sensor information received from the sensor management unit 140 can detect that an instruction to electronic data presenter like are projected and the indicated screen or the like.

Here, a method of acquiring the moving direction of the object based on the sensor information will be described with reference to FIG. FIG. 9 is a diagram for explaining an example of an object moving direction acquisition method.
In FIG. 9, the determination unit 150 acquires the position coordinates of the person object having an RFID tag, the screen that is a tangible object, and the projector in the XYZ space from the RFID readers 1431 to 143 m ′ ″ of the sensor management unit 140.

Thereafter, the determination unit 150 acquires a method for moving the object using the position coordinates of the RFID tag at a plurality of times. Further, it is determined whether the person object is sitting or standing by determining movement in the vertical direction (Z-axis direction in FIG. 9 ) or the like.

  In addition, according to the above configuration, metadata can be determined based on complex information called direction information. Therefore, as an example, compared to the case of determining “adjacent” metadata from position information alone. Thus, it is possible to determine more specific metadata having a directional meaning to the content “faced”.

  According to this configuration, since the metadata can be determined based on a specific positional relationship of the object, as an example, it is possible to determine metadata indicating more specific content having a positional relationship meaning to the content. it can.

  According to this configuration, since the metadata can be determined based on the moving direction of the object, as an example, it is said that “received” compared to the case where the metadata “matched” is determined only from the position information. It is possible to determine metadata indicating more specific operation contents having a moving direction meaning in the contents.

Next, a method for acquiring the object arrangement direction (viewing direction) will be described with reference to FIG. FIG. 10 is a diagram for explaining an example of a method for obtaining the viewing direction of an object.
In FIG. 10, a tangible object and a person each have a set of RFID tags. The set of RFID tags is fixed so that both XY coordinates do not match. Each set of RFID tags is identified by information that identifies the RFID tag. Further, each object has a set of RFID tags in accordance with a rule for arranging one RFID tag ahead of the other tag in the arrangement direction.

  Therefore, based on the fact that each set of RFID tags can be identified, the arrangement rule for arranging one RFID tag in front of the arrangement direction, and the position information of each RFID can be acquired. 150 determines the arrangement direction (viewing direction) of the object.

  According to this configuration, since the metadata can be determined based on the arrangement direction of the object, as an example, compared to the case where the metadata “adjacent” is determined only from the position information, It is possible to determine metadata indicating a more specific state having a directional meaning of “is present”.

  In addition, according to this configuration, since the metadata can be determined based on the viewing direction of the object, as an example, compared to the case where the metadata “adjacent” is determined only from the position information, “facing each other”. It is possible to determine metadata indicating a more specific state having a directional meaning of “Yes”.

  Here, it is desirable that the logical expression used by the determination unit 150 to determine metadata is a logical expression that requires the least amount of calculation for determination. Therefore, the determination unit 150 transmits the created logical expression to the determination unit 160. Thereafter, the determination unit 150 receives the lightweight logical expression determined by the determination unit 160.

  Next, the determination unit 150 determines metadata based on the received lightweight logical expression, sensor information, and environment information if necessary. Finally, the determination unit 150 transmits the sensor information received from the sensor management unit 140, the determined metadata, and the object identification information to the sensor information recording unit 190.

  The determination unit 160 is connected to the determination unit 150, the deformation rule database 170, and the search unit 230. The determination unit 160 receives a logical expression from the determination unit 150. Next, the determination unit 160 transforms the received logical expression into a plurality of other logical expressions representing the same content. Thereafter, the determination unit 160 determines a light-weight logical expression that is a logical expression that requires a small amount of calculation for the determination performed by the determination unit among the modified logical expressions.

Here, the process of determining the lightweight logical expression of the determination unit will be described with reference to FIG. FIG. 11 is an example of a flowchart illustrating the lightweight logical expression determination process of the determination unit.
First, the determination unit 160 receives a logical expression from the determination unit 150 (step ST1). Next, the determination unit 160 sequentially refers to the transformation rules of the logical expressions stored in the transformation rule database 170, and determines whether there is a transformation rule that can be applied to the received logical expression (step ST2). If it is determined that there is a transformation rule that can be applied to the received logical expression, the determination unit 160 performs the process of step ST4, and otherwise performs the process of step ST5.

Here, a deformation law used to reduce the amount of calculation of the logical expression will be described with reference to FIG. FIG. 12 is a diagram for explaining an example of a deformation law for reducing the amount of calculation of a logical expression.
FIG. 12 shows a logical expression describing a case where the regions Ar and Br have some spatial relationship and a case where the time intervals Ai and Bi have some temporal relationship. Yes.

  According to the Egenhofer spatial relationship model shown in FIG. 5, the region Ar and Br are spatially related. The region Ar includes the region Br (Ar Contain Br), and the region Ar is the region. The region Ar and the region Br are present (Ar Equal Br), the region Ar is in contact with the region Br (Ar Meet Br), and the region Ar is the region Br. (Ar Cover Br), the area Ar is covered with the area Br (Ar CoveredBy Br), or the area Ar overlaps the area Br (Ar Overlap Br). Therefore, the case where the regions Ar and Br have some spatial relationship can be described by the logical expression R1.

  In addition, the fact that the regions Ar and Br are spatially related means that if the double negation is used, the regions Ar and Br are not spatially related (Ar Disjoint Br) (not) ) And meaning content is the same. Therefore, the case where the areas Ar and Br have some spatial relationship can be described also by the logical expression R2.

  Further, the logical expression R1 is composed of seven binary association operators representing “Contain”, “Inside”, “Equal”, “Meet”, “Cover”, “CoveredBy”, and “Overlap”, and six logical sums “or”. . On the other hand, the logical expression 2 is composed of a binary combination operator that represents one spatial relationship called Disjoint and one logical negation called not. Therefore, the calculation amount of the logical expression can be obtained by calculating in advance the calculation amounts of the binary combination operator and the unary operator constituting the logical expression. Thereby, a transformation rule for transforming a logical expression with a large amount of calculation into a logical expression representing a logical content with a small amount of calculation can be determined in advance.

  According to Allen's temporal relevance model shown in FIG. 4, the temporal relevance that the time interval Ai is not before Bi is not (before (Ai, Bi), as shown in the logical expression I2. Can write).

  Further, the temporal relevance that the time interval Ai is not before Bi is equal to time interval Ai equal to Bi (equal (Ai, Bi)), and time interval Ai is later than Bi (after (Ai). Bi)), Bi is included in the time interval Ai (during (Ai, Bi)), Ai is included in the time interval Bi (contain (Ai, Bi)), and the time interval Ai overlaps with Bi. (Overlaps (Ai, Bi)), the time interval Bi overlaps with Ai (overwrapped_by (Ai, Bi)), the time interval Ai touches Bi (meets (Ai, Bi)), and the time interval Ai is simultaneously with Bi Either start (starts (Ai, Bi)) or time interval Ai ends at the same time as Bi (finishes (Ai, Bi)). Therefore, the logical expression I2 expressing the semantic content that the time interval Ai is not before Bi can be described by the logical expression I1.

  Furthermore, the calculation amount of the logical expression can be obtained by calculating in advance the calculation amounts of the binary combination operator and the unary operator constituting the logical expression. Thereby, a transformation rule for transforming a logical expression with a large amount of calculation into a logical expression representing a logical content with a small amount of calculation can be determined in advance.

  In addition, for a logical expression described using a new operator “reverse”, the amount of calculation for performing the “reverse” operation by transforming the sequence that is the target of the “reverse” operation into a logical expression that is sorted in reverse order in advance. Can be reduced.

  According to this configuration, in order to determine whether or not to apply the deformation rule by registering in the information processing apparatus in advance various deformation rules that are deformed into different logical expressions with the same logical contents in advance, for example, Compared with the method of transforming logical expressions according to mathematical rules such as distribution rules, combination rules, and exchange rules, which are mathematical properties, not only can the transformation rules be more flexible, but also the expansion and modification of the transformation rules. Is easy.

Here, returning to FIG. 11, the description of the process of determining the lightweight logical expression of the determination unit will be continued.
In step ST3, when determining that there is a modification rule that can be applied to the received logical expression, the determination unit 160 transforms the logical expression by applying all of the modification rules determined to be applicable (step ST4).

  In step ST3, when the determination unit 160 determines that there is no modification rule that can be applied to the received logical expression, or after executing step ST4, the determination section 160 deforms the logical expression by applying a mathematical distribution rule. Then, it is determined whether the calculation amount required for the determination performed by the determination unit 150 is reduced based on the modified logical expression (step ST5). The determination unit 160 executes the process of step ST6 when determining that the calculation amount is reduced by applying a mathematical distribution rule, and executes the process of step ST7 otherwise.

  In step ST5, when determining that the calculation amount is reduced by applying a mathematical distribution rule, the determination unit 160 applies the distribution rule to the logical expression to reduce the weight of the logical expression (step ST6). ).

  In step ST5, when the determination unit 160 determines that the calculation amount is not reduced by applying the mathematical distribution rule, or after executing step ST6, the determination unit 160 applies the mathematical combination rule. The expression is modified, and it is determined whether the calculation amount required for the determination performed by the determination unit 150 is reduced based on the modified logical expression (step ST7). The determination unit 160 executes the process of step ST8 when it is determined that the calculation amount is reduced by applying a mathematical combination rule, and otherwise executes the process of step ST9.

  In Step ST7, when the determination unit 160 determines that the calculation amount is reduced by applying a mathematical combination rule, the determination unit 160 applies the combination rule to the logical expression to reduce the weight of the logical expression (Step ST8). ).

  In step ST7, the determination unit 160 determines that the calculation amount is not reduced by applying the mathematical combination rule, or after executing step ST8, the determination unit 160 applies the mathematical exchange rule to perform the logic. The expression is modified, and it is determined whether the calculation amount required for the determination performed by the determination unit 150 is reduced based on the modified logical expression (step ST9). The determination unit 160 executes the process of step ST10 when it is determined that the calculation amount is reduced by applying a mathematical exchange rule, and otherwise executes the process of step ST11.

  In step ST9, when determining that the calculation amount is reduced by applying a mathematical exchange rule, the determination unit 160 applies the exchange rule to the logical expression to reduce the weight of the logical expression (step ST10). ).

In step ST9, the determination unit 160 determines that the calculation amount is not reduced by applying a mathematical exchange rule , or after executing step ST10, the logical expression in which the calculation amount is reduced by the above processing. (The logical expression received from the determination unit 150 when the calculation amount cannot be reduced) is transmitted to the determination unit 150 (step ST11). Then, the determination part 160 complete | finishes execution of a process.

Here, an example of reducing the amount of calculation using a mathematical law will be described with reference to FIG. FIG. 13 is a diagram for explaining an example of reducing the amount of calculation using the coupling law.
FIG. 13 shows a logical expression expressing a case where the person object A and the person object B are in contact with the whiteboard H.

  Saying that the person object A and the person object B are in contact with the whiteboard H is equivalent to saying that the whiteboard H is in contact with the person object A and the person object B. Therefore, the logical expression 21 and the logical expression 22 represent the same logical contents. Further, the logical expression 22 can be derived by applying the exchange rule to the logical expression 21, and conversely, the logical expression 21 can be derived by applying the exchange rule to the logical expression 22.

Here, the calculation amount required for the determination unit 150 to determine the metadata based on the logical expression 21 is equal to the calculation amount based on the logical expression 22. Therefore, the calculation amount of the logical expression 21 cannot be reduced by applying the exchange rule .

  Next, when the person object A and the person object B are in contact with the whiteboard H, the person object A is in contact with the whiteboard H and the person object A is in contact with the whiteboard H. Equal to say. Therefore, the logical expression 22 and the logical expression 23 represent the same logical contents. Further, the logical expression 23 can be derived by applying the distribution rule to the logical expression 22, and conversely, the logical expression 22 can be derived by applying the coupling rule to the logical expression 23.

  Here, the calculation amount required for the determination unit 150 to determine metadata based on the logical expression 22 is a calculation that calculates the operator “meets” once rather than the calculation amount based on the logical expression 23. Less by the amount. Therefore, the calculation amount of the logical expression 22 cannot be reduced by applying the distribution rule. On the contrary, the logical expression 23 can reduce the amount of calculation by applying a coupling rule, and the lightweight logical expression 21 or 22 can be derived.

According to this configuration, since the logical expression can be transformed by a logical operation, as an example, a modification rule that transforms into a logical expression with a small amount of calculation is registered in the information processing apparatus in advance, and the modification rule is applied. Compared with the method for determining whether or not it is possible, it is possible to reduce the labor of the administrator resulting from the addition or change of the deformation rule.
Furthermore, according to this configuration, since the metadata can be determined after determining a logical expression with a small amount of calculation required for determination, as an example, compared with the case of determining without determining a logical expression with a small amount of calculation It is possible to suppress an increase in determination time accompanying an increase in the number of sensor information.

  The deformation rule database 170 is connected to the determination unit 160. The transformation rule database 170 stores transformation rules for transforming a logical expression having a large calculation amount into a logical expression having a small calculation amount. In the deformation rule database 170, the determination unit 160 refers to the deformation rules to be accumulated.

  The metadata database 180 is connected to the determination unit 150 and the matching maintenance unit 210. The metadata database 180 stores the metadata definition and metadata described in FIG. 6 in association with each other.

Moreover, the relationship between the metadata which is not attached | subjected to the same object at the same time is also accumulate | stored. The relationship between metadata is, for example, the consistency between the metadata attached to the same object at the same time “facing each other” and “sitting facing each other” because the semantic contents of the metadata do not contradict each other. However, the metadata “ discussion ” and “silence” to which the same object is attached at the same time are in a relationship that the semantic contents of the metadata are inconsistent with each other and thus are not consistent.

  The metadata database 180 sets priorities for combinations of a plurality of metadata whose consistency in semantic content is lost by attaching the same object at the same time, and the relationship between the priorities and each object and each object. It is accumulated in association with.

  The metadata database 180 is referred to by the determination unit 150 for metadata and metadata definitions. In the metadata database 180, the consistency maintaining unit 210 refers to the relationship between metadata and the metadata and the priority order.

  The sensor information recording unit 190 is connected to the determination unit 150 and the sensor information database 200. The sensor information recording unit 190 stores the sensor information, metadata, object identification information, and time information received from the determination unit 150 in association with the sensor information database 200.

  The sensor information database 200 is connected to the sensor information recording unit 190, the matching maintenance unit 210, and the search unit 230. The sensor information database 200 stores sensor information, metadata, object identification information, and time information associated by the sensor information recording unit 190.

  The sensor information database 200 is referred to by the matching maintenance unit 210 and the search unit 230 for the stored sensor information, metadata, object identification information, and time information.

Here, an example of the table configuration of the sensor information database will be described with reference to FIG. FIG. 14 is a diagram for explaining an example of a table configuration of the sensor information database.
The sensor information database 200 includes a management table TA, one or more sensor information tables TS, and a TM metadata table.

  The management table TA includes, for example, dates that are bibliographic information such as conferences to be recorded, information that identifies meeting rooms, persons who participated in conferences, tangible and intangible objects used in conferences, etc. Accumulate and manage.

  The sensor information table TS stores, for example, sensor information, object identification information, and time information detected in a conference or the like to be recorded in association with each other.

  The metadata table TM stores, for example, metadata determined by the determination unit 150 based on sensor information and logical expressions detected in a conference or the like to be recorded in association with time information and object identification information. Further, the metadata is stored in association with the identification information of the object related to the metadata. For example, an object related to metadata “editing” refers to a person object involved by editing an electronic data object to which metadata is attached. Note that one or two or more pieces of metadata are associated and stored in the same time information and object.

  According to this configuration, since the object is detected as sensor information and the metadata is determined based on the direction information, in two or more objects, the spatial relationship between each object and the object and time Can be stored in association with each other.

  The matching maintenance unit 210 is connected to the metadata database 180, the sensor information database 200, the display unit 250, and the input unit 240. The consistency maintaining unit 210 refers to the metadata stored in the metadata database 180.

  The consistency maintaining unit 210 determines that there is consistency when there is one piece of metadata assigned to the same object at the same time. On the other hand, if there are a plurality of metadata, it is determined whether the semantic contents of the plurality of metadata are consistent with each other. This is determined based on the referenced relationship by referring to the relationship between the metadata stored in the metadata database 180.

  If the consistency maintaining unit 210 determines that some of the metadata assigned to the same object at the same time is not consistent with each other, the consistency maintaining unit 210 selects one of the metadata that is not consistent with each other or Other metadata is deleted from the metadata database 180, leaving only a plurality of metadata that are consistent with each other.

  Specifically, the consistency maintaining unit 210 refers to the relationships and priorities between metadata associated with the metadata stored in the metadata database 180, and among the metadata that are not consistent with each other. The remaining metadata is deleted from the metadata database 180, leaving only the metadata having consistency in descending order of metadata.

According to this configuration, it is not necessary to make a precise determination based on a logical expression, so that the configuration of the metadata determination unit can be simplified. Further, it is possible to flexibly cope with addition or change of metadata.
Further, according to this configuration, since the metadata is determined based on a predetermined priority order, for example, compared with the method of determining the metadata based on the user's selection, the data is stored in the sensor information storage unit. It is possible to suppress an increase in user management labor accompanying an increase in the number of metadata.

  As another method for maintaining consistency, the consistency maintaining unit 210 transmits metadata that is determined not to be consistent with each other and a message that prompts the user 300 to select metadata, to the display unit 250. Thereafter, one or more metadata selected by the user 300 is received from the input unit 250. Thereafter, other metadata is deleted from the metadata database 180, leaving only the received metadata.

According to this configuration, it is not necessary to make a precise determination based on a logical expression, so that the configuration of the determination unit can be simplified. Further, it is possible to flexibly cope with addition or change of metadata.
Further, according to this configuration, since the metadata is determined based on the user's selection, the determination is performed by the metadata determination unit, for example, as compared to the method of determining the metadata based on a predetermined priority order. A discrepancy between the metadata and the metadata recognized by the user through actual observation hardly occurs.

  The environment information database 220 is connected to the determination unit 150 and the search unit 230. The environment information database 220 is stored in association with information representing the imaging area of the video cameras 111 to 11n in coordinates and information for identifying the video camera. Further, since the area occupied by the object cannot be detected only by the position coordinates detected by the RFID readers 1431 to 143m ′ ″, the area occupied by all objects and the object identification information are stored in advance. Information stored in the environment information database 220 is referred to as environment information.

  The environment information database 220 is referred to by the determination unit 150 to determine the spatial relationship between the area occupied by the accumulated object and the object identification information. In addition, the environment information database 220 is referred to by the search unit 230 for searching for a moving image, the area occupied by the accumulated object, the object identification information, the camera identification information, and the imaging area.

  The search unit is connected to the moving image database 130, the determination unit 160, the sensor information database 200, the environment information database 220, the display unit 250, and the input unit 240.

  The search unit 230 receives the search condition from the input unit 240. Next, the search unit 230 generates a logical expression representing the received search condition. Thereafter, the search unit 230 transmits the generated logical expression to the determination unit 160. Next, the search unit 230 receives the lightweight logical expression from the determination unit 160.

  Thereafter, the search unit 230 uses the moving image database 130 based on the received logical expression, the metadata stored in the sensor information database 200 (and the environment information stored in the environment information database 220 if necessary). , Search for information stored in the sensor information database 200 or both. Thereafter, the search unit 230 transmits the search result to the display unit 250.

  As a first specific example, an example of searching for an object characterized by metadata “discussion” will be described. Further, an example in which a moving image obtained by capturing an object characterized by metadata “discussion” is searched from the moving image database 130 will be described.

  First, the search unit 230 acquires a lightweight logical expression that represents a search condition for searching for an object characterized by metadata “discussion”. Thereafter, the search unit 230 searches the metadata table TM of the sensor information database 200 for object identification information that satisfies the lightweight logical expression, and acquires time information associated with the searched object identification information.

  According to this configuration, since an object can be searched based on metadata, the object can be specified using the metadata that characterizes the object as a clue. As an example, when the “discussion” characterizes the behavior of a person who is an object in a meeting, the person can be identified using the metadata “discussion” as a clue.

  Thereafter, the search unit 230 acquires the position information of the object identified by the searched object identification information at the searched time from the sensor information database 200. Next, the search unit 230 specifies the video camera that captured the searched object based on the imaging region stored in the environment information database 220.

  Next, the search unit 230 acquires a moving image captured by the identified video camera from the moving image database 130, and acquires a moving image that matches the searched time information, thereby obtaining a moving image as a search result. .

  According to this configuration, the moving image can be searched based on the metadata, and as an example, the moving image of the object to be imaged can be searched based on the metadata representing the feature of the object. As an example, in order to specify a moving image that captures the motion of an object characterized by “discussion”, it can be specified by specifying the moving image using the metadata “discussion” as a clue.

  As a second specific example, an example in which metadata is searched based on an object will be described. More specifically, there is an electronic file that represents a plurality of versions of a conference slide, metadata such as the projection history of a specific version of the electronic file is searched, and for each projection history that is the searched metadata. An example of acquiring a projector and a projection target screen will be described.

  First, the search unit 230 acquires a lightweight logical expression that represents a search condition using object identification information of a specific version of an electronic file as a search key. Then, metadata such as projection history can be acquired by acquiring data satisfying the lightweight logical expression from the sensor information database 200.

  In particular, when a related object that is a part of data satisfying a lightweight logical expression is an object related to metadata “projection”, it is a screen, a projector, an operator, an instructor (presenter), etc. It is possible to acquire a projector and a projection destination screen for each history.

  In addition, the search unit 230 obtains a lightweight logical expression that represents a search condition using the object identification information of the specific version of the electronic file and the metadata “projection” as search keys, thereby projecting the projection history of the specific version of the electronic file. Can be obtained.

  According to this configuration, the metadata can be searched based on the object, so that the metadata representing the state or action of the object at a specific time can be acquired using the object as a clue. As an example, the projection history of an electronic file that is metadata can be acquired using an electronic file that is an object as a clue.

Next, a third specific example will be described with reference to FIG. FIG. 15 is a diagram for explaining an example of a search process for searching for an editor based on an editing history.
In FIG. 15, the person identified by the object identification information 010 in the electronic file A ′ representing the conference slide has been modified to a different version of the file A ″ by additionally editing the word “data mining”. Represents. The modified electronic file A ″ may be further edited by adding the word “extracted” by the person identified by the object identification information 020 and modifying it to a different version of the file A ′ ″. Represents.

  Here, regarding the process of searching for the electronic file created by the process of editing the words “data mining” and “extraction” in order, the editor of the electronic file, and the moving image capturing the state at the time of editing explain.

  First, the search unit 230 acquires a lightweight logical expression that represents a search condition. Specifically, the spatial relationship between an object called “data mining” and “extraction” and an electronic file is described by a logical expression. Since the electronic file A ″ contains the words “data mining” and “extraction”, the spatial relevance can be expressed as (A ″ container B) and (A ″ containC).

  Further, since the electronic file A is overwritten and modified by a different version of the file A ′, the temporal relationship can be expressed as (A meet A ′). Further, the relationship between the electronic files A ′ and A ″ can be expressed as (A ′ meet A ″). Therefore, both logical expressions can be combined and expressed as (A meet A ′) and (A ′ meet A ″).

  Thereafter, the search unit 230 acquires object identification information of a file to be searched by acquiring data satisfying the lightweight logical expression from the sensor information database. In addition, the search unit 230 acquires related object identification information and time information associated with the acquired object identification information and metadata “edit” from the sensor information database. Thereby, the person who edited the file and the file relationship before and after editing can be acquired.

  Further, the search unit 230 acquires a moving image obtained by capturing a person object or the like obtained by editing an electronic file to be searched based on the acquired time information and object identification information or related object identification information from the moving image database.

  The display unit 250 includes, for example, a liquid crystal display, a plasma display, or a CRT display. The display unit 250 is connected to the matching maintenance unit 210 and the search unit 230. The display unit 250 receives a message for prompting selection of metadata and the metadata to be selected from the matching maintenance unit 210, and displays the received search result. The display unit 250 receives the search result from the search unit 230 and displays the received search result.

  The input unit 240 is composed of, for example, a keyboard or a touch panel. The input unit 240 is connected to the matching maintenance unit 210 and the search unit 230. The input unit 240 receives metadata selected by the user 300. The input unit 240 transmits the input metadata to the matching maintenance unit 210. The input unit 240 receives a search condition from the user 300. The input unit 240 transmits the input search condition to the search unit 230.

  The information processing method according to the present invention is realized by the information processing apparatus 100. The information processing method of the present invention can be realized as a program that is executed by controlling a computer. This program can be provided by being stored and distributed in a magnetic disk, an optical disk, a semiconductor memory, or other recording media, or distributed via a network.

  In the above embodiment, the case where the CCD camera is used has been described. However, the present invention is not limited to this. For example, a camera using a CMOS image sensor or the like may be employed.

  In the above embodiment, the case where the video camera is fixed has been described. However, the present invention is not limited to this. For example, the video camera is not fixed, the sensor management unit 140 detects the imaging area of the video camera, and the environment It is also possible to employ a configuration in which the time information and the imaging area are stored in the information database 220 in association with each other.

  In the above embodiment, the case where the deformation rule is determined based on the spatial relationship between the regions in the Egenhofer spatial relationship model has been described. However, the present invention is not limited to this, and for example, the Egenhofer space Spatial relationship between a region and a line segment, spatial relationship between a line segment and a line segment, spatial relationship between a region and a point, spatial relationship between a line segment and a point, a point It is also possible to adopt a configuration in which the deformation rule is determined based on the spatial relationship between the point and the point and the temporal relationship in the Allen time interval model.

  In the above embodiment, an electronic file or the like has been described as an intangible object. However, the present invention is not limited to this. For example, a time interval of two days is treated as an intangible object, and the time interval object is It is also possible to employ a configuration used for search conditions.

1 is a configuration diagram of an image processing apparatus according to an embodiment of the present invention. It is a block diagram of the camera management part which concerns on one Example of this invention. It is a block diagram of the sensor management part which concerns on one Example of this invention. It is a figure for demonstrating the time interval model of Allen. It is a figure for demonstrating the spatial relevance model of Egenhofer. It is a figure for demonstrating an example of the definition of the metadata determined by the determination part. It is a figure for demonstrating an example of the analysis method of slide data. It is a figure for demonstrating a new operator. It is a figure for demonstrating an example of the acquisition method of an object moving direction. It is a figure for demonstrating an example of the acquisition method of an object arrangement | positioning direction. It is an example of the flowchart which shows the lightweight logic formula determination process of a determination part. It is a figure for demonstrating an example of the deformation | transformation law which makes the calculation amount of a logical formula light. It is a figure for demonstrating an example of the weight reduction of the computational complexity using a coupling law. It is a figure for demonstrating an example of the table structure which a sensor database has. It is a figure for demonstrating an example of the search process which searches for an editor based on an edit history.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Information processing apparatus 110 ... Camera management part
111 to 11n: Video camera 120: Moving image recording unit
130 ... moving image DB (moving image storage means) 140 ... sensor management unit ( input means)
1401-140m ... file server
1411-141m '... mail server 1421-142m''... projector
1431 to 143m '''... RFID reader 1441 to 143m''''... microphone 150 ... determination unit (determination means) 151 ... technical term storage unit
160: Determination unit (determination means) 170 ... Deformation rule DB
180 ... metadata DB 190 ... sensor information recording unit
200 ... sensor information DB (sensor information storage means)
210 ... matching maintenance unit (matching maintenance means) 220 ... environment information DB
230: Search unit (search means) 240 ... Input unit
250: Display unit 300: User
C001 to C00n: ... Camera ID DS ... Slide data
I: Time interval m1. emp: ... end time of time interval m1
m1. stp: ... start time of time interval m1 m2. emp: ... the start time of time interval m2
m2. stp: ... start time of time interval m2 P ... time
S ... Screen TA ... Management table
TM ... Metadata table TS ... Sensor information table

Claims (15)

Input means for inputting the sensor information output by one or more sensors that detect, as sensor information, a target object that is at least one of a person, a tangible object, and an intangible object;
Determination means for determining explanation information for explaining the object based on sensor information input by the input means;
An operation for transforming a logical expression representing at least one of the specific positional relationship and temporal relationship between the objects into a plurality of other logical expressions representing the same contents as the logical expression and constituting the modified logical expression A determination unit that determines a light-weight logical expression, which is a logical expression with a small amount of calculation, based on a calculation amount required for the determination performed by the determination unit for each child ;
The information processing apparatus according to claim 1, wherein the determination unit determines the description information based on the lightweight logical expression determined by the determination unit and the sensor information . 2. The information processing according to claim 1 , wherein the determination unit determines the light-weight logical expression by modifying the logical expression by a combination of a distribution rule and a combination rule, which are mathematical laws, and logical negation. apparatus. 2. The determination means according to claim 1 , wherein the light-weight logical expression is determined using a deformation rule defined by Allen's temporal relevance and a logical expression deformation rule that reduces a calculation amount. The information processing apparatus described. The determination unit is a modified rules determined by the spatial relationship of Egenhofer, determining the weight logic equation using the transformation rules of logical expressions to reduce the calculation amount, it in claim 1, wherein The information processing apparatus described. 4. The information according to claim 3 , wherein the deformation rule determined by the temporal relevance of Allen is a deformation rule that deforms from the left side to the right side while satisfying the following logical expression (1). Processing equipment.
(Equation 1)
equal (Ai, Bi) or after (Ai, Bi) or duration (Ai, Bi) or contains (Ai, Bi) or overlaps (Ai, Bi) or overlaid_by (Ai, Bi) or met (Ai, Bi) or met (Ai, Bi) or start (Ai, Bi) or finishes (Ai, Bi) = not (before (Ai, Bi)) (1)
However, Ai and Bi represent time intervals. 5. The information according to claim 4 , wherein the deformation rule defined by the Egenhofer spatial relationship is a deformation rule that deforms from the left side to the right side by satisfying the following logical expression (2). Processing equipment.
(Equation 2)
(Ar Contain Br) or (Ar Inside Br) or (Ar Equal Br) or (Ar Meet Br) or (Ar Cover Br) or (Ar CoveredBy Br) or (Ar Overlap Br) = not Ar・ (2)
However, Ar and Br represent regions. The information processing according to any one of claims 1 to 6 , further comprising sensor information storage means for storing the explanation information determined by the determination means, the sensor information, and time information in association with each other. apparatus. The object has a priority;
8. The information processing apparatus according to claim 7 , further comprising a consistency maintaining unit that maintains the consistency of the contents of the plurality of pieces of explanation information stored in the sensor information storage unit based on the priority order. 8. The information processing apparatus according to claim 7 , further comprising: a consistency maintaining unit that maintains the consistency of the contents of the plurality of pieces of explanation information stored in the sensor information storage unit based on a user selection. The input means inputs the sensor information output by a sensor that detects an object at the time of capturing a moving image as sensor information;
The information processing apparatus according to claim 5 , further comprising moving image storage means for storing the moving image and time information in association with each other . The information processing apparatus according to claim 10 , further comprising search means for searching for the moving image based on the explanation information determined by the determination means. The information processing apparatus according to claim 10 , wherein the search unit searches for the object based on the explanation information determined by the determination unit. The information processing apparatus according to claim 10 , wherein the search unit searches the explanation information determined by the determination unit based on the object. An input step of inputting the sensor information output by one or more sensors that detect, as sensor information, an object to be described that is at least one of a person, a tangible object, and an intangible object;
A determination step of determining explanation information for explaining the object based on the sensor information input in the input step;
For each operator constituting the deformed logical expression, a logical expression representing at least one of a specific positional relationship and temporal relationship between objects is transformed into other logical expressions representing the same contents as the logical expression. A determination step of determining a light-weight logical expression that is a logical expression with a small amount of calculation based on the calculation amount required for the determination performed by the determination step of :
The information processing method according to claim 1, wherein the determining step determines the explanation information based on the lightweight logical expression determined by the determining step and the sensor information . Computer
Input means for inputting the sensor information output by one or more sensors that detect, as sensor information, a target object that is at least one of a person, a tangible object, and an intangible object;
Determination means for determining explanation information for explaining the object based on sensor information input by the input means;
An operation for transforming a logical expression representing at least one of the specific positional relationship and temporal relationship between the objects into a plurality of other logical expressions representing the same contents as the logical expression and constituting the modified logical expression based on the calculation amount required for the determination performed by the said determining means for each child, an information processing program to function as a determining means for determining the amount of calculation less logical expression a is lightweight formulas,
The information processing program characterized in that the determination means determines the explanation information based on the lightweight logical expression determined by the determination means and the sensor information .

Images