Continuing our previous research to visually extract and visually and conceptually match weapons, this study develops
a method to determine whether a set of weapon parts visually extracted from images taken from different scenes can be
assembled as a firing weapon. This new approach identifies potential weapons in the ontology via tracing detected
necessary and sufficient parts through their meronymic relation to the whole weapon. A fast algorithm for identifying
potential weapons that can be assembled from a given set of detected parts is presented.
We empirically test the capacity of an improved system to identify not just images of individual guns, but partially occluded guns and their parts appearing in a videoframe. This approach combines low-level geometrical information gleaned from the visual images and high-level semantic information stored in an ontology enriched with meronymic part-whole relations. The main improvements of the system are handling occlusion, new algorithms, and an emerging meronomy. Well-known and commonly deployed in ontologies, actual meronomies need to be engineered and populated with unique solutions. Here, this includes adjacency of weapon parts and essentiality of parts to the threat of and the diagnosticity for a weapon. In this study video sequences are processed frame by frame. The extraction method separates colors and removes the background. Then image subtraction of the next frame determines moving targets, before morphological closing is applied to the current frame in order to clean up noise and fill gaps. Next, the method calculates for each object the boundary coordinates and uses them to create a finite numerical sequence as a descriptor. Parts identification is done by cyclic sequence alignment and matching against the nodes of the weapons ontology. From the identified parts, the most-likely weapon will be determined by using the weapon ontology.
In the present paper we study the problem of weapon identification and threat assessment from a single image with a partially occluded weapon. This problem poses very severe restrictions. To successfully identify a weapon from its parts we extend the first firearm ontology with the meronymic (partonomic) principle which lets us distinguish parts of a gun (e.g., lock, barrel, stock, scope). Adding classes of meronymic information provides meta-data (necessary for threat assessment) and allows for fast and accurate search. Searching for a weapon is treated conceptually as searching for the sum of its parts. An expanding active contour and morphological techniques are applied to partition weapons and extract boundaries, and a minimal inscribed complex polygon. Finite numerical sequences are generated, from the extracted geometric features, and are used to label partonomic nodes and perform quick and accurate search. The paper reports experimental results on weapons partitioning and search.
The work that established and explored the links between visual hierarchy and conceptual ontology of firearms for the purpose of threat assessment is continued. The previous study used geometrical information to find a target in the visual hierarchy and through the links with the conceptual ontology to derive high-level information that was used to assess a potential threat. Multiple improvements and new contributions are reported. The theoretical basis of the geometric feature extraction method was improved in terms of accuracy. The sample space used for validations is expanded from 31 to 153 firearms. Thus, a new larger and more accurate sequence of visual hierarchies was generated using a modified Gonzalez’ clustering algorithm. The conceptual ontology is elaborated as well and more links were created between the two kinds of hierarchies (visual and conceptual). The threat assessment equation is refined around ammunition-related properties and uses high-level information from the conceptual hierarchy. The experiments performed on weapons identification and threat assessment showed that our system recognized 100% of the cases if a weapon already belongs to the ontology and in 90.8% of the cases, determined the correct third ancestor (level concept) if the weapon is unknown to the ontology. To validate the accuracy of identification for a very large data set, we calculated the intervals of confidence for our system.
The present work is part of an ongoing larger project.2, 3, 11, 12 The goal of this project is to develop a system
capable of automatic threat assessment for instances of firearms use in public places. The main components
of the system are: an ontology of firearms;1, 14 algorithms to create the visual footprint of the firearms,1, 14 to
compare visual information,2, 3, 11, 12 to facilitate search in the ontology, and to generate the links between the
conceptual and visual ontologies; as well as a formula to calculate the threat of individual firearms, firearms
classes, and ammunition types in different environments.
One part of the dual-level ontology for the properties of the firearms captures key visual features used to
identify their type or class in images, while the other part captures their threat-relevant conceptual properties.
The visual ontology is the result of image segmentation and matching methods, while the conceptual ontology
is designed using knowledge-engineering principles and populated semi-automatically from Web resources.
The focus of the present paper is two-fold. On the one hand, we will report on an update of the initial
threat formula, based on the substantially increased population of the firearm ontology, including ammunition
types and comparisons to actual incidents, and allowing for an overall more accurate assessment. On the other
hand, the linking algorithms between the visual and conceptual ontologies are elaborated for faster transfer of
information leading to an improvement in accuracy of the threat assessment.
The present work is a part of a larger project on recognizing and identifying weapons from a single image and assessing threats in public places. Methods of populating the weapon ontology have been shown. A clustering-based approach of constructing visual hierarchies on the base of extracted geometric features of weapons has been proposed. The convergence of a sequence of visual hierarchy trees to a conceptual hierarchy tree has been discussed. For illustrative purposes, from the growing conceptual ontology, a conceptual hierarchy tree has been chosen as a point of convergence for a sequence of visual hierarchy trees. A new approach is defined, on the base of the Gonzalez' algorithm, to generate the visual hierarchies. The closest visual hierarchy tree is selected as the search environment for a query weapon. A method of threat assessment is proposed. This method uses the attribute-rich conceptual hierarchy tree to evaluate the results from the visual hierarchy tree search. The two trees are linked at the leaf-level, because the visual hierarchy closest to the conceptual has the same distribution of the leaf nodes. A set of experimental results are reported to validate the theoretical concepts. A portion of the existing weapon ontology is used for this purpose.