Quantum 3D Microscope Q&A

Quantum uses a patented non-linear stereo-photometry method. The bullet surface is imaged with a set of LEDs at specific locations. Analysis of the intensity of each pixel within the set of LED images allows Quantum to determine local slopes. The topography is recovered by a numerical integration calculation over the set of slopes.

The Quantum 3D Microscope is based on the new IBIS BULLETTRAX model released in 2019. The new BULLETTRAX provides calibration capabilities for measurement traceability. Its wider field of view offers more viewing context, but also allows for new methods like the subclass detection warning feature. The RBL model will evolve based on the 3D data acquired using the new non-linear stereo-photometry method. (See questions on the dual-purpose use of Quantum with IBIS.)

Lateral resolution is slightly less than 3 µm/pixel.
Depth resolution is around 0.3 µm.

The acquired surface is a band that is 2.8 mm (1/9″) wide and that tracks down or around the object.

The width of the acquisition is 2.8 mm (1/9″). Multiple passes would be required to capture the entire sidewall body of a cartridge case. When using Quantum as a 3D microscope you can target specific areas of interest between samples; Entire wraparound images are not always necessary.

Objects must be within the following dimensions to fit through the opening:

Width: Up to 45 mm (1.8″)

Height: Up to 28 mm (1.1″)

Length: Up to 50 mm (2.0″)

During the acquisition, the maximum distance from the rotation axis to the surface being acquired is 28 mm (1.1″).

A wraparound acquisition of a 9 mm bullet currently takes approximately 11 minutes. Most of that time is completely automated, so the user is not tied to the acquisition machine while it is acquiring. Quantum acquisition times for damaged bullets will vary based on the size of the surface being acquired as well as the severity of the damage present. The more complex surfaces take more time. The example acquired during the online event presentation, which had three LEAs and moderate damage, took eight minutes and 50 seconds, including acquisition setup and validation. Had there been a fourth LEA it would have taken longer – Had there been only two LEAs it would have been quicker. Our experience is that most deformed objects can be acquired within 15 minutes.

Yes, wraparound or targeted-region acquisitions of cartridge cases can be done in Quantum. The Quantum S1 model can acquire cartridge case chamber marks, ejection port marks, extractor marks, and other marks on the side of a cartridge case.

The first model of Quantum (S1) is not well-suited to acquire the cartridge case’s entire head, or large flat areas, but it can acquire wraparounds or targeted regions of cartridge case chamber marks, ejection port marks, extractor marks, and other marks on the side of a cartridge case.

Implementing a Quantum 3D Microscope for cartridge cases (heads) is the next logical step, and another Quantum model will be specialized for cartridge case heads and larger flat objects with impressed and striated toolmarks. We have started work on the more complex components that are required, so expect more news to come as we progress. Over the years, our 3D technology has helped firearm examiners with difficult cases involving bullets that were difficult to examine under conventional comparison microscopes. We applied our expertise to find solutions and made scientific breakthroughs to develop objective identification of bullets.

Calibration is based on ISO17025 certified targets. Each Quantum 3D Microscope has certified targets that are traceable through ISO17025 certified suppliers. A calibration report is issued for the system comparing its actual measurements with the calibrated measurements. Each 3D acquisition is linked to the last calibration check information.

Users can perform calibration checks either periodically or manually, to validate the measurements. If using an external reference target, the 3D acquisition data can be exported for analysis.

During the online event presentation, we explained the Best Phase model that is optimized for the average of the two highest LEA scores at the best phase. Quantum also has a Best LEA model that uses the highest LEA-to-LEA score. This model will work for fragments that have only one LEA to compare, as well as bullets and fragments with multiple LEAs. The Best LEA and Best Phase models each have different separation line and FMR isolines.

The Likelihood Ratio is a challenge because it is computed from both the non-match and match score distributions. For a statistical distribution to be valid, it must be generated from a high number of scores, and we have many more non-match scores at our disposal than match scores. Therefore, we decided to focus on the FMR for the moment, because it is only computed from the non-match score distribution. We also found that the match score distribution is much more sensitive to the particular firearm while the non-match distribution is much more stable. We might consider computing the Likelihood Ratio in future versions of Quantum, if required by the community.

In our casework examples, we found that any 5 test fires were useful in representing a firearm’s markings. However, it is common to use the first test fires after recovering the firearm as these are more representative of the barrel’s performance when it was last used.

The current algorithm allows some stretching in the comparison process. The largest amount of stretching allowed was found by optimizing the separation of the match and non-match score distributions. Allowing for stretching increases both the match and non-match score values. When the allowed stretching become too large, the non-matches are favored and the match and non-match score distributions start to overlap.

The system does not eliminate matches but provides a false match error rate (FMR) for each score. It is the responsibility of the experts to provide conclusions, but the FMR is a quantitative measure that can support their conclusion.

The concept of false elimination must be defined with caution in the context of bullet identification. In contrast to biometrics which are stable over time, there might be significant variations on bullet toolmarks from one firing to the next. Sometimes, a bullet pair is a known match, but its level of similarity is not consistent with a high score value. In our opinion, it is not appropriate to define such a case as a false elimination because the expert would not classify such a pair as a conclusive match (even if it is a known match).

Quantum generates a score consistent with the similarity between two bullets. As discussed in our paper, our training set consisted of 422 known matching pairs of 9 mm bullets, among which only 55% could be matched visually. This visual analysis is unbiased as it was done prior to any algorithm development. All except 8 of the scores of the known non-matches fall below the separation line (for FMR = 1/10000). Five of the 235 visually matching pairs (2%) were found slightly below the separation line which is consistent with the observed level of similarity.

The learning was based on acquisitions performed on a Quantum acquisition unit at Ultra-FT from the dataset of different firearm manufacturers that are representative of the population of handguns in an American urban environment. There is no mechanism for users of Quantum to perform system “learning”.

The FMR=1/10000 line is not a formal decision line. Matches are not always above this line and Non-Matches are not always below the line. Therefore, we prefer to use the expression « separation line ». This line approximately separates the bullet pairs that are very similar from those that look significantly different as judged from visual analysis on a conventional comparison microscope or a virtual comparison microscope. So, there is no implication related to changing the definition of the separation line on the graph, which could be set at 1/100 or 1/1000. However, we have noted that the level of similarity becomes significant for scores above the FMR = 1/10000 line. It remains the decision of the expert to conclude that a given bullet pair is a conclusive match or not. The FMR computed from the score is provided as support to that conclusion.

This functionality was not discussed during the online event presentation. To our knowledge, the most difficult bullets, in terms of subclass characteristics, are similar to the Norris dataset, which corresponds to 10 pairs of bullets fired from consecutively manufactured barrels and 15 unknowns. For this data, most marks are found to be strong subclass marks. Most of the marks are common to the whole set, and the level of similarity between known non-matches is typical of that found for good matches. Subclass marks cannot be detected automatically from the topography of a bullet, and experts must rely on the actual barrel to find subclass marks. However, Quantum will include an algorithm which detect long marks that cross most of the width of the image. A warning will be issued to the user if the number of such marks is above a critical value, which implies that strong subclass might be present, and, if so, that the FMR is not valid.

Different models must be developed for each of these rifling types because bullets are marked very differently, and match and non-match distributions will likely be significantly different. New algorithms and similarity scores for polygonal and unrifled are planned for future development, but we are dependent on feasibility with the underlying science.

Yes, the RBL method will be provided for other calibers, but initially only for conventional rifling. Support for polygonal and unrifled is planned for future development.

The statistical model presented during the online event presentation was based on a database of 9 mm copper-jacketed bullets, but Quantum will support other calibers and material compositions.

IBIS compares a given exhibit against a large database with a fast algorithm. Computing time of each individual bullet-to-bullet correlation is as short as possible. However, the total computing time of a correlation request between one bullet and a large database can be of the order of hours.

In Quantum, small datasets are considered since it is specifically designed for case work. The correlation algorithm takes more time per bullet-to-bullet comparison, but the total computing time for the correlation of a typical dataset is in seconds or minutes.

A strong advantage of the PMS and LCS scores is their interpretability. The two types of scores constitute complementary viewpoints which take their root in the method used by firearm experts. The respective value of these two scores strongly agrees with the observed similarity of any given pair of bullets in a way consistent with the pattern matching or line counting viewpoints. However, computing a False Match Error Rate (FMR) value from a two-dimensional score distribution constitutes a challenge, especially in the high score regime where non-match data is scarce. We thus apply a standard dimensionality reduction method, which is widely used by the machine learning community, at the price of a lower interpretability. Dimensionality reduction is the process of reducing the number of variables under consideration by obtaining a smaller set of principal variables which capture most of the behavior of the original multi-dimensional statistical distribution. The resulting score in Quantum is the projected distance D, which is a weighted sum of the PMS and LCS. In this process, we are in fact defining a single score that linearly combines the PMS and the LCS. This score can be used to define an FMR, but it cannot be interpreted as easily as the individual PMS and LCS. We thus propose to keep the 2D graph (PMS-LCS) representation for ease of interpretation but use their weighted linear combination D for the computation of the FMR.

The grayed-out points represent the similarity scores from the dataset of 406 bullets (our training set) used to define the separation line and the FMR isolines. They are drawn in the background for reference. In contrast, the colored points are the results obtained from comparing the samples being analyzed in the current workspace.

This step is explained in our paper under section 4.7 [Objective Identification of Bullets Based on 3D Pattern Matching and Line Counting Scores. International Journal of Pattern Recognition and Artificial Intelligence, 2019, 33(11)]. The large number of non-match scores (80K) allows us to extrapolate the behavior of the non-match score distribution based on the 1% (i.e., 800) largest scores. The shape of the FMR function in the high score regime is then found to be an exponentially decreasing function, that is, its logarithm is a linear function, as displayed in Fig.15. A fitting procedure is applied to define the best fitting line, which is then used to extrapolate the FMR function in the score domain where non-match data are not available. Such extrapolation is evidently highly speculative due to missing data from the non-match distribution. However, the exponential fitting is much more conservative than the Gaussian fitting one would expect based on the overall score distribution.

When we trained the RBL comparison algorithms, we used test fires for practical reasons. It is challenging to accumulate a large dataset of damaged and fragmented bullets that are part of known matches. However, we were able to simulate the effects of deformation and missing LEAs as follows. The comparison algorithm has been calibrated to allow some stretching when comparing any pair of LEAs. A compromise was found in order to improve the results for the matches without being too permissive for nonmatches. We also studied the behavior of the comparison algorithms of our pristine bullet dataset when artificially excluding one or more LEAs from the comparison. This strategy, based on our past experience with IBIS Search, allowed us to develop the Best LEA method in addition to the Best Phase method. The Best LEA method performs best when many LEAs are missing.

The similarity scores, and associated FMR, represent the similarity of the markings present on the compared bullets, whether they were fired from the same firearm or not. If their agreement is weak then the scores will be low and the FMR will represent that reality. The conclusion that is presented in court should be similar to the one that would be provided based on conventional comparison microscopy. As long as the FMR is consistent with the observable similarity, then its value can be reliably assessed.

The adjustments in the Comparison Viewer to enhance the 3D viewing are preserved with the « Screenshots » that can be saved. A Comparison Viewer Screenshot saves the exact viewing state so that you can return to it later and continue from where you left off.

Up to 6 acquisitions can be viewed simultaneously side-by-side in Surface View and in Shape View.

The Quantum 3D Microscope application saves the data and results under the current Workspace folder. These files can be freely accessed with Windows Explorer and can be stored or archived on the customer’s network or on USB removeable media.

The X3P export capability of the Quantum 3D Microscope will make it possible for laboratories to submit X3P data for the NIST Ballistics Toolmark Research Database (NBTRD). We have designed this product with an emphasis on flexible data access and management, and the sharing of data is fundamentally important.

The latest model of IBIS BULLETTRAX (2019) can be re-used (shared) for the Quantum 3D Microscope. Unfortunately, Quantum requires newer technology that is not available in the older models of IBIS BULLETTRAX (2004-2018), which would need to be upgraded. The new BULLETTRAX provides calibration capabilities for measurement traceability. Its wider field of view offers more viewing context, but also allows for new methods like the subclass detection warning feature. The RBL method’s algorithms are trained on 3D data acquired using its non-linear stereo-photometry method, and future development will be based on this new BULLETTRAX data.

The recent BULLETTRAX units already deployed would have to be retrofitted to certify them with the calibration targets required by Quantum. Going forward, BULLETTRAX will include those targets, and will consequently allow dual-purpose use with Quantum.

The Quantum 3D Microscope operates as a standalone instrument. It is not a component of an IBIS Search network.

New IBIS BULLETTRAX acquisition units do not currently (as of June 2019) have the same ISO 17025 certified targets within them. However, there are plans to retrofit these units to install the calibration targets required by Quantum.

The Quantum 3D Microscope software only works with the Quantum or IBIS 3D acquisition units. However, you can purchase additional Application Extensions which consist of software that can run on other computers to perform comparison viewing and quantitative analysis.

Standard training is expected to be up to 5 days, covering acquisition, comparison viewing, quantitative analysis, and workspace and data management.

The Quantum 3D Microscope does not currently include features to directly interface with other systems. However, the data and results are stored in a workspace folder that is easily accessible, so that files can be exchanged with other systems such as a LIMS.