

ORIGINAL ARTICLE 

Year : 2019  Volume
: 5
 Issue : 2  Page : 8794 

Specificity of characteristic marks on cartridge cases from 3070 consecutive firings of a Chinese Norinco QSZ92 9 mm Pistol
Feng Dong^{1}, Yabin Zhao^{2}, Yaping Luo^{3}, Weifang Zhang^{4}, Kaifeng Zhang^{2}
^{1} School of Forensic Science, People's Public Security University of China, Beijing; Shanghai Key Laboratory of Crime Scene Evidence, Shanghai Research Institute of Criminal Science and Technology, Shanghai, China ^{2} School of Forensic Science, People's Public Security University of China, Beijing, China ^{3} Graduate School, People's Public Security University of China, Beijing, China ^{4} Shanghai Key Laboratory of Crime Scene Evidence, Shanghai Research Institute of Criminal Science and Technology, Shanghai, China
Date of Web Publication  26Jun2019 
Correspondence Address: Yaping Luo Graduate School, People's Public Security University of China, No. 1, Muxidi South Street, Xicheng District, Beijing 100038 China
Source of Support: None, Conflict of Interest: None  Check 
DOI: 10.4103/jfsm.jfsm_6_19
The specificity of marks was evaluated by differences between marks left by the same firearm (within variability) and differences between marks left by different firearms (between variability). The within variability and between variability of firing pin impressions and breech face marks on cartridge cases from over 3070 consecutive firings of a Chinese Norinco QSZ92 9 mm pistol were investigated based on the automated ballistic identification system Evofinder^{®}. The first 20 cartridge cases and one random specimen out of every set of 10 consecutive cartridge cases of the remaining 3050 cases were entered into the Evofinder^{®} system. Thus, 325 cartridges fired from the same pistol were used to estimate the within variability. Then, 88 cartridges fired from 88 different pistols of the same model (one cartridge each) were used to estimate the between variability. This study established a database containing 413 cartridges. We used the 325 cartridges fired from the same pistol as a correlation baseline in the database. Both firing pin and breech scores and gaps between these scores were used to analyze the variability. In the score evaluation process, a likelihood ratio system was used to evaluate the likelihood ratio performance. The Evofinder^{®} system correctly discriminated the matches, and the likelihood ratios provided strong support for the true state of the hypothesis. In addition, we found that the score gap was usually greater than the other score differences by more than 1000% in firing pin list and 100% in the breech list. Further analysis of the score and gap can help examiners use the Evofinder^{®} system more efficiently.
Keywords: Automatic correlation, Evofinder^{®}, firearm identification, likelihood ratio
How to cite this article: Dong F, Zhao Y, Luo Y, Zhang W, Zhang K. Specificity of characteristic marks on cartridge cases from 3070 consecutive firings of a Chinese Norinco QSZ92 9 mm Pistol. J Forensic Sci Med 2019;5:8794 
How to cite this URL: Dong F, Zhao Y, Luo Y, Zhang W, Zhang K. Specificity of characteristic marks on cartridge cases from 3070 consecutive firings of a Chinese Norinco QSZ92 9 mm Pistol. J Forensic Sci Med [serial online] 2019 [cited 2019 Sep 17];5:8794. Available from: http://www.jfsmonline.com/text.asp?2019/5/2/87/261533 
Introduction   
Every firearm bears its own fingerprint. The manufacturing process leaves unique signatures on the ejector, firing pin, breech face (BF), extractor, and other components on each firearm. During the firing and ejection processes, these signatures produce a number of marks on cartridge cases that can be used for comparison and case gun matching. The firing pin and BF marks become the specificity of the individual marks on cartridge cases.
Many studies have investigated whether firearm marks change significantly after many rounds are fired from a firearm.^{[1],[2],[3],[4],[5],[6],[7],[8],[9],[10],[11],[12]} Kirk et al.^{[12]} concluded that no significant changes occur to firing pin and BF marks by estimating the changes over 200 consecutive discharges based on an IBIS^{®} Heritage™ system. In general, slight variations have been observed, but it still is possible for firearm examiners to match cartridge cases during consecutive discharges.
In a study by Riva and Champod,^{[13]} a scorebased likelihood ratio (SLR) system was constructed to decrease the subjective component affecting the firearm identification. The SLR method became prevalent in forensic science, as it can deal with some situations where the likelihood ratio is hard to calculate and provide an objective measurement of the evidence source for investigators. A more detailed description of the SLR method can be found in Meuwly study,^{[14]} and this method has been applied or studied in various branches of forensic science, including fingerprinting,^{[15]} firearms,^{[16],[17]} and handwriting.^{[18],[19],[20]}
In articles by Zhang,^{[21],[22]} five pistols, labeled as A, B, C, D, and E, each fired a total of 3070 rounds for each pistol. The first 20 cartridge cases of each pistol were entered into Evofinder^{®}. From the 21^{st} to the 3070^{th} shot, every ten cartridge cases were collected and one random sample of each group was entered. Thus, a database of 325 × 5 = 1625 specimens was established. The first and every 100^{th} specimen of each pistol were used to make correlations against the database. The correlation results exhibited the reproducibility of marks on cartridge cases fired from the same firearm. The limitations stated that correlations were performed only for certain specimens and the gaps (see Section 2.2.3) in Evofinder^{®} correlation results were not fully analyzed.
To further evaluate the specificity of marks and use the score properly, particularly the gap between scores, in this study, the within (same pistol) variability and between (pistol) variability were examined by different methods. An SLR system was constructed using the Evofinder^{®} system, and thereby, the performance of this ratio system on similarity scores was studied. The gap was studied according to its size in established correlation lists and occurrence numbers in each correlation list. The results confirmed the specificity of marks on cartridge cases fired from the selected Chinese Norinco QSZ92 9 mm pistol.
Materials And Methods   
Input of data and correlation
The 3070 cartridge cases fired from pistol A (selected at random, serial number: 012160) were reused in this article. The ammunition used was model DAP92 9 mm ammunition. The first 20 cartridge cases and one random specimen from every set of the next ten consecutive cartridge cases of the remaining 3050 cartridge cases were entered into the Evofinder^{®} system (325 specimens in all). Each specimen was named by a combination of the pistol name and the shot number; for example, A0001 is the case of the first cartridge fired from pistol A and case A1000 is one of the 991^{st}–1000^{th} cartridges fired from pistol A. More detailed descriptions of the case collection and scanning are given in Zhang et al. and Zhang and Luo studies.^{[21],[22]}
In addition, three consecutive rounds were fired from each of 88 additional QSZ92 9 mm pistols. One random specimen out of three cartridge cases for every pistol was chosen, and the selected 88 cartridge cases from different sources were entered into the Evofinder^{®} system by the same parameters with the 325 specimens from pistol A. Thus, a database of 413 (325 + 88) specimens was established.
In the correlation process, we used each specimen of pistol A, selected as described above, to start a correlation in the database. A total of 325 correlation results were produced by the Evofinder^{®} system. Each correlation result contains two separate lists: firing pin impression (FPI) and BF mark. In each list of FPI or BF, the Evofinder^{®} system provided a score that ranged from 0 to 1 for each known matching specimen or known nonmatching specimen. The specimen analysis system (SAS) within Evofinder^{®} applied in this article was the upgraded software version 6.4, versus version 6.3 in.^{[21],[22]}
Statistics analysis of the correlation results
The fitdistrplus, e1071, and other packages within the opensource RStudio software were applied to the statistical processes in this study.
Probability density function
The crucial step for performing the kernel density estimation was the appropriate determination of the smoothing parameter. According to the default method in the kernel density function of RStudio software, the determination of smoothing parameter was given by Silverman.^{[23]} We used the standard parameters in RStudio software to establish the probability density function.
Description of the scorebased likelihood ratio method
We formulated two firearms hypotheses at the source level:
 Prosecution hypothesis (usually denoted as H_{p}): The cartridge case in question was fired from pistol A
 Defense hypothesis (usually denoted as H_{d}): The questioned cartridge case was not fired from the pistol A but by another Norinco QSZ92 9 mm pistol.
The SLR is based on the ratio of probabilities under these two opposite hypotheses. Different from the calculation of the likelihood ratio method, the SLR method uses distances or similarities to derive a score in a pairwise correlation. The calculation equation can be interpreted by:
Where S_{p} and S_{d} are the scores collected from within and between distributions, respectively; I is the background information; and f is a probability density function.
To form the within distribution, the mutual correlations between 325 cartridge cases fired from pistol A were selected from 325 lists. The Evofinder^{®} comparison system missed some correlations when running the SAS, so the practical correlations were less than the theoretical correlations. The number of theoretical correlations is 52,650 (325× [325 − 1]/2), whereas the number of practically observed correlations is 52,557 within the distribution. For between distribution, the correlations between 325 cartridge cases fired from pistol A and 88 cartridge cases fired from 88 different pistols were selected from 325 lists. The number of theoretical correlations is 28,600 (325 × 88), whereas the number of practically observed correlations is 28,566. From these data, we obtained 52,557 SLRs when it is known that H_{p} is true and conversely 28,566 SLRs when it is known that H_{d} is true. In this article, the FPI and BF mark were studied to calculate the SLRs.
To judge the goodness of fit for FPI and BF mark scores, the fitdistrplus package in RStudio software was used. The rates of misleading evidence of H_{d} (RMED) and the rates of misleading evidence of H_{p} (RMEP) were reported to illuminate the performance of the SLR.
Analysis of gap
A gap is a difference between two consecutive match list scores that is exceptionally greater than the other score differences.^{[24]} In a practical examination, only a few matching cartridge cases would be entered into the reference ballistic imaging database (RBID). If no matching cartridges were entered into the RBID, the gap would usually be the maximum score difference between two consecutive nonmatching scores. If a matching cartridge exists in the RBID, the gap would generally be the difference between the matching score and the maximum score of nonmatching specimens. We mainly focused on the latter gap because its existence and variation exhibited mark specificity. Under these circumstances, we analyzed the gap from the aspect of number of occurrences, which reflects the within variability and the aspect of size, which reflects the between variability.
In each list, we looked for the maximum scores of known nonmatching specimens. The value of a was derived by every score of known matching specimen minus the maximum score of known nonmatching specimens. Then, we deleted the scores of known matching specimens in each list and calculated the maximum score difference between two consecutive scores of known nonmatching specimens.
The number of occurrences was used to evaluate how many known matching specimens in each FPI or BF list have a gap that is obviously greater than the other score differences. If 10% was selected as a requirement for gaps,^{[24]} we calculated the occurrences of known matching specimens that have a gap in each list by determining whether a was 10% greater than the maximum score difference between two consecutive scores of known nonmatching specimens.
The size was used to evaluate whether a was large enough to let investigators find a match quickly. We calculated the size relationship of a and the maximum score difference between two consecutive scores of known nonmatching specimens. To accomplish this, we chose every 100^{th} cartridge case, that is, 30 cartridge cases for pistol A were used to start a correlation. The number of known matching specimens was calculated when a was 10%, 50%, 100%, 500%, and 1000% greater than the maximum score difference between two consecutive scores of known nonmatching specimens.
Results and Discussion   
Density estimates of within distribution and between distribution
[Figure 1] shows the probability density functions of FPI and BF mark scores. For FPI scores, results show a better separation than the BF mark scores between the within and between distributions, with a limited overlap between the two. In addition, the within distribution of FPI densities is slightly left skewed, whereas the within distribution of BF mark densities is right skewed. Some descriptive statistics of the data are summarized in [Table 1].  Figure 1: Density distribution of the similarity scores for firing pin impression (red color) and breech face mark (green color). Within distribution (52,557 correlations) represents the known matching specimens and between distribution (28,566 correlations) represents the known nonmatching specimens
Click here to view 
 Table 1: Descriptive statistics for the similarity scores of within distribution and between distribution
Click here to view 
The large range of the within distribution proves the within variability, which means that the firing pin and BF marks on cartridge cases fired from the same firearm have variations. The separation of the within and between distributions proves the between variability, which means that the FPIs and BF marks on cartridge cases varied from different firearms. The scores of the within distribution are closer to 1 than the between distribution, so we can also determine that the degree of within variability is less than that of the between variability.
The FPI and BF mark scores from the within and between distributions were used to calculate the area under the receiver operating characteristic curve (AUC). The AUC value of the BF mark scores was 0.9826, whereas the AUC value of firing pin scores was 0.9998. These results confirm that the Evofinder^{®} system can correctly distinguish the known matching specimens and known nonmatching specimens.
Score evaluation with the scorebased likelihood ratio method
Scores of firing pin impression
We used the Cullen and Frey graph and quantilequantile plot in RStudio software, and we assumed a Gaussian distribution for FPI scores in the within and between distributions. After appropriate data fitting, the performance of the SLR could be weighted by a Tippett plot for FPI scores [Figure 2].  Figure 2: Tippett plot for the performance of scorebased likelihood ratio for firing pin scores
Click here to view 
Scores of breech face mark
The distributions for BF mark scores in the within and between distributions were fitted by kernel density estimator, as the distribution of scores cannot be fitted well by a common family distribution. [Figure 3] shows the Tippett plot for BF mark scores. We note that the trueH_{p} curve is nearly a straight line at the right end of the horizontal axis; this is because some null numbers generated from the kernel density estimation of the within and between distributions that reflect the SLR are infinite.  Figure 3: Tippett plot for the performance of scorebased likelihood ratio for breech face scores (scorebased likelihood ratio at 1010 is cutted)
Click here to view 
First conclusions
The rates of misleading evidence exhibit the good performance of FPI and BF mark scores, especially for FPI scores. In [Figure 2] and [Figure 3], the trueH_{p} curve and trueH_{d} curve have large vertical separation^{[25]} which also reflects the excellent discriminating power of the Evofinder^{®} system for Norinco cartridge cases.
The range of SLRs with the corresponding span of scores is summarized in [Table 2]. The SLRs mainly concentrate in the range of 1000 to infinity for FPI scores (99.8%) and concentrate in the range of 10 to infinity for BF mark scores (85.2%). According to the verbal equivalents of the calculated likelihood ratio values in Evett et al. study,^{[26]} the SLRs calculated from the firing pin scores and BF mark scores can give a very strong support of the true state hypothesis.  Table 2: Ranges of scores and correlations are given the range of scores in the within distribution (52,557 correlations) for various scorebased likelihood ratio ranges
Click here to view 
In [Figure 4], we combine the FPI and BF mark scores for each known matching and nonmatching specimens. Obviously, the within distribution is again separated from the between distribution. The known matching scores form two clusters, with a gap between them at an FPI score of around 0.22. In addition, the scatter plot of known matching scores looks like a right trapezoid. This may be attributed to the algorithm within the SAS, which acts as a “black box” for investigators.  Figure 4: Scatter plot representing the firing pin score and breech face score for each known matching specimen (within distribution, indicated by green circles) and known nonmatching specimens (between distribution, indicated by red circles)
Click here to view 
Based on the combined scores, we applied a support vector machine to calculate the error rates. We chose the Cclassification type and the radial basis function as the parameters in RStudio software. The classification results are shown in [Table 3]. It was found that the accuracy rates are 0.9998 using these parameters. Meanwhile, the specificity and sensitivity were calculated based on the classification result: the specificity was 28,564/(28,564 + 2)≈0.99993, and the sensitivity was 52,543/(52,543 + 14)≈0.99973.  Table 3: Classification results of the combined scores obtained with a support vector machine
Click here to view 
Evaluation of the gap
The existence of the gap reflects that the FPIs and BF marks on cartridge cases fired from different firearms have variations. The gap variation between known matching and nonmatching scores reflects that the FPI and BF marks on cartridge cases fired from the same firearm have slight variations. Therefore, the gap is a good metric for evaluating the within variability and between variability.
Analysis of gap with number of occurrences
Gap of firing pin scores
In the 325 FPI correlation lists, there are 308 lists in which all the known matching specimens have a gap of firing pin score. In the other 17 lists, at least 1 known matching specimen in each list does not meet the 10% requirement that defines a gap. The specimens under examination of 17 lists and occurrences of known matching specimens that have a gap in each list are shown in [Table 4]. From [Table 4], 12 lists occur after specimen A2000. This reflects that the most easily distinguished FPIs may occur up to 2000 firings for the Norinco QSZ92 9 mm pistol.  Table 4: Corresponding to the specimens under examination (at least one known matching specimen in these lists did not meet the 10% requirement to have a gap), the occurrences of known matching specimens that have a gap in each firing pin impression list
Click here to view 
The gap of breech face score
There was no BF list in which all the known matching specimens meet the 10% requirement to have a gap in the BF mark score. Especially, when A0060 was under examination, only one known matching specimen in this BF list had a gap. We also found an interesting relationship between the numbers of occurrences and shots fired [Figure 5]. The occurrences gradually increase from A0001 until A0500, after which they approach 324 and fluctuate stably around 324. This reflects that the most easily distinguished BF marks may range up to 500 shots for the Norinco QSZ92 9 mm pistol.  Figure 5: Number of occurrences as a function of shot number from 1 to 3070 shots
Click here to view 
Analysis of gap with gap size relationship
Gap of firing pin score
We chose some correlation lists to analyze the size relationship of a and the maximum score difference between two consecutive scores of known nonmatching specimens. The results [Table 5] show that the gap size is usually greater than the other score differences, more than approximately 1000% in the FPI list.  Table 5: Size relationship of a and the maximum score difference between consecutive nonmatching firing pin impression scores when some specimens (e.g., A0001 and A0100) were correlated against the database
Click here to view 
Gap of breech face score
The results for size relationship of a and the maximum score difference between two consecutive scores of known nonmatching specimens are summarized in [Table 6]. Except when A0001, A0100, and A2400 were under examination, the size of the gap was usually greater than the other score differences more than approximately 100% in BF list.  Table 6: Size relationship of a and the maximum score differences between two consecutive nonmatching breech face mark scores when some specimens (e.g., A0001 and A0100) were correlated against the database
Click here to view 
Limitations
The whole theory and application on the gap are not yet scientifically sound. As more specimens are entered into the RBID, the gap becomes more arbitrary. However, the gap is also a good metric for practical work. In this article, we only focused on the gap between the known matching and nonmatching scores because it is more useful than the pure scores to prove the between variability.
Only 88 different Norinco QSZ92 9 mm pistols were fired to form the between distribution. Meanwhile, the influence of the gun user is also a limitation. In future work, the influence of the user should be considered and analyzed.
The current study focused on one randomly selected Norinco QSZ92 9 mm pistol, and therefore, some results shown in this article are not enough to generalize to the type of weapon.
Conclusions   
The Evofinder^{®} system, as a secondgeneration electronic comparison system, offers an exceptionally discriminating power for a randomly selected Norinco QSZ92 9 mm pistol. The Tippett plots and AUC value strengthen this conclusion. The overlap of the within distribution and between distribution for FPI and BF mark scores is small. In addition, the SLRs can provide strong support for the true state of the hypothesis.
The within variability and between variability of marks on cartridge cases over 3070 consecutive firings from a Norinco QSZ92 9 mm pistol were evaluated by the Evofinder^{®} system. The gap (difference <10%) reflects that there exists slight variation between consecutive discharges, which is the within variability, but this difference is obviously lower than the variation between the cartridge cases fired from different firearms, which is the between variability. The number of gap occurrences suggests that the most easily distinguished FPIs may be produced up to 2000 firings, whereas the most easily distinguished BF marks may be produced after 500 firings for the Norinco QSZ92 9 mm pistol. The gap size is usually greater than the other score differences by more than approximately 1000% in the FPI list and 100% in the BF list. Firearms investigators could use the gap of BF mark and FPI scores to find a hit for match, but more factors must be considered if there is no gap in the BF mark score.
Financial support and sponsorship
This work has received funding from the Opening Project of Shanghai Key Laboratory of Crime Scene Evidence (2018XCWZK18).
Conflicts of interest
There are no conflicts of interest.
References   
1.  Hamby J. Identification of projectiles. AFTE J 1974;6:22. 
2.  Shem R, Striupaitis P. Comparison of 501 consecutively fired bullets and cartridge cases from a 25 caliber Raven pistol. AFTE J 1983;15:10912. 
3.  Kirby S. Comparison of 900 consecutively fired bullets and cartridge cases from a 455 caliber S&W revolver. AFTE J 1983;15:11326. 
4.  Ogihara Y, Kubota M, Sanada M, Fukuda K, Uchiyama T, Hamby J. Comparison of 5000 consecutively fired bullets and cartridge cases from a 45 caliber M1911A1 pistol. AFTE J 1989;21:33143. 
5.  Bonfanti MS, De Kinder J. The influence of the use of firearms on their characteristic marks. AFTE J 1999;31:31823. 
6.  Vinci F, Campobasso CP, Bailey JA. Morphological study of class and individual characteristics produced by firing 2500 cartridges in a 45 caliber semiautomatic pistol. AFTE J 2005;37:368. 
7.  Gouwe J, Hamby JE, Norris SA. Comparison of 10,000 consecutively fired cartridge cases from a model 22 Glock. 40 S&W caliber semiautomatic pistol. AFTE J 2008;40:57. 
8.  Saribey AY, Hannam AG, Tarimci C. An investigation into whether or not the class and individual characteristics of five Turkish manufactured pistols change during extensive firing. J Forensic Sci 2009;54:106872. 
9.  Mikko D, Miller J, Flater J. Reproducibility of tool marks on 20,000 bullets fired through an M240 machine gun barrel. AFTE J 2012;44:24853. 
10.  Grom TL, Demuth WE. IBIS correlation results of cartridge cases collected over the course of 500 firings from a Glock pistol. AFTE J 2012;44:3613. 
11.  Wong C. The intercomparison of 1,000 consecutivelyfired 9mm luger bullets and cartridge cases from a Ruger P89 pistol utilizing both pattern matching and quantitative consecutive matching striae as criteria for identification. AFTE J 2013;45:26772. 
12.  Kirk JN, Law EF, Morris KB. Estimation of changes in breech face and firing pin marks over consecutive discharges and its impact on an IBIS® heritage™ system. Forensic Sci Int 2017;278:4751. 
13.  Riva F, Champod C. Automatic comparison and evaluation of impressions left by a firearm on fired cartridge cases. J Forensic Sci 2014;59:63747. 
14.  Meuwly D. Forensic individualisation from biometric data. Sci Justice 2006;46:20513. 
15.  Leegwater AJ, Meuwly D, Sjerps M, Vergeer P, Alberink I. Performance study of a scorebased likelihood ratio system for forensic fingermark comparison. J Forensic Sci 2017;62:62640. 
16.  Song J, Vorburger TV, Chu W, Yen J, Soons JA, Ott DB, et al. Estimating error rates for firearm evidence identifications in forensic science. Forensic Sci Int 2018;284:1532. 
17.  Riva F, Hermsen R, Mattijssen E, Pieper P, Champod C. Objective evaluation of subclass characteristics on breech face marks. J Forensic Sci 2017;62:41722. 
18.  Davis LJ, Saunders CP, Hepler A, Buscaglia J. Using subsampling to estimate the strength of handwriting evidence via scorebased likelihood ratios. Forensic Sci Int 2012;216:14657. 
19.  Hepler AB, Saunders CP, Davis LJ, Buscaglia J. Scorebased likelihood ratios for handwriting evidence. Forensic Sci Int 2012;219:12940. 
20.  Chen XH, Champod C, Yang X, Shi SP, Luo YW, Wang N, et al. Assessment of signature handwriting evidence via scorebased likelihood ratio based on comparative measurement of relevant dynamic features. Forensic Sci Int 2018;282:10110. 
21.  Zhang K, Luo Y, Zhou P. Reproducibility of characteristic marks on fired cartridge cases from five Chinese Norinco QSZ92 9×19mm pistols. Forensic Sci Int 2017;278:7886. 
22.  Zhang K, Luo Y. Slight variations of breech face marks and firing pin impressions over 3070 consecutive firings evaluated by evofinder®. Forensic Sci Int 2018;283:8593. 
23.  Silverman BW. Density estimation for statistics and data analysis. Florida: CRC press; 1998. 
24.  Rahm J. Evaluation of an electronic comparison system and implementation of a quantitative effectiveness criterion. Forensic Sci Int 2012;214:1737. 
25.  Zadora G, Martyna A, Ramos D, Aitken C. Statistical Analysis in Forensic Science: Evidential Value of Multivariate Physicochemical Data. Chichester: John Wiley & Sons; 2014. p. 3738. 
26.  Evett IW, Jackson G, Lambert JA, McCrossan S. The impact of the principles of evidence interpretation on the structure and content of statements. Sci Justice 2000;40:2339. 
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]
