autoemx.core.fitter.peak_identification module

Automatic element identification

Identifies the elements present in an X-ray spectrum, starting from a (possibly empty) list of elements known to be present.

Candidates are generated from the X-ray line table and verified through full-spectrum fitting, starting from the most intense unexplained peak:

1. Fit the spectrum with the elements already in the model (fixed, substrate and identified elements).

2. Search the fit residual for significant peaks with a top-hat filter, which suppresses any locally linear mismatch of the background (Statham, Anal. Chem. 49 (1977) 2149).

3. For the most intense unexplained peak, list the elements with a strong line within the detector resolution, plus their known overlap partners (common_peak_overlaps.json).

4. Screen the candidates with a filtered weighted least-squares fit of the templates of their line families to the residual. Lines of a family have fixed relative weights, so a candidate whose other lines are not observed is penalised. If a higher-energy family of the candidate is well excited, it must be observed too (confirmation lines; Newbury, Microsc. Microanal. 11 (2005) 545).

5. Verify the best candidates with a full refit. A candidate is accepted if it decreases the chi-square of the fit (Poisson variance plus a relative model uncertainty) by more than a threshold, and if the fitted area of its line exceeds the Currie detection limit L_D = 2.71 + 3.29 sqrt(B) (Currie, Anal. Chem. 40 (1968) 586).

6. Repeat until no significant peak is left; then remove identified elements that the final fit does not need (backward pruning).

This is the iterative qualitative-quantitative strategy of Newbury & Ritchie, Microsc. Microanal. 24 (2018) 350. Statistical tests on nested fits are not strictly valid when the null hypothesis is a peak area of zero (Protassov et al., ApJ 571 (2002) 545), so all thresholds are empirical.

The fitting is delegated to a callable (see make_xsp_fit_function), so the identification loop is independent of the fitter.

class autoemx.core.fitter.peak_identification.PeakIDSettings(min_peak_significance: float = 5.0, low_energy_threshold_keV: float = 1.0, low_energy_multiplier: float = 1.5, element_multipliers: ~typing.Dict[str, float] = <factory>, artifact_min_fraction: ~typing.Dict[str, float] = <factory>, candidate_window_sigma: float = 2.5, merged_peak_shift_keV: float = 0.025, min_line_weight: float = 0.1, confirm_overvoltage: float | None = None, confirm_significance: float = 3.0, model_rel_uncertainty: float = 0.01, min_peak_gain: float = 9.0, min_supported_peaks_overlap_check: int = 2, min_delta_deviance: float = 100.0, ambiguity_margin: float = 30.0, max_full_fits_per_peak: int = 4, max_new_elements: int = 10, max_iterations: int = 30, z_range: ~typing.Tuple[int, int] = (5, 83), excluded_elements: ~typing.Tuple[str, ...] = ('Tc', 'Pm', 'Ne', 'Kr', 'Xe', 'Rn'), prune: bool = True, max_start_redchi_fraction: float | None = None, strong_min_filtered_gain: float = 100.0, background_ratio_bounds: ~typing.Tuple[float, float] = (0.8, 1.25), background_bands_keV: ~typing.Tuple[~typing.Tuple[float, float], ...] = ((1.0, 2.5), (2.5, 5.0), (5.0, 100.0)), min_local_gain_fraction: float = 0.2, contradiction_rel_loss: float = 0.05, linear_prefilter_fraction: float = 0.25, full_fit_overlaps: bool = False, overlap_check: bool = True, refit_after_accept: bool = False, final_fit: bool = False)

Bases: object

Thresholds of the element identification.

Variables:

• min_peak_significance (float) – Minimum significance (in standard deviations) of a residual peak, and of the candidate line family that explains it.

• low_energy_multiplier (low_energy_threshold_keV,) – Thresholds are multiplied by low_energy_multiplier for lines below low_energy_threshold_keV, where absorption edges and chemical effects leave structure in the residual.

• element_multipliers (dict) – Threshold multipliers for specific elements (none by default).

• artifact_min_fraction (dict) – Elements also produced by the detector (Si: internal fluorescence of the Si detector, ~0.05% of the counts): they are kept only if the fitted area of their main line exceeds this fraction of the total counts.

• candidate_window_sigma (float) – Lines within this many detector sigmas of a residual peak are considered as candidates.

• merged_peak_shift_keV (float) – Extra width of the candidate window, accounting for the shift of the maximum of unresolved La+Lb and Ma+Mb peaks from the tabulated energies.

• min_line_weight (float) – Minimum weight of a line (relative to the reference line of its family) to be used to match a peak.

• confirm_significance (confirm_overvoltage,) – A higher-energy family with overvoltage above confirm_overvoltage (default: the overvoltage used by XSp_Quantifier to select ideal reference lines, ideal_ref_line_overvoltage) must be observed with at least confirm_significance standard deviations for a candidate to be accepted.

• model_rel_uncertainty (float) – Relative uncertainty of the spectrum model (peak shapes, line weights, background), added to the Poisson variance as (f·model)². Systematic model errors grow with the counts while Poisson noise grows with their square root, so without this term every small model imperfection becomes significant in high-count (e.g. summed) spectra.

• min_peak_gain (float) – Local fit improvement (decrease of the chi-square within a resolved peak of a candidate) for the peak to support the candidate; a decrease of the same amount with the opposite sign makes the peak contradict it. A candidate is rejected if any of its resolved peaks predicted above the detection limit contradicts it (a peak predicted where there is none), whatever the total fit improvement.

• min_supported_peaks_overlap_check (int) – Minimum number of resolved peaks supporting a candidate found by the overlap check (an element absorbed by overlapping peaks must improve the fit at several of its peaks to be told apart from model mismatch).

• min_delta_deviance (float) – Minimum decrease of the fit statistic Σ (y − μ)² / (μ + (f·μ)²) for a candidate to be accepted. In the full-fit pruning, an element is also kept if its filtered (peak) gain reaches this value while the total statistic still decreases.

• ambiguity_margin (float) – Candidates whose decrease of the fit statistic is within this margin of the best one are reported as ambiguous.

• max_full_fits_per_peak (int) – Maximum number of screened candidates verified with a full fit, when full fits are needed (overlaps with elements in the model). Otherwise all screened candidates are tested with fast linear fits.

• max_new_elements (int) – Maximum number of identified elements.

• max_iterations (int) – Maximum number of residual peaks examined.

• z_range ((int, int)) – Atomic numbers of the elements proposed as candidates (default B to Bi: all elements above Bi are radioactive and essentially absent from samples; use (5, 92) to include Th and U).

• excluded_elements (tuple of str) – Elements never proposed as candidates (default: Tc and Pm, without stable isotopes, and the noble gases Ne, Kr, Xe, Rn, absent from solids; Ar is allowed, as it can be implanted by ion milling or sputtering).

• strong_min_filtered_gain (float) – Identified elements are classified as ‘strong’ (to be added to the fit and quantified) if (1) in the full fit without them, at least one of their resolved peaks is a clear positive peak of the filtered residual (min_peak_significance, with the threshold multipliers), (2) the chi-square of the top-hat filtered residual (insensitive to smooth background changes) improves by at least strong_min_filtered_gain when they are added, (3) the background is reliable, and (4) no other candidate explains the peak equally well. Otherwise they are ‘possible’ (only reported, for manual checking): e.g. heavy elements whose many weak lines fill a background or absorption mismatch, without a peak of their own in the residual.

• background_bands_keV (background_ratio_bounds,) – Background health: ratio of data to model in each energy band, in channels without significant peaks (excluding fitted peaks and unexplained peaks of the residual). Bands are checked separately, so that a wrong background shape (deficit in one band, excess in another) is not averaged out. If a band is outside the bounds in the starting fit, the fit is repeated with the background scale K fixed (see background_refit_function of identify_elements), and the refit is kept if it reduces the worst band deviation without creating a background deficit that the starting fit did not have (a model above the data is harmless, e.g. when a missing major element changes the background composition). If the data still exceed the model by more than the upper bound in a band (background deficit, which any element could fill), also after adding the identified elements (a missing major element changes the background too), all identified elements are ‘possible’.

• max_start_redchi_fraction (float or None) – If set, the identification is skipped if the reduced chi-square of the starting fit exceeds this fraction of the total counts (0.0016 is the criterion of XSp_Quantifier for unreliable fits, quant flag 1). Off by default: a missing element also makes the starting fit poor.

• linear_prefilter_fraction (float) – For candidates overlapping peaks of the current fit, the linear test underestimates the fit improvement (frozen shapes of the overlapping peaks have partly absorbed the candidate), so it is only used as a pre-filter at this fraction of min_delta_deviance; the full fit then applies the threshold.

• contradiction_rel_loss (float) – A predicted peak contradicts a candidate only if the local fit worsens by more than this fraction of the candidate’s total fit improvement (and by more than min_peak_gain), so that small mismatches of weak lines (line weights) do not reject an element with strong evidence elsewhere.

• min_local_gain_fraction (float) – Minimum fraction of the fit improvement brought by a candidate that must come from peak regions (where the candidate or other elements have significant peaks), rather than from the background. Rejects elements that improve the fit by changing the background (e.g. a heavy element rescuing a fit stuck with too little background), rather than by explaining peaks.

• prune (bool) – Whether to remove identified elements that the final fit does not need. Uses full fits (one with all identified elements, one without each of them), run only if elements were identified.

• overlap_check (bool) – Whether to test the overlap partners (overlap table) of the elements in the model, even without residual peaks: an element whose peaks are absorbed by overlapping peaks of other elements leaves no residual peak, but still improves the fit. Partners are tested with linear fits (tied shifts); the best one passing the thresholds is confirmed with a full fit, and the check is repeated until none is confirmed.

• full_fit_overlaps (bool) – Whether candidates overlapping peaks of the current fit are tested with full fits instead of linear fits (in both cases, with energy shifts tied to the overlapping peaks).

• refit_after_accept (bool) – Whether to confirm each element accepted with a linear test by a full fit, so that the next residual peaks are searched with the fitter’s peak shapes. Needed for high-count spectra, where the mismatch of the linear templates becomes significant; off by default for single spectra.

• final_fit (bool) – Whether the final model must come from a full fit. Off by default, since the caller refits anyway with the identified elements (e.g. quantification after the identification of missing elements).

min_peak_significance: float = 5.0

low_energy_threshold_keV: float = 1.0

low_energy_multiplier: float = 1.5

element_multipliers: Dict[str, float]

artifact_min_fraction: Dict[str, float]

candidate_window_sigma: float = 2.5

merged_peak_shift_keV: float = 0.025

min_line_weight: float = 0.1

confirm_overvoltage: float | None = None

confirm_significance: float = 3.0

model_rel_uncertainty: float = 0.01

min_peak_gain: float = 9.0

min_supported_peaks_overlap_check: int = 2

min_delta_deviance: float = 100.0

ambiguity_margin: float = 30.0

max_full_fits_per_peak: int = 4

max_new_elements: int = 10

max_iterations: int = 30

z_range: Tuple[int, int] = (5, 83)

excluded_elements: Tuple[str, ...] = ('Tc', 'Pm', 'Ne', 'Kr', 'Xe', 'Rn')

prune: bool = True

max_start_redchi_fraction: float | None = None

strong_min_filtered_gain: float = 100.0

background_ratio_bounds: Tuple[float, float] = (0.8, 1.25)

background_bands_keV: Tuple[Tuple[float, float], ...] = ((1.0, 2.5), (2.5, 5.0), (5.0, 100.0))

min_local_gain_fraction: float = 0.2

contradiction_rel_loss: float = 0.05

linear_prefilter_fraction: float = 0.25

full_fit_overlaps: bool = False

overlap_check: bool = True

refit_after_accept: bool = False

final_fit: bool = False

multiplier(el: str, energy_keV: float) → float

Threshold multiplier for a line of an element.

class autoemx.core.fitter.peak_identification.FitOutcome(elements: ~typing.Tuple[str, ...], model: ~numpy.ndarray, deviance: float, line_areas: ~typing.Dict[str, float], components: ~typing.Dict[str, ~numpy.ndarray] = <factory>, background: ~numpy.ndarray | None = None, approximate: bool = False, line_centers: ~typing.Dict[str, float] = <factory>, redchi: float | None = None)

Bases: object

Result of a fit with a given set of elements, as returned by the fit function.

elements: Tuple[str, ...]

model: ndarray

deviance: float

line_areas: Dict[str, float]

components: Dict[str, ndarray]

background: ndarray | None = None

approximate: bool = False

line_centers: Dict[str, float]

redchi: float | None = None

class autoemx.core.fitter.peak_identification.IdentifiedElement(element: str, status: str, matched_line: str | None = None, line_energy_keV: float | None = None, peak_significance: float | None = None, delta_deviance: float | None = None, net_area: float | None = None, detection_limit: float | None = None, ambiguous_with: List[str] = <factory>, overlaps_with: List[str] = <factory>, has_standard: bool | None = None, from_pool: bool = False, via_overlap_check: bool = False, from_peak_assignment: bool = False, filtered_gain: float | None = None, residual_peak_sigma: float | None = None, residual_peak_ok: bool | None = None, note: str = '', tier: str = '')

Bases: object

An element of the final model, with the evidence for its presence.

element: str

status: str

matched_line: str | None = None

line_energy_keV: float | None = None

peak_significance: float | None = None

delta_deviance: float | None = None

net_area: float | None = None

detection_limit: float | None = None

ambiguous_with: List[str]

overlaps_with: List[str]

has_standard: bool | None = None

from_pool: bool = False

via_overlap_check: bool = False

from_peak_assignment: bool = False

filtered_gain: float | None = None

residual_peak_sigma: float | None = None

residual_peak_ok: bool | None = None

note: str = ''

tier: str = ''

describe() → str

class autoemx.core.fitter.peak_identification.UnexplainedPeak(energy_keV: float, significance: float, net_area: float, note: str = '')

Bases: object

A significant residual peak that no candidate element could explain.

energy_keV: float

significance: float

net_area: float

note: str = ''

class autoemx.core.fitter.peak_identification.ElementIDResult(fixed_elements: List[str], substrate_elements: List[str], elements: List[IdentifiedElement], unexplained_peaks: List[UnexplainedPeak], energy_vals: ndarray, spectrum_vals: ndarray, model: ndarray | None, n_fits: int, elapsed_s: float, n_candidate_tests: int = 0, skipped_reason: str = '', background_note: str = '')

Bases: object

Result of the element identification.

fixed_elements: List[str]

substrate_elements: List[str]

elements: List[IdentifiedElement]

unexplained_peaks: List[UnexplainedPeak]

energy_vals: ndarray

spectrum_vals: ndarray

model: ndarray | None

n_fits: int

elapsed_s: float

n_candidate_tests: int = 0

skipped_reason: str = ''

background_note: str = ''

property new_elements: List[str]

Identified elements (including ambiguous identifications), in order of identification.

property strong_new_elements: List[str]

Identified elements with strong evidence, to be added to the fit.

property possible_new_elements: List[str]

Identified elements with weak evidence, only reported as possibly present.

property quantifiable_new_elements: List[str]

Strong elements that can be quantified (unknown standards are assumed available).

property unquantifiable_new_elements: List[str]

fitted and reported as present, but not quantified.

Type:

Strong elements without a standard

summary() → str

plot(ax=None, title: str = '')

Plot spectrum, final model, residual and identified elements.

autoemx.core.fitter.peak_identification.get_sigma_function(microscope_ID: str) → Callable

Detector Gaussian sigma (keV) as a function of energy (keV), for a microscope.

autoemx.core.fitter.peak_identification.element_line_families(el: str, beam_energy: float, energy_range: Tuple[float, float]) → Dict[str, Tuple[Tuple[str, float, float], ...]]

Public accessor of the fitted line families of an element (see _element_families).

autoemx.core.fitter.peak_identification.top_hat_matrix(energy_vals: ndarray, sigma_fn: Callable) → ndarray

Top-hat filter as a matrix (filtered = F @ spectrum), with width following the detector resolution.

Each row has a positive central lobe about one FWHM wide and two negative side lobes of half that width, summing to zero, so that any background that is linear across the filter is removed. Rows whose filter does not fit within the spectrum are zero.

References: F. H. Schamber, in X-Ray Fluorescence Analysis of Environmental Samples, T. G. Dzubay (ed.), Ann Arbor Science (1977) 241; P. J. Statham, Anal. Chem. 49 (1977) 2149.

autoemx.core.fitter.peak_identification.gaussian_template(energy_vals: ndarray, lines: Iterable[Tuple[float, float]], sigma_fn: Callable) → ndarray

Sum of Gaussian peaks of area weight (counts) at the given (energy, weight) pairs.

autoemx.core.fitter.peak_identification.fit_statistic(y: ndarray, mu: ndarray, rel_uncertainty: float) → float

Chi-square of a fit, with Poisson variance plus a relative model uncertainty: Σ (y − μ)² / (μ + (f·μ)²).

autoemx.core.fitter.peak_identification.poisson_deviance(y: ndarray, mu: ndarray) → float

Poisson deviance 2 Σ [y ln(y/μ) − (y − μ)].

class autoemx.core.fitter.peak_identification.ResidualPeak(channel: 'int', energy_keV: 'float', significance: 'float', net_area: 'float')

Bases: object

channel: int

energy_keV: float

significance: float

net_area: float

autoemx.core.fitter.peak_identification.filtered_gain(y: ndarray, model_without: ndarray, model_with: ndarray, F: ndarray, var: ndarray) → float

Chi-square improvement of the top-hat filtered residual: fit improvement of peaks, not of the background.

autoemx.core.fitter.peak_identification.background_band_ratios(y: ndarray, energy_vals: ndarray, outcome: FitOutcome, bands: Sequence[Tuple[float, float]], exclude: ndarray | None = None, min_model_counts: float = 50.0) → List[Tuple[Tuple[float, float], float]]

Ratio of data to model in each energy band, in channels without significant peaks (excluded with exclude). Bands whose peak-free model counts are below min_model_counts are skipped.

autoemx.core.fitter.peak_identification.make_xsp_fit_function(spectrum_vals: ndarray, energy_vals: ndarray, spectrum_lims: Tuple[int, int], microscope_ID: str, meas_mode: str, det_ch_offset: float, det_ch_width: float, beam_energy: float, emergence_angle: float, tot_sp_counts: float, substrate_elements: Sequence[str] = (), is_particle: bool = False, sp_collection_time: float | None = None, fit_tolerance: float = 0.001, initial_par_vals: Dict[str, float] | None = None, fixed_par_vals: Dict[str, float] | None = None) → Callable[[Sequence[str]], FitOutcome]

Fit function for identify_elements, using XSp_Fitter.

spectrum_vals and energy_vals are the values within the spectrum limits, while tot_sp_counts must be the total counts of the full spectrum, as in XSp_Quantifier: the fitter derives the initial background scale from it, and the background fit is sensitive to this starting value. The returned callable takes the sample elements (fitted as elements to quantify; substrate elements are added) and returns a FitOutcome. Results are cached by element set.

initial_par_vals are passed to XSp_Fitter.fit_spectrum, e.g. the starting background scale {‘K’: XSp_Quantifier.get_starting_K_val()} for particles, as in quantification (without it, particle geometry parameters may compensate for the background intensity and the fit may end with too little background). fixed_par_vals are set and not varied, e.g. {‘K’: K} to keep the background scale at the value fitted above 5 keV, when the particle geometry parameters drift the background away from the data.

autoemx.core.fitter.peak_identification.identify_elements(spectrum_vals: ndarray, energy_vals: ndarray, fit_function: Callable[[Sequence[str]], FitOutcome], beam_energy: float, sigma_fn: Callable, fixed_elements: Sequence[str] = (), substrate_elements: Sequence[str] = (), settings: PeakIDSettings | None = None, has_standard: Callable[[str], bool] | None = None, excluded_elements: Iterable[str] = (), initial_outcome: FitOutcome | None = None, priority_elements: Iterable[str] = (), background_refit_function: Callable[[Sequence[str]], FitOutcome] | None = None, meas_type: str = 'EDS', verbose: bool = False) → ElementIDResult

Identify the elements present in a spectrum, in addition to the fixed and substrate elements.

Parameters:

• spectrum_vals (array) – Spectrum counts and energies (keV), within the fitted spectrum limits.

• energy_vals (array) – Spectrum counts and energies (keV), within the fitted spectrum limits.

• fit_function (callable) – Takes a list of sample elements and returns a FitOutcome of the spectrum fitted with those elements (substrate elements are handled by the fit function). See make_xsp_fit_function.

• beam_energy (float) – Beam energy (keV).

• sigma_fn (callable) – Detector sigma (keV) as a function of energy (keV). See get_sigma_function.

• fixed_elements (list of str) – Elements known to be present. May be empty: the peaks of the raw spectrum are then assigned first (see _assign_peaks), and the starting fit includes the assigned elements, which are verified like the other identified elements.

• substrate_elements (list of str) – Substrate elements, always part of the model. Their peaks cannot be attributed to the sample.

• settings (PeakIDSettings, optional) – Identification thresholds.

• has_standard (callable, optional) – Returns whether an element can be quantified. Used only for reporting.

• excluded_elements (iterable of str) – Elements never proposed (in addition to settings.excluded_elements).

• initial_outcome (FitOutcome, optional) – Fit of the spectrum with the fixed elements, if already available.

• background_refit_function (callable, optional) – Fit function with the background scale fixed (e.g. make_xsp_fit_function with fixed_par_vals={‘K’: K}, K from the fit above 5 keV). Used instead of fit_function if the background of the starting fit is outside settings.background_ratio_bounds.

• priority_elements (iterable of str) – Pool of elements likely to be present (e.g. identified in other spectra of the same sample). For each residual peak, pool elements that pass the screening are verified first, and the other candidates are fitted only if no pool element explains the peak. This saves full fits, at the cost of a bias towards the pool for overlapping candidates.

• meas_type (str) – Measurement method, for the overlap table.

Return type:

ElementIDResult