autoemx.config.schema_models.clustering module

class autoemx.config.schema_models.clustering.DBSCANParams(*, eps: float | None = None, min_samples: int = 3, metric: str = 'euclidean')

Bases: BaseModel

Parameters for DBSCAN clustering.

Only relevant when ClusteringConfig.method is "dbscan". Grouped in a dedicated sub-model so method-specific knobs do not pollute the top-level clustering config and remain individually typed and validated.

eps is a distance in the clustering geometry, whose scale differs by roughly an order of magnitude between fractions (Euclidean) and CLR coordinates (Aitchison). When left as None, the geometry-specific default in DEFAULT_EPS is used.

DEFAULT_EPS: ClassVar[Dict[str, float]] = {'aitchison': 0.3, 'euclidean': 0.05}

eps: float | None

min_samples: int

metric: str

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod validate_eps(value: float | None) → float | None

resolved_eps(geometry: str) → float

Return eps if set, otherwise the default for geometry (“euclidean” or “aitchison”).

classmethod validate_min_samples(value: int) → int

classmethod validate_metric(value: str) → str

class autoemx.config.schema_models.clustering.AitchisonParams(*, detection_limit_percent: float = 0.5, auto_near_zero_percent: float = 1.0, auto_max_near_zero_fraction: float = 0.1)

Bases: BaseModel

Parameters for clustering in Aitchison geometry.

Only relevant when ClusteringConfig.geometry is "aitchison" or "auto". Before the CLR transform, compositions are closed and low values are replaced multiplicatively (Martín-Fernández et al., 2003) with 0.65 x detection_limit_percent:

• exact zeros of every element (a measured zero means “below detection”);

• all values below the detection limit of trace elements, i.e. elements whose median fraction across the clustered compositions is below 2 x the detection limit.

Measured sub-detection-limit values of major elements are kept. This prevents noisy trace elements from dominating log-ratio distances without collapsing real low concentrations of major elements.

With geometry="auto", Euclidean geometry is used instead of Aitchison when either condition below holds on the clustered compositions, since log-ratios then tend to amplify noise or over-split:

• at most two elements are present (log-ratio space is one-dimensional);

• in at least auto_max_near_zero_fraction of the spectra, some major element (median >= 2 x the detection limit) is below auto_near_zero_percent, e.g. phases with largely disjoint elements, or mixture lines reaching an end-member.

detection_limit_percent: float

auto_near_zero_percent: float

auto_max_near_zero_fraction: float

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod validate_detection_limit_percent(value: float) → float

classmethod validate_auto_near_zero_percent(value: float) → float

classmethod validate_auto_max_near_zero_fraction(value: float) → float

class autoemx.config.schema_models.clustering.ClusterMergeParams(*, dip_alpha: float = 0.05, max_gap_ratio: float = 1.0, dbscan_min_samples: int = 5, dbscan_eps_factor: float = 3.0)

Bases: BaseModel

Parameters for merging k-means clusters that are pieces of one continuous population.

Only used when ClusteringConfig.auto_merge_clusters is True, with k-means and an automatically chosen number of clusters. k-means can cut a continuous spread of compositions, e.g. a mixture line between two phases, into several clusters. After clustering, two clusters are merged when all of the following hold, in the clustering geometry:

• their compositions projected onto the axis joining the two cluster centers form a single mode (Hartigan’s dip test, p-value above dip_alpha);

• the empty gap between them along that axis is below max_gap_ratio times the spread (standard deviation) of the larger cluster;

• DBSCAN connects them through dense neighbourhoods, with min_samples = dbscan_min_samples and eps = dbscan_eps_factor times the median distance of the compositions to their dbscan_min_samples-th nearest neighbour.

Pairs are merged one at a time, most compatible first, and the criteria are re-evaluated after each merge. Clusters are never split.

dip_alpha: float

max_gap_ratio: float

dbscan_min_samples: int

dbscan_eps_factor: float

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod validate_dip_alpha(value: float) → float

classmethod validate_positive(value: float) → float

classmethod validate_min_samples(value: int) → int

class autoemx.config.schema_models.clustering.MixtureParams(*, max_n_phases: int = 4, max_recon_error: float = 0.4, max_recon_error_binary: float = 1.0, recon_error_alpha: float = 15.0, conf_sigma: float = 0.5, single_phase_max_rms_dist: float = 0.03, single_phase_min_ref_conf: float = 0.5, nmf_min_mixture_conf: float = 0.5, equivalent_recon_error_tol: float = 0.01, max_reported_mixtures: int = 5, report_within_conf_ratio: float = 0.9, min_reported_conf_ratio: float = 0.5, collapse_equivalent_mixtures: bool = True, equivalent_span_tol: float = 0.005)

Bases: BaseModel

Parameters for decomposing clusters into mixtures of candidate phases (ref_formulae).

Clusters are fitted as non-negative combinations of candidate phases (NNLS), scored by the exponential reconstruction error mean(exp(recon_error_alpha * |X - WH|) - 1), which is mapped to a confidence score exp(-error^2 / (2 * conf_sigma^2)).

Adding phases always lowers the reconstruction error, so only the fewest phases that the shape of the cluster requires are used: a binary mixture spreads along a line between two phases (with a width set by measurement noise), a ternary one over a plane. Pairs of candidate phases are always tested, and listed when their error is below max_recon_error_binary (1, confidence ~0.14; only pairs below max_recon_error explain the cluster, weaker pairs are listed for inspection). Combinations of 3 up to max_n_phases phases are only tested when no combination with fewer phases reaches an error below max_recon_error, stopping at the first number of phases that does. max_n_phases = 2 restricts the analysis to binary mixtures. The default max_recon_error (0.4, confidence ~0.73) separates single-phase standards, known binary mixtures and ternary mixtures measured on a Phenom XL.

Combinations whose reconstruction errors differ by less than equivalent_recon_error_tol explain the cluster equally well, e.g. collinear candidate phases such as CaO and Ca4Ta2O9 in a CaO-Ta2O5 mixture. Among these, the combination with fewer phases and then with the phases closest to each other in composition space (smallest sum of pairwise distances) is ranked first.

All mixtures are saved in the ledger. Clusters.csv shows a mixture only if its confidence is at least min_reported_conf_ratio of the best one in the cluster, and it is among the first max_reported_mixtures or its confidence is within report_within_conf_ratio of the best one. A note in Clusters.csv gives the number of mixtures only saved in the ledger. With collapse_equivalent_mixtures, mixtures whose phases span the same mixing line or plane as a better-ranked mixture (within equivalent_span_tol, in fractions) are equivalent decompositions and are not shown either, e.g. pairs of Sr-Ta oxides that all lie on the SrO-Ta2O5 line.

Clusters are not decomposed if they are considered single-phase: their RMS distance from the centroid is below single_phase_max_rms_dist and they match a candidate phase with confidence above single_phase_min_ref_conf. If no mixture of candidate phases reaches a confidence of nmf_min_mixture_conf, the cluster is also decomposed into two phases of unknown composition with free NMF.

max_n_phases: int

max_recon_error: float

max_recon_error_binary: float

recon_error_alpha: float

conf_sigma: float

single_phase_max_rms_dist: float

single_phase_min_ref_conf: float

nmf_min_mixture_conf: float

equivalent_recon_error_tol: float

max_reported_mixtures: int

report_within_conf_ratio: float

min_reported_conf_ratio: float

collapse_equivalent_mixtures: bool

equivalent_span_tol: float

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod validate_max_n_phases(value: int) → int

classmethod validate_positive(value: float) → float

classmethod validate_max_reported_mixtures(value: int) → int

classmethod validate_conf_ratio(value: float) → float

classmethod validate_equivalent_span_tol(value: float) → float

classmethod validate_non_negative(value: float) → float

classmethod validate_confidence(value: float) → float

class autoemx.config.schema_models.clustering.ClusteringConfig(*, clustering_id: int = 0, method: str = 'kmeans', geometry: str = 'auto', features: str = 'at_fr', k_forced: int | None = None, k_resolved: int | None = None, k_finding_method: str = 'silhouette', max_k: int = 6, ref_formulae: List[str] = <factory>, do_matrix_decomposition: bool = True, max_analytical_error_percent: float | None = 5.0, min_bckgrnd_cnts: float | None = 5.0, quant_flags_accepted: List[int] = <factory>, dbscan: DBSCANParams = <factory>, aitchison: AitchisonParams = <factory>, auto_merge_clusters: bool = True, cluster_merge: ClusterMergeParams = <factory>, mixture: MixtureParams = <factory>)

Bases: BaseModel

Configuration for clustering of compositions and their filtering.

ALLOWED_METHODS: ClassVar[Tuple[str, ...]] = ('kmeans', 'dbscan')

ALLOWED_GEOMETRIES: ClassVar[Tuple[str, ...]] = ('euclidean', 'aitchison', 'auto')

LEGACY_GEOMETRY: ClassVar[str] = 'euclidean'

LEGACY_AUTO_MERGE_CLUSTERS: ClassVar[bool] = False

clustering_id: int

method: str

geometry: str

features: str

k_forced: int | None

k_resolved: int | None

k_finding_method: str

max_k: int

ref_formulae: List[str]

do_matrix_decomposition: bool

max_analytical_error_percent: float | None

min_bckgrnd_cnts: float | None

quant_flags_accepted: List[int]

dbscan: DBSCANParams

aitchison: AitchisonParams

auto_merge_clusters: bool

cluster_merge: ClusterMergeParams

mixture: MixtureParams

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod validate_clustering_id_input(v: Any) → Any

classmethod validate_clustering_id(value: int) → int

classmethod validate_non_empty_strings(value: str) → str

classmethod validate_method(value: str) → str

classmethod validate_geometry(value: str) → str

classmethod validate_max_k(value: int) → int

classmethod validate_k(value: int | None) → int | None

validate_k_semantics() → ClusteringConfig

classmethod validate_max_analytical_error_percent(value: float | None) → float | None

classmethod validate_min_bckgrnd_cnts(value: float | None) → float | None

classmethod validate_ref_formulae(values: List[str]) → List[str]

classmethod validate_quant_flags_accepted(values: List[int]) → List[int]

fingerprint_payload() → Dict[str, Any]

fingerprint() → str

fingerprint_differences(other: ClusteringConfig) → Dict[str, Dict[str, Any]]

class autoemx.config.schema_models.clustering.ClusteringResult(*, centroids: List[List[float]] = <factory>, els_std_dev_per_cluster: List[List[float]] = <factory>, centroids_other_fr: List[List[float]] = <factory>, els_std_dev_per_cluster_other_fr: List[List[float]] = <factory>, n_points_per_cluster: List[int] = <factory>, wcss_per_cluster: List[float] = <factory>, rms_dist_cluster: List[float] = <factory>, rms_dist_cluster_other_fr: List[float] = <factory>, refs_assigned_rows: Dict[str, ~typing.Any]]=<factory>, wcss: float, sil_score: float, tot_n_points: int, clusters_assigned_mixtures: List[Any] = <factory>)

Bases: BaseModel

Cluster analysis artifacts produced by one analysis run.

centroids: List[List[float]]

els_std_dev_per_cluster: List[List[float]]

centroids_other_fr: List[List[float]]

els_std_dev_per_cluster_other_fr: List[List[float]]

n_points_per_cluster: List[int]

wcss_per_cluster: List[float]

rms_dist_cluster: List[float]

rms_dist_cluster_other_fr: List[float]

refs_assigned_rows: List[Dict[str, Any]]

wcss: float

sil_score: float

tot_n_points: int

clusters_assigned_mixtures: List[Any]

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class autoemx.config.schema_models.clustering.ClusteringAnalysis(*, config: ClusteringConfig, result: ClusteringResult | None = None)

Bases: BaseModel

Bundle of the clustering config and the resulting analysis artifacts.

config: ClusteringConfig

result: ClusteringResult | None

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod backfill_legacy_geometry(data: Any) → Any

Configs saved before geometry existed were clustered in Euclidean geometry, and configs saved before auto_merge_clusters existed without merging clusters.