autoemx.core.em_runtime.particle_finder module

Automated Electron Microscopy (EM) Particle Analysis Toolkit

This module provides the EM_Particle_Finder class for particle detection, selection of X-ray spectra (EDS, WDS) acquisition spots, and particle size/statistics analysis. It is designed to interface with an EM driver and an EM_Controller object to streamline the workflow from image acquisition to data export.

Currently only supports SEM.

Main Class

EM_Particle_Finder

Automates particle detection, mask generation, and X-ray spectra spot selection on detected particles. Supports both fully automated and manual (user-guided) collection modes. Provides methods for collecting particle statistics and managing frame navigation.

Example Usage

# EDS spot acquisition workflow: >>> EM_controller = EM_Controller(…) >>> EM_controller.initialise_SEM() >>> particle_finder = EM_Particle_Finder(EM_controller, PowderMeasurementConfig, …) >>> while particle_finder.go_to_next_particle(): … xy_spot_list = particle_finder.get_XS_acquisition_spots_coord_list() … for (x,y) in xy_spot_list: … EM_controller.acquire_XS_spectrum()

# Particle analysis workflow: >>> EM_controller = EM_Controller(…) >>> EM_controller.initialise_SEM() >>> particle_finder = EM_Particle_Finder(EM_controller, PowderMeasurementConfig, results_dir=”./results”) >>> particle_finder.get_particle_stats(n_par_target=500)

Notes

  • Requires an initialised EM_Controller object, unless used in development mode

  • Requires a working microscope driver and appropriate hardware configuration (see the documentation page “Add new Electron Microscope Driver”), in particular: is_at_EM (True if running at the electron microscope), get_image_data(width, height, channel) (acquire an image), get_frame_width() (current frame width, in mm) and move_to(x, y) (move the stage).

Created on Wed Jul 31 09:28:07 2024

@author: Andrea

autoemx.core.em_runtime.particle_finder.FRAME_EDGE_MARGIN_PX = 3

Particles whose bounding box is within this many pixels of the frame edge are left out of particle size statistics (their size is not known), and the others corrected accordingly (see particle_statistics.edge_correction_factor).

autoemx.core.em_runtime.particle_finder.STATS_SAVE_EVERY_N_FRAMES = 5

A particle size statistics measurement in progress is saved to the sample ledger every this many frames (and at the end of each size band), so that it can be resumed if interrupted.

autoemx.core.em_runtime.particle_finder.min_valid_diameter_um(pixel_size_um: float, min_par_pixels: int) → float[source]

Equivalent diameter (µm) of a particle covering min_par_pixels pixels of size pixel_size_um.

autoemx.core.em_runtime.particle_finder.plan_size_bands_um(d_min_um: float, d_max_um: float, im_width_px: int, min_par_pixels: int, frame_width_range_um: Tuple[float, float] | None = None) → List[Tuple[float, float, float]][source]

Size bands of a particle size statistics measurement, as (d_min, d_max, frame_width) in µm, largest first.

Bands are planned from the smallest particles up, since they are the hardest to find: the first band is searched with the largest frame in which particles of d_min_um still cover min_par_pixels pixels, and goes up to the largest particles fitting that frame (a tenth of its width); the next band starts there, and so on. A band whose frame reaches MAX_FRAME_WIDTH_UM (or the largest frame of the microscope) takes all the particles left, up to d_max_um. Any narrow remainder is thus measured with the largest frames.

Parameters:

  • d_min_um (float) – Diameter range, in µm.

  • d_max_um (float) – Diameter range, in µm.

  • im_width_px (int) – Width of the frame images, in pixels.

  • min_par_pixels (int) – Minimum number of pixels of a particle for its area to be measured.

  • frame_width_range_um ((float, float), optional) – Smallest and largest frame width of the microscope, in µm. If its smallest frame is too large for the smallest particles to cover min_par_pixels pixels, the first band starts above d_min_um.

Raises:

ValueError – If the limits are not finite and positive, or the frames are too narrow in pixels for particles of min_par_pixels pixels to fit them.

autoemx.core.em_runtime.particle_finder.diameter_um_to_area_um2(diameter_um: float) → float[source]

Convert equivalent circular diameter (µm) to area (µm²).

autoemx.core.em_runtime.particle_finder.FRAME_WIDTH_PER_MAX_DIAMETER = 10.0

Frame width of particle size statistics relative to the largest particles it measures, and largest frame width.

autoemx.core.em_runtime.particle_finder.frame_width_um_from_max_area(max_area_par: float) → float[source]

Search-frame width (µm) from maximum accepted particle area (µm²).

autoemx.core.em_runtime.particle_finder.is_binary_mask(par_mask: ndarray) → bool[source]

Whether a particle mask is binary (bool, or uint8 with values 0 and 255 only), not a label image.

autoemx.core.em_runtime.particle_finder.label_image_stats(labels: ndarray) → Tuple[int, ndarray, ndarray][source]

Number of labels (background included), statistics and centroids of a label image, as returned by cv2.connectedComponentsWithStats (labels absent from the image have zero statistics).

class autoemx.core.em_runtime.particle_finder.StatsProgress(n_par_target: int, powder_meas_cfg: PowderMeasurementConfig, bands: List[ParticleSizeBand], measurement: SizeMeasurement | None = None, segmentation_id: int | None = None)[source]

Bases: object

Particle size statistics measurement being acquired: target, settings, working records of its size bands (with the particles of their frames), and its record in the attached ledger with the segmentation used.

n_par_target: int
powder_meas_cfg: PowderMeasurementConfig
bands: List[ParticleSizeBand]
measurement: SizeMeasurement | None = None
segmentation_id: int | None = None
class autoemx.core.em_runtime.particle_finder.EM_Particle_Finder(EM_controller, powder_meas_cfg: PowderMeasurementConfig, xsp_spot_selector: Callable[[XSpSpotSelectionContext], Sequence[Tuple[int, int]]] | None = None, results_dir: str | None = None, verbose: bool = True, development_mode: bool = True)[source]

Bases: object

Automated particle analysis and X-ray spectra (EDS, WDS) acquisition in an electron microscope (EM).

This class provides methods for particle detection, selection of X-ray spectra (EDS, WDS) acquisition spots, and particle size/statistics analysis. It is designed to interface with an EM driver and an EM_Controller object to streamline the workflow from image acquisition to data export.

Main Methods

initialise_SEM()

Wakes up the SEM microscope, sets measurement parameters, and evaluates locations to scan for particles.

go_to_next_particle()

Locates and moves to the next detected particle.

get_XS_acquisition_spots_coord_list()

Determines X-ray spectra (EDS, WDS) acquisition spots on the currently detected particle.

get_particle_stats()

Scans the full sample, collecting particle size distribution statistics.

Examples

# EDS spot acquisition workflow: >>> EM_controller = EM_Controller(…) >>> EM_controller.initialise_SEM() >>> particle_finder = EM_Particle_Finder(EM_controller, PowderMeasurementConfig, …) >>> while particle_finder.go_to_next_particle(): … xy_spot_list = particle_finder.get_XS_acquisition_spots_coord_list() … for (x,y) in xy_spot_list: … EM_controller.acquire_XS_spectrum()

# Particle analysis workflow: >>> EM_controller = EM_Controller(…) >>> EM_controller.initialise_SEM() >>> particle_finder = EM_Particle_Finder(EM_controller, PowderMeasurementConfig, results_dir=”./results”) >>> particle_finder.get_particle_stats(n_par_target=500)

ivar EM:

Reference to the parent EM_Controller instance (must be initialised before use).

vartype EM:

EM_Controller

ivar powder_meas_cfg:

Configuration object for powder measurement (see dataclass for details).

vartype powder_meas_cfg:

PowderMeasurementConfig

ivar par_selection_mode:

Particle navigation mode from powder_meas_cfg (‘auto’ or ‘manual’).

vartype par_selection_mode:

str

ivar results_dir:

Directory for saving result images and data.

vartype results_dir:

str or None

ivar verbose:

If True, print progress and information to the console.

vartype verbose:

bool

ivar development_mode:

If True, enables extra visualizations and debug image saving.

vartype development_mode:

bool

ivar Internal Attributes:

ivar ——————-:

ivar _sample_id:

Identifier for the sample. Inherited from EM_Controller.

vartype _sample_id:

str

ivar _im_width:

Image width in pixels. Inherited from EM_Controller.

vartype _im_width:

int

ivar _im_height:

Image height in pixels. Inherited from EM_Controller.

vartype _im_height:

int

ivar tot_par_cntr:

Counter for the total number of particles analyzed.

vartype tot_par_cntr:

int

ivar analyzed_pars:

Particle geometry records collected during particle size statistics. When a ledger is attached, this list is the same object as ledger.particles.

vartype analyzed_pars:

list of ParticleInfo

Notes

  • This class assumes that an EM_Controller object is already initialised and passed as EM_controller.

  • All configuration validation is performed in the respective dataclasses.

  • Attributes with a leading underscore are for internal use and should not be accessed directly by users.

current_par_area_um2: float | None
current_par_coordinates_mm: tuple | None
analyzed_pars: List[ParticleInfo]
pending_eds_frames: List[Tuple[Tuple[float, float, float], ParticleStatsFrame, dict | None]]
property is_manual_particle_selection: bool

Whether particle navigation uses manual centering prompts.

property is_list_particle_selection: bool

Whether particle navigation follows a caller-supplied id/coordinate list.

property uses_absolute_particle_ids: bool

True when tot_par_cntr is a ledger particle id (list-id mode).

property par_selection_mode: str
set_particle_targets(particle_ids: Iterable[int] | None = None, particle_coords_mm: Iterable[Sequence[float]] | None = None, ledger=None) → None[source]

Configure list-mode targets (ledger particle ids XOR stage coordinates).

Parameters:

  • particle_ids – Existing ParticleInfo.id values. Requires ledger; each id must exist and have absolute stage coordinates.

  • particle_coords_mm – Absolute stage positions (x, y) in mm for new particles.

  • ledger – Sample ledger used to resolve and validate particle_ids.

go_to_next_particle(particle_id: int | None = None, coordinates: Sequence[float] | None = None)[source]

Moves the microscope to the next particle, centering and zooming on it. It also re-adjusts brightness, contrast and focus.

Modes

  • auto: find the next particle in the search-frame grid.

  • manual: prompt the user to center a particle.

  • list: consume the next target from set_particle_targets (ledger particle id or absolute stage coordinates in mm). Explicit particle_id / coordinates arguments also select a specific target directly.

param particle_id:

Existing ledger particle id to revisit. Updates that ParticleInfo.

type particle_id:

int, optional

param coordinates:

Absolute stage (x, y) in mm. Creates a new particle on acquisition.

type coordinates:

sequence of float, optional

returns:

True if it could successfully move to a particle. False if no more particles are present in the sample or if execution is stopped by the user.

rtype:

bool

segment_frame(frame_image: ndarray, pixel_size_um: float, d_range_um: Tuple[float, float], segmentation_id: int, with_outlines: bool = False)[source]

Particles of a saved frame image valid for particle size statistics, with the current segmentation settings, among the diameters d_range_um searched in the frame (raised to the smallest particles of at least min_par_pixels pixels): the result of segmentation segmentation_id of the frame (FrameSegmentation), and with with_outlines the outlines of its particles (see particle_outlines), as a tuple.

is_particle_at_frame_edge(stats, i)[source]

Determines whether a particle is located at or near the edge of the image frame.

A margin is used to account for detection tolerances. If any part of the particle’s bounding box is within pixel_margin pixels of the image border, it is considered to be at the edge.

Parameters:

  • stats (ndarray) – Connected components statistics array (as returned by OpenCV), where each row corresponds to a detected particle and columns to bounding box info (LEFT, TOP, WIDTH, HEIGHT, etc.).

  • i (int) – Index of the particle in the stats array.

Returns:

True if the particle is at or near the edge of the image frame, False otherwise.

Return type:

bool

prepare_mask_for_visualization(mask: ndarray) → ndarray[source]

Prepare a segmentation mask for visualization.

Behavior:

  • Binary masks with values in {0, 1} are scaled to [0, 255].

  • Binary masks with values in {0, 255} are returned unchanged.

  • Integer label masks have brightness reversed so higher labels are brighter, with a minimum brightness of 30 for positive values. Background (0) remains black.

  • Floating-point masks are normalized to [0, 255] with the same rules for positive values and background.

get_XS_acquisition_spots_coord_list(n_tot_sp_collected, par_image=None, pixel_size_um=None, results_dir=None)[source]

Returns a list of pixel coordinates (in the current image) for X-ray spectrum spot collection on a particle.

The function finds a suitable particle mask, erodes the mask to avoid particle edges, finds bright regions (or peak spots), and selects up to powder_meas_cfg.max_spectra_per_par spots per particle with a minimum distance between them (determined through ‘powder_meas_cfg.par_mask_margin’). The selection strategy for features and spacing can be controlled via powder_meas_cfg.par_feature_selection and powder_meas_cfg.par_spot_spacing.

Parameters:

  • n_tot_sp_collected (int) – Counter for the total number of spectra collected (used for labeling).

  • par_image (ndarray, optional) – Grayscale image of the current frame. If not provided and running at the SEM, the image is acquired.

  • pixel_size_um (float, optional) – Pixel size in micrometers. Required if not running at the SEM.

  • results_dir (str, optional) – Directory to save result images.

Returns:

  • selected_points (list[tuple[int, int]]) – List of selected (x, y) spot coordinates in image pixel units.

  • Potential Improvements

  • ———————

  • - Select points to ensure all phases are individuated, biasing spot selection to ensure representation – of phases with different bright/dark contrast. Currently, only the brightest spots are picked, which may miss some phases.

  • - Add detection of both peaks and valleys to ensure both bright and dark spots are tested.

  • - Exclude points that are close to larger particles along the X-ray emission path to the EDS detector, – as these may absorb emitted X-rays and degrade spectral signal.

attach_stats_ledger(ledger, ledger_path: str | None = None) → None[source]

Attach a SampleLedger whose particles list stores PSD ParticleInfo records.

get_particle_stats(n_par_target, count_eds_frames: bool = False)[source]

Analyze frames to collect and save particle size statistics.

With automatic navigation, the diameter range is split into size bands, planned from the smallest particles up (see plan_size_bands_um()): each band is searched with the largest FOV in which its smallest particles cover powder_meas_cfg.min_par_pixels pixels, up to the largest particles fitting that FOV. Bands are collected largest first, finer bands in subframes of the frames visited. The bands, with the number and total area of the frames analysed in each, are recorded with the statistics in the sample ledger. The statistics are weighted by band, by the particles found per unit area scanned (see autoemx.core.particle_statistics.band_particle_weights()). Top-level frames use normal labels (A0); finer bands use hierarchical ids (A0_a0) for sub-frames inside visited parent FOVs.

The measurement is recorded in the sample ledger (SampleLedger.size_measurements) with the segmentation of its settings (SampleLedger.segmentations), and each frame analysed as a frame of the ledger with the particles found in it (SampleLedger.frames), saved regularly. A measurement interrupted with the same target and settings is resumed: completed bands are kept, and the interrupted band continues in the frames not analysed yet. With different settings, a new measurement starts (the interrupted one is kept, with its frames). Frames, measurements and statistics of earlier measurements stay in the ledger.

The statistics (added to SampleLedger.size_results as the active result) use the frames of this measurement and, if powder_meas_cfg.include_eds_auto_particles, the frames searched by EDS on the sample (or, without them, the particles measured by EDS with automatic particle selection).

With powder_meas_cfg.size_stats_ci_tolerance, the measurement then goes on, collecting about 50 % more particles in every band per round, until the 95 % confidence intervals of powder_meas_cfg.size_stats_ci_quantities are within the tolerance (or the frame or time limit is reached, or no frame is left).

With count_eds_frames (measurement continuing an EDS acquisition of the sample, with automatic navigation), the frames searched by EDS (frames of the ledger, if powder_meas_cfg.include_eds_auto_particles) count too: the particles found in them count towards the target of the size band of their diameter, the frames overlapping them are not searched again for the same sizes, and finer bands are also searched inside them.

Parameters:

  • n_par_target (int) – Target number of particles to find per size band (particles already in the ledger are not counted, unless counted in the frames searched by EDS with count_eds_frames); the minimum when stopping on precision.

  • count_eds_frames (bool, optional) – Whether to count the frames searched by EDS (default: False).

Returns:

Summary statistics DataFrame, or None if no particles were found.

Return type:

pandas.DataFrame or None

save_particle_statistics(output_file_suffix='', particles=None, bands=None, confidence_intervals=None, eds_scans=())[source]

Compute the size statistics of the analysed particles and save the particle sizes, the statistics and the histogram in the sample folder (see autoemx.core.particle_statistics.save_particle_statistics()).

Parameters:

  • output_file_suffix (str, optional) – String added to output file name

  • particles (list of ParticleInfo, optional) – Particles included in the statistics (default: all the analysed particles).

  • bands (list of ParticleSizeBand, optional) – Size bands of the measurement, to weight the statistics by band.

  • confidence_intervals (dict, optional) – 95 % confidence intervals of the statistics (default: computed).

  • eds_scans (list of ParticleSizeBand, optional) – Frames searched by EDS on the sample, included in the statistics.

Returns:

DataFrame containing the calculated particle size statistics.

Return type:

pandas.DataFrame