torchref.refinement.targets.difference module

class torchref.refinement.targets.difference.DifferenceXrayTarget(dataset_collection=None, data_light=None, data_dark=None, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, use_work_set=True, verbose=0)[source]

Bases: Target

Target for time-resolved crystallography comparing light/dark states.

Computes difference structure factors and compares against observed differences:

Uses Gaussian NLL with proper error propagation:

  • σ_diff = sqrt(σ_light² + σ_dark²)

  • NLL = 0.5 * (ΔF_obs - ΔF_calc)² / σ_diff² + log(σ_diff) + 0.5*log(2π)

Supports two initialization modes:

  1. DatasetCollection mode (recommended): Pass a DatasetCollection with pre-aligned datasets. This is more efficient and ensures consistency with other targets using the same data.

  2. Separate datasets mode: Pass individual ReflectionData objects. HKL matching is performed automatically.

Parameters:

  • dataset_collection (DatasetCollection, optional) – Collection containing ‘dark’ and ‘light’ datasets (pre-aligned HKL). If provided, data_light and data_dark are ignored.

  • data_light (ReflectionData, optional) – Reflection data for the light (excited) state.

  • data_dark (ReflectionData, optional) – Reflection data for the dark (ground) state.

  • model_light (ModelFT or MixedModel) – Model for the light state structure factor calculation.

  • model_dark (ModelFT) – Model for the dark state structure factor calculation.

  • scaler_light (ScalerBase, optional) – Scaler for the light state F_calc. Can be shared with other targets.

  • scaler_dark (ScalerBase, optional) – Scaler for the dark state F_calc. Can be shared with other targets.

  • use_work_set (bool, optional) – If True, compute loss on work set. Default is True.

  • verbose (int, optional) – Verbosity level. Default is 0.

Examples

Using DatasetCollection (recommended for sharing scalers):

# Create collection with aligned HKL
collection = DatasetCollection()
collection.add_dataset('dark', data_dark, set_as_reference=True)
collection.add_dataset('light', data_light)

# Create shared scalers
scaler_dark = IsotropicScaler(data=collection['dark'], model=model_dark)
scaler_light = IsotropicScaler(data=collection['light'], model=model_mixed)

# Create targets that share scalers
xray_dark = GaussianXrayTarget(
    data=collection['dark'], model=model_dark, scaler=scaler_dark
)
xray_light = GaussianXrayTarget(
    data=collection['light'], model=model_mixed, scaler=scaler_light
)
diff_target = DifferenceXrayTarget(
    dataset_collection=collection,
    model_light=model_mixed,
    model_dark=model_dark,
    scaler_light=scaler_light,
    scaler_dark=scaler_dark,
)

# Combined loss
loss = xray_dark() + xray_light() + diff_target()

Using separate datasets:

diff_target = DifferenceXrayTarget(
    data_light=data_light,
    data_dark=data_dark,
    model_light=model_light,
    model_dark=model_dark,
)
loss = diff_target()

With mixed model for partial occupancy:

mixed_light = MixedModel([model_dark, model_light], [0.7, 0.3])
diff_target = DifferenceXrayTarget(
    dataset_collection=collection,
    model_light=mixed_light,
    model_dark=model_dark,
    scaler_light=scaler_light,
    scaler_dark=scaler_dark,
)
name: str = 'difference_xray'
__init__(dataset_collection=None, data_light=None, data_dark=None, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, use_work_set=True, verbose=0)[source]

Initialize DifferenceXrayTarget.

property dataset_collection

DatasetCollection if using collection mode.

property data_light: ReflectionData

Light state reflection data.

property data_dark: ReflectionData

Dark state reflection data.

property model_light: ModelFT

Light state model.

property model_dark: ModelFT

Dark state model.

property scaler_light: Scaler

Light state scaler.

property scaler_dark: Scaler

Dark state scaler.

property hkl: Tensor

Common HKL indices for both datasets.

Returns the aligned HKL from DatasetCollection if available, otherwise the matched HKL computed from separate datasets.

get_delta_F_obs()[source]

Get observed difference structure factors with error propagation.

Returns:

  • delta_F_obs (torch.Tensor) – ΔF_obs = F_light_obs - F_dark_obs

  • sigma_diff (torch.Tensor) – σ_diff = sqrt(σ_light² + σ_dark²)

  • mask (torch.Tensor) – Boolean mask for work/test set selection and valid data.

Return type:

Tuple[Tensor, Tensor, Tensor]

get_delta_F_calc(fcalc_light=None, fcalc_dark=None, recalc=False)[source]

Compute calculated difference structure factors.

ΔF_calc = |F_light_calc| - |F_dark_calc|

Parameters:

  • fcalc_light (torch.Tensor, optional) – Pre-computed light state structure factors.

  • fcalc_dark (torch.Tensor, optional) – Pre-computed dark state structure factors.

  • recalc (bool, optional) – Force recalculation if True. Default is False.

Returns:

ΔF_calc for all reflections (full size, use mask from get_delta_F_obs).

Return type:

torch.Tensor

forward(fcalc_light=None, fcalc_dark=None, recalc=False)[source]

Compute Gaussian NLL loss for difference structure factors.

NLL = 0.5 * (ΔF_obs - ΔF_calc)² / σ_diff² + log(σ_diff) + 0.5*log(2π)

Parameters:

  • fcalc_light (torch.Tensor, optional) – Pre-computed light state structure factors.

  • fcalc_dark (torch.Tensor, optional) – Pre-computed dark state structure factors.

  • recalc (bool, optional) – Force recalculation if True. Default is False.

Returns:

Mean NLL loss value.

Return type:

torch.Tensor

stats(fcalc_light=None, fcalc_dark=None)[source]

Get statistics for difference refinement.

Parameters:

  • fcalc_light (torch.Tensor, optional) – Pre-computed light state structure factors.

  • fcalc_dark (torch.Tensor, optional) – Pre-computed dark state structure factors.

Returns:

Statistics dict with correlation, R_diff, etc.

Return type:

dict

class torchref.refinement.targets.difference.PhaseInformedDifferenceTarget(dataset_collection, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, phase_source='difference', use_work_set=True, verbose=0)[source]

Bases: Target

Phase-informed difference target for time-resolved crystallography.

Uses model phases to create complex observed differences, then compares with calculated complex differences:

ΔF_calc = F_mixed_calc - F_dark_calc (complex) ΔF_obs_complex = ΔF_obs * exp(i * φ) (using model phases) Loss = |ΔF_obs_complex - ΔF_calc|² / σ_diff²

The phase source can be configured: - “dark”: Use dark model phases (stable reference) - “difference”: Use phase of calculated difference ΔF_calc (self-consistent) - “mixed”: Use mixed/light model phases

Using current model phases is standard practice in difference Fourier methods. The iterative nature of refinement self-corrects any phase bias, and the localized nature of difference peaks allows detection of weak signals.

Parameters:

  • dataset_collection (DatasetCollection) – Collection containing ‘dark’ and ‘light’ datasets.

  • model_light (ModelFT or MixedModel) – Model for the light/excited state.

  • model_dark (ModelFT) – Model for the dark/ground state.

  • scaler_light (Scaler, optional) – Scaler for light state F_calc.

  • scaler_dark (Scaler, optional) – Scaler for dark state F_calc.

  • phase_source (str, optional) – Source for phases: “dark”, “difference”, or “mixed”. Default is “difference”.

  • use_work_set (bool, optional) – If True, compute loss on work set only. Default is True.

  • verbose (int, optional) – Verbosity level. Default is 0.

Examples

Using difference phases (recommended):

target = PhaseInformedDifferenceTarget(
    dataset_collection=collection,
    model_light=mixed_model,
    model_dark=model_dark,
    phase_source="difference",
)

Using dark phases:

target = PhaseInformedDifferenceTarget(
    dataset_collection=collection,
    model_light=mixed_model,
    model_dark=model_dark,
    phase_source="dark",
)
name: str = 'phase_informed_difference'
__init__(dataset_collection, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, phase_source='difference', use_work_set=True, verbose=0)[source]

Initialize target.

Parameters:

verbose (int, optional) – Verbosity level. Default is 0.

property hkl: Tensor

Common HKL indices.

forward(fcalc_light=None, fcalc_dark=None, recalc=True)[source]

Compute phase-informed difference loss.

Parameters:

  • fcalc_light (torch.Tensor, optional) – Pre-computed light state structure factors.

  • fcalc_dark (torch.Tensor, optional) – Pre-computed dark state structure factors.

  • recalc (bool, optional) – Force recalculation if True. Default is True.

Returns:

Mean weighted squared error.

Return type:

torch.Tensor

stats(fcalc_light=None, fcalc_dark=None)[source]

Get statistics for the difference refinement.

Returns:

Dictionary with loss, correlation, R_diff, etc.

Return type:

dict

class torchref.refinement.targets.difference.TaylorCorrectedDifferenceTarget(dataset_collection, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, use_work_set=True, verbose=0)[source]

Bases: Target

Taylor-corrected difference target for time-resolved crystallography.

Uses an exact Taylor expansion to properly account for the phase shift between dark and light states when constructing observed complex differences:

ΔF_obs = exp(i*φ_dark) * [F_obs_dark * (exp(i*dφ) - 1) + dF_obs * exp(i*dφ)]

Where:

  • dφ = φ_light_calc - φ_dark_calc (phase rotation from model)

  • dF_obs = F_obs_light - F_obs_dark (observed amplitude difference)

This formulation:

  1. Uses the exact complex exponential (no small-angle approximation)

  2. Properly accounts for both the amplitude difference and phase rotation

  3. Eliminates the false minimum that causes refinement to stop at ~70%

The loss is computed as:

Loss = |ΔF_obs_corrected - ΔF_calc|² / σ_diff²

Parameters:

  • dataset_collection (DatasetCollection) – Collection containing ‘dark’ and ‘light’ datasets.

  • model_light (ModelFT or MixedModel) – Model for the light/excited state.

  • model_dark (ModelFT) – Model for the dark/ground state.

  • scaler_light (Scaler, optional) – Scaler for light state F_calc.

  • scaler_dark (Scaler, optional) – Scaler for dark state F_calc.

  • use_work_set (bool, optional) – If True, compute loss on work set only. Default is True.

  • verbose (int, optional) – Verbosity level. Default is 0.

Examples

Basic usage:

target = TaylorCorrectedDifferenceTarget(
    dataset_collection=collection,
    model_light=mixed_model,
    model_dark=model_dark,
)

With scalers:

target = TaylorCorrectedDifferenceTarget(
    dataset_collection=collection,
    model_light=mixed_model,
    model_dark=model_dark,
    scaler_light=scaler_light,
    scaler_dark=scaler_dark,
)
name: str = 'taylor_corrected_difference'
__init__(dataset_collection, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, use_work_set=True, verbose=0)[source]

Initialize target.

Parameters:

verbose (int, optional) – Verbosity level. Default is 0.

property hkl: Tensor

Common HKL indices.

forward(fcalc_light=None, fcalc_dark=None, recalc=True)[source]

Compute Taylor-corrected difference loss.

The observed complex difference is constructed using the exact Taylor expansion:

ΔF_obs = exp(i*φ_dark) * [F_obs_dark * (exp(i*dφ) - 1) + dF_obs * exp(i*dφ)]

Parameters:

  • fcalc_light (torch.Tensor, optional) – Pre-computed light state structure factors.

  • fcalc_dark (torch.Tensor, optional) – Pre-computed dark state structure factors.

  • recalc (bool, optional) – Force recalculation if True. Default is True.

Returns:

Mean weighted squared error.

Return type:

torch.Tensor

compute_free_metrics(fcalc_light=None, fcalc_dark=None)[source]

Compute loss and correlation on the FREE (test) set.

This is the key metric for detecting overfitting in the α-δF degeneracy. The correct solution should have better free set metrics.

Returns:

Dictionary with ‘free_loss’ and ‘free_correlation’.

Return type:

dict

stats(fcalc_light=None, fcalc_dark=None)[source]

Get statistics for the difference refinement.

Returns:

Dictionary with loss, correlation, R_diff, etc.

Return type:

dict

class torchref.refinement.targets.difference.RiceDifferenceTarget(dataset_collection, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, use_work_set=True, verbose=0)[source]

Bases: Target

Rice-distribution difference target for time-resolved crystallography.

Works in complex space by grafting detached model phases onto observed amplitudes, then taking the complex difference. The magnitude of this complex difference is always non-negative, enabling a proper Rice distribution likelihood.

The procedure:

  1. Reconstruct complex observed structure factors using detached model phases:

    F_obs_light_complex = F_obs_light * exp(i * φ_calc_light)
F_obs_dark_complex  = F_obs_dark  * exp(i * φ_calc_dark)

  2. Form complex differences:

    ΔF_obs_complex = F_obs_light_complex - F_obs_dark_complex
ΔF_calc        = F_calc_light - F_calc_dark

  3. Compute strictly positive amplitudes:

    A_obs = |ΔF_obs_complex|   (always ≥ 0)
ν     = |ΔF_calc|          (always ≥ 0)

  4. Apply Rice distribution NLL:

    NLL = -log(A) + log(σ²) + (A² + ν²)/(2σ²)
      - log(I₀(A·ν/σ²))

The Rice distribution naturally models the magnitude of a complex signal plus Gaussian noise, making it statistically appropriate for comparing amplitudes that are always positive by construction.

Parameters:

  • dataset_collection (DatasetCollection) – Collection containing ‘dark’ and ‘light’ datasets.

  • model_light (ModelFT or MixedModel) – Model for the light/excited state.

  • model_dark (ModelFT) – Model for the dark/ground state.

  • scaler_light (Scaler, optional) – Scaler for light state F_calc.

  • scaler_dark (Scaler, optional) – Scaler for dark state F_calc.

  • use_work_set (bool, optional) – If True, compute loss on work set only. Default is True.

  • verbose (int, optional) – Verbosity level. Default is 0.

Examples

Basic usage:

target = RiceDifferenceTarget(
    dataset_collection=collection,
    model_light=mixed_model,
    model_dark=model_dark,
)

With scalers:

target = RiceDifferenceTarget(
    dataset_collection=collection,
    model_light=mixed_model,
    model_dark=model_dark,
    scaler_light=scaler_light,
    scaler_dark=scaler_dark,
)
name: str = 'rice_difference'
__init__(dataset_collection, model_light=None, model_dark=None, scaler_light=None, scaler_dark=None, use_work_set=True, verbose=0)[source]

Initialize target.

Parameters:

verbose (int, optional) – Verbosity level. Default is 0.

property hkl: Tensor

Common HKL indices.

forward(fcalc_light=None, fcalc_dark=None, recalc=True)[source]

Compute Rice distribution NLL loss for difference structure factors.

Parameters:

  • fcalc_light (torch.Tensor, optional) – Pre-computed light state structure factors.

  • fcalc_dark (torch.Tensor, optional) – Pre-computed dark state structure factors.

  • recalc (bool, optional) – Force recalculation if True. Default is True.

Returns:

Mean Rice NLL loss value.

Return type:

torch.Tensor

compute_free_metrics(fcalc_light=None, fcalc_dark=None)[source]

Compute loss and correlation on the FREE (test) set.

Returns:

Dictionary with ‘free_loss’ and ‘free_correlation’.

Return type:

dict

stats(fcalc_light=None, fcalc_dark=None)[source]

Get statistics for the Rice difference refinement.

Returns:

Dictionary with loss, correlation, R_diff, etc.

Return type:

dict