assert 0.8942165551551718 > 0.94 + where 0.8942165551551718 = <iotbx.merging_statistics.merging_stats object at 0x7efc415f7510>.cc_one_half + where <iotbx.merging_statistics.merging_stats object at 0x7efc415f7510> = <iotbx.merging_statistics.dataset_statistics object at 0x7efc415f7f50>.overall
Stacktrace
dials_data = <DataFetcher: /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data>
tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_scale_and_filter_image_gr0')
@pytest.mark.xfail(
sys.platform == "darwin" and platform.machine() == "arm64",
reason="CC1/2 somewhat differs for unknown reasons",
)
def test_scale_and_filter_image_group_mode(dials_data, tmp_path):
"""Test the scale and filter command line program."""
location = dials_data("multi_crystal_proteinase_k", pathlib=True)
command = [
"dials.scale",
"filtering.method=deltacchalf",
"stdcutoff=3.0",
"mode=image_group",
"max_percent_removed=6.0",
"max_cycles=6",
"d_min=2.0",
"group_size=5",
"unmerged_mtz=unmerged.mtz",
"scale_and_filter_results=analysis_results.json",
"error_model=None",
]
for i in [1, 2, 3, 4, 5, 7, 10]:
command.append(location / f"experiments_{i}.json")
command.append(location / f"reflections_{i}.pickle")
result = procrunner.run(command, working_directory=tmp_path)
assert not result.returncode and not result.stderr
assert (tmp_path / "scaled.refl").is_file()
assert (tmp_path / "scaled.expt").is_file()
assert (tmp_path / "analysis_results.json").is_file()
result = get_merging_stats(tmp_path / "unmerged.mtz")
assert result.overall.r_pim < 0.135 # 12/07/21 was 0.129,
> assert result.overall.cc_one_half > 0.94 # 12/07/21 was 0.953
E assert 0.8942165551551718 > 0.94
E + where 0.8942165551551718 = <iotbx.merging_statistics.merging_stats object at 0x7efc415f7510>.cc_one_half
E + where <iotbx.merging_statistics.merging_stats object at 0x7efc415f7510> = <iotbx.merging_statistics.dataset_statistics object at 0x7efc415f7f50>.overall
../lib/python3.7/site-packages/dials/tests/algorithms/scaling/test_scale.py:571: AssertionError
Standard Output
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_10.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_1.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_2.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_3.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_4.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_5.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_7.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/experiments_8.json
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_10.pickle
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_1.pickle
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_2.pickle
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_3.pickle
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_4.pickle
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_5.pickle
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_7.pickle
Downloadinghttps://dials.diamond.ac.uk/regression_data/multi_crystal_proteinase_k/reflections_8.pickle
DIALS (2018) Acta Cryst. D74, 85-97.https://doi.org/10.1107/S2059798317017235
DIALS 3.8
The following parameters have been modified:
output {
unmerged_mtz = "unmerged.mtz"
}
weighting {
error_model {
error_model = basic None
}
}
cut_data {
d_min = 2
}
filtering {
method = None *deltacchalf
deltacchalf {
max_percent_removed = 6
mode = dataset *image_group
group_size = 5
stdcutoff = 3
}
output {
scale_and_filter_results = "analysis_results.json"
}
}
input {
experiments = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/experiments_1.json
experiments = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/experiments_2.json
experiments = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/experiments_3.json
experiments = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/experiments_4.json
experiments = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/experiments_5.json
experiments = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/experiments_7.json
experiments = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/experiments_10.json
reflections = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/reflections_1.pickle
reflections = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/reflections_2.pickle
reflections = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/reflections_3.pickle
reflections = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/reflections_4.pickle
reflections = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/reflections_5.pickle
reflections = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/reflections_7.pickle
reflections = /home/jenkins/workspace/CCP4/series-8.0-linux/devtools/test-install/lib/python3.7/cctbx/dials_data/multi_crystal_proteinase_k/reflections_10.pickle
}
Checking for the existence of a reflection table
containing multiple datasets
Found 7 reflection tables & 7 experiments in total.
Dataset ids are: 0,1,2,3,4,5,6
Space group being used during scaling is P 4
Using median unit cell across experiments : (68.5017, 68.5017, 103.944, 90, 90, 90)
Scaling models have been initialised for all experiments.
================================================================================
The experiment id for this dataset is 0.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Combined 20 partial reflections with other partial reflections
Excluding 16972/20029 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16522
criterion: excluded for scaling, reflections: 10409
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 1.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Combined 32 partial reflections with other partial reflections
Excluding 16887/19947 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16455
criterion: excluded for scaling, reflections: 10354
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 2.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Combined 23 partial reflections with other partial reflections
Excluding 16958/20027 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16499
criterion: excluded for scaling, reflections: 10432
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 3.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Combined 29 partial reflections with other partial reflections
Excluding 16949/19995 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16514
criterion: excluded for scaling, reflections: 10381
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 4.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Combined 18 partial reflections with other partial reflections
Excluding 17016/20081 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16578
criterion: excluded for scaling, reflections: 10498
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 5.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Combined 21 partial reflections with other partial reflections
Excluding 17044/20111 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16585
criterion: excluded for scaling, reflections: 10471
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 6.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16843/19930 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16380
criterion: excluded for scaling, reflections: 10246
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
Configuring a MultiScaler to handle the individual Scalers.
Determining symmetry equivalent reflections across datasets.
Using quasi-random reflection selection. Selecting from 3395 symmetry groups
with <I/sI> > 1.0 (7480 reflections)). Selection target of 78.47 reflections
from each dataset, with a total number between 2197.26 and 2636.71.
Summary of cross-dataset reflection groups chosen (726 groups, 1975 reflections):
+----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----+
| d-range | n_groups | n_refl | 0 | 1 | 2 | 3 | 4 | 5 | 6 |
|----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----|
| 35.439 - 8.685 | 48 | 103 | 15 | 17 | 15 | 17 | 15 | 9 | 15 |
| 8.685 - 6.236 | 41 | 95 | 13 | 15 | 12 | 19 | 12 | 12 | 12 |
| 6.236 - 5.118 | 49 | 116 | 22 | 15 | 16 | 18 | 15 | 15 | 15 |
| 5.118 - 4.444 | 41 | 102 | 12 | 13 | 13 | 28 | 12 | 12 | 12 |
| 4.444 - 3.981 | 50 | 119 | 16 | 15 | 16 | 27 | 15 | 15 | 15 |
| 3.981 - 3.638 | 48 | 120 | 15 | 16 | 17 | 27 | 15 | 15 | 15 |
| 3.638 - 3.37 | 33 | 96 | 12 | 16 | 16 | 16 | 12 | 12 | 12 |
| 3.37 - 3.155 | 31 | 88 | 12 | 15 | 12 | 12 | 12 | 12 | 13 |
| 3.155 - 2.975 | 37 | 96 | 13 | 12 | 13 | 19 | 12 | 12 | 15 |
| 2.975 - 2.824 | 31 | 92 | 12 | 15 | 16 | 13 | 12 | 12 | 12 |
| 2.824 - 2.693 | 29 | 89 | 12 | 14 | 12 | 14 | 12 | 12 | 13 |
| 2.693 - 2.579 | 31 | 93 | 12 | 12 | 13 | 13 | 12 | 12 | 19 |
| 2.579 - 2.478 | 28 | 94 | 12 | 14 | 17 | 12 | 12 | 12 | 15 |
| 2.478 - 2.389 | 30 | 88 | 13 | 14 | 13 | 12 | 12 | 12 | 12 |
| 2.389 - 2.308 | 30 | 94 | 14 | 19 | 13 | 12 | 12 | 12 | 12 |
| 2.308 - 2.235 | 29 | 90 | 12 | 15 | 13 | 13 | 12 | 13 | 12 |
| 2.235 - 2.169 | 38 | 110 | 16 | 17 | 15 | 15 | 15 | 15 | 17 |
| 2.169 - 2.108 | 30 | 89 | 12 | 12 | 12 | 15 | 12 | 12 | 14 |
| 2.108 - 2.052 | 33 | 93 | 12 | 13 | 16 | 13 | 12 | 12 | 15 |
| 2.052 - 2.0 | 39 | 108 | 15 | 15 | 15 | 16 | 17 | 15 | 15 |
+----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----+
Summary of reflections chosen for minimisation from each dataset (7480 total):
+--------------+------------------+----------------------+----------------------+--------------------+
| Dataset id | reflections | randomly selected | randomly selected | combined number |
| | connected to | reflections | reflections | of reflections |
| | other datasets | within dataset | across datasets | |
|--------------+------------------+----------------------+----------------------+--------------------|
| 0 | 272 | 84 | 1131 | 1131 |
| 1 | 294 | 70 | 1136 | 1136 |
| 2 | 285 | 98 | 1134 | 1134 |
| 3 | 331 | 153 | 1020 | 1020 |
| 4 | 260 | 184 | 996 | 996 |
| 5 | 253 | 150 | 953 | 953 |
| 6 | 280 | 96 | 1110 | 1110 |
| total | 1975 | 835 | 7480 | 7480 |
+--------------+------------------+----------------------+----------------------+--------------------+
Completed configuration of MultiScaler.
================================================================================
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 1.17
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 7480 | 3.5357 |
| 1 | 7480 | 3.1227 |
| 2 | 7480 | 2.7053 |
| 3 | 7480 | 2.442 |
| 4 | 7480 | 2.2514 |
| 5 | 7480 | 2.2082 |
| 6 | 7480 | 2.158 |
| 7 | 7480 | 2.1225 |
| 8 | 7480 | 2.0978 |
| 9 | 7480 | 2.0919 |
| 10 | 7480 | 2.0818 |
| 11 | 7480 | 2.0763 |
| 12 | 7480 | 2.065 |
| 13 | 7480 | 2.0469 |
| 14 | 7480 | 2.0461 |
| 15 | 7480 | 2.006 |
| 16 | 7480 | 1.9972 |
| 17 | 7480 | 1.9911 |
| 18 | 7480 | 1.9851 |
| 19 | 7480 | 1.9813 |
| 20 | 7480 | 1.9752 |
| 21 | 7480 | 1.9723 |
| 22 | 7480 | 1.9675 |
| 23 | 7480 | 1.9625 |
| 24 | 7480 | 1.9551 |
| 25 | 7480 | 1.9495 |
| 26 | 7480 | 1.9445 |
| 27 | 7480 | 1.9372 |
| 28 | 7480 | 1.9349 |
| 29 | 7480 | 1.9279 |
| 30 | 7480 | 1.9261 |
| 31 | 7480 | 1.9254 |
| 32 | 7480 | 1.9246 |
| 33 | 7480 | 1.9232 |
| 34 | 7480 | 1.9204 |
| 35 | 7480 | 1.9172 |
| 36 | 7480 | 1.9147 |
| 37 | 7480 | 1.9104 |
| 38 | 7480 | 1.9085 |
| 39 | 7480 | 1.9069 |
| 40 | 7480 | 1.9056 |
| 41 | 7480 | 1.9039 |
| 42 | 7480 | 1.9023 |
| 43 | 7480 | 1.9016 |
| 44 | 7480 | 1.9009 |
| 45 | 7480 | 1.8996 |
| 46 | 7480 | 1.8976 |
| 47 | 7480 | 1.8938 |
| 48 | 7480 | 1.8893 |
| 49 | 7480 | 1.8844 |
| 50 | 7480 | 1.8771 |
| 51 | 7480 | 1.8758 |
| 52 | 7480 | 1.8748 |
| 53 | 7480 | 1.8731 |
| 54 | 7480 | 1.8718 |
| 55 | 7480 | 1.8703 |
| 56 | 7480 | 1.8696 |
| 57 | 7480 | 1.8664 |
| 58 | 7480 | 1.8657 |
| 59 | 7480 | 1.8619 |
| 60 | 7480 | 1.8599 |
| 61 | 7480 | 1.857 |
| 62 | 7480 | 1.8524 |
| 63 | 7480 | 1.8501 |
| 64 | 7480 | 1.8484 |
| 65 | 7480 | 1.8478 |
| 66 | 7480 | 1.8469 |
| 67 | 7480 | 1.845 |
| 68 | 7480 | 1.8438 |
| 69 | 7480 | 1.8414 |
| 70 | 7480 | 1.8397 |
| 71 | 7480 | 1.8371 |
| 72 | 7480 | 1.8359 |
| 73 | 7480 | 1.8341 |
| 74 | 7480 | 1.8329 |
| 75 | 7480 | 1.832 |
| 76 | 7480 | 1.8318 |
| 77 | 7480 | 1.8314 |
| 78 | 7480 | 1.8309 |
| 79 | 7480 | 1.8296 |
| 80 | 7480 | 1.8279 |
| 81 | 7480 | 1.8269 |
| 82 | 7480 | 1.8244 |
| 83 | 7480 | 1.8227 |
| 84 | 7480 | 1.8226 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
29 outliers have been identified.
4 outliers identified from normalised intensity analysis (E² > 100.0)
Performing multi-dataset profile/summation intensity optimisation.
+-----------------+---------+---------+
| Combination | CC1/2 | Rmeas |
|-----------------+---------+---------|
| prf only | 0.9158 | 0.17387 |
| sum only | 0.79775 | 0.2551 |
| Imid = 307.53 | 0.87004 | 0.21258 |
| Imid = 29830.43 | 0.95316 | 0.25774 |
| Imid = 2983.04 | 0.95427 | 0.24051 |
| Imid = 298.3 | 0.86789 | 0.21283 |
+-----------------+---------+---------+
Profile intensities determined to be best for scaling.
Combined outlier rejection has been performed across multiple datasets,
29 outliers have been identified.
4 outliers identified from normalised intensity analysis (E² > 100.0)
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 0.21
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 7449 | 1.6257 |
| 1 | 7449 | 1.6221 |
| 2 | 7449 | 1.6186 |
| 3 | 7449 | 1.6158 |
| 4 | 7449 | 1.614 |
| 5 | 7449 | 1.6119 |
| 6 | 7449 | 1.6104 |
| 7 | 7449 | 1.6097 |
| 8 | 7449 | 1.6085 |
| 9 | 7449 | 1.6077 |
| 10 | 7449 | 1.6069 |
| 11 | 7449 | 1.6061 |
| 12 | 7449 | 1.6059 |
| 13 | 7449 | 1.6054 |
| 14 | 7449 | 1.6053 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
29 outliers have been identified.
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 0.05
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 7451 | 1.6047 |
| 1 | 7451 | 1.6046 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Calculating error estimates of inverse scale factors.
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
29 outliers have been identified.
3 outliers identified from normalised intensity analysis (E² > 100.0)
The reflection table variances have been adjusted to account for the
uncertainty in the scaling models for all datasets
205 reflections excluded: scale factor < 0.001
Using median unit cell across experiments : (68.5017, 68.5017, 103.944, 90, 90, 90)
Performed cycle of scaling.
Performing a round of filtering.
Read 140120 predicted reflections
Selected 21217 scaled reflections
Combined 1 partial reflections with other partial reflections
Removed 2437 reflections below partiality threshold
Resolution bins
0: 48.437, 6.277
1: 6.277, 4.457
2: 4.457, 3.644
3: 3.644, 3.158
4: 3.158, 2.826
5: 2.826, 2.581
6: 2.581, 2.390
7: 2.390, 2.236
8: 2.236, 2.108
9: 2.108, 2.000
Summary of input data:
# Datasets: 7
# Groups: 30
# Reflections: 18779
# Unique reflections: 14720
CC 1/2 mean: 76.464
CC 1/2 excluding group 0: 76.336
CC 1/2 excluding group 1: 76.129
CC 1/2 excluding group 2: 76.517
CC 1/2 excluding group 3: 75.331
CC 1/2 excluding group 4: 76.636
CC 1/2 excluding group 5: 76.404
CC 1/2 excluding group 6: 76.458
CC 1/2 excluding group 7: 75.914
CC 1/2 excluding group 8: 75.446
CC 1/2 excluding group 9: 75.815
CC 1/2 excluding group 10: 76.353
CC 1/2 excluding group 11: 76.000
CC 1/2 excluding group 12: 75.254
CC 1/2 excluding group 13: 75.996
CC 1/2 excluding group 14: 76.011
CC 1/2 excluding group 15: 80.682
CC 1/2 excluding group 16: 77.758
CC 1/2 excluding group 17: 76.256
CC 1/2 excluding group 18: 76.038
CC 1/2 excluding group 19: 75.976
CC 1/2 excluding group 20: 91.883
CC 1/2 excluding group 21: 76.164
CC 1/2 excluding group 22: 76.371
CC 1/2 excluding group 23: 76.287
CC 1/2 excluding group 24: 75.903
CC 1/2 excluding group 25: 77.254
CC 1/2 excluding group 26: 76.418
CC 1/2 excluding group 27: 76.356
CC 1/2 excluding group 28: 76.129
CC 1/2 excluding group 29: 76.438
Dataset: 20, ΔCC½: -15.419
Dataset: 15, ΔCC½: -4.218
Dataset: 16, ΔCC½: -1.294
Dataset: 25, ΔCC½: -0.790
Dataset: 4, ΔCC½: -0.172
Dataset: 2, ΔCC½: -0.053
Dataset: 6, ΔCC½: 0.006
Dataset: 29, ΔCC½: 0.026
Dataset: 26, ΔCC½: 0.046
Dataset: 5, ΔCC½: 0.060
Dataset: 22, ΔCC½: 0.093
Dataset: 27, ΔCC½: 0.108
Dataset: 10, ΔCC½: 0.111
Dataset: 0, ΔCC½: 0.128
Dataset: 23, ΔCC½: 0.177
Dataset: 17, ΔCC½: 0.208
Dataset: 21, ΔCC½: 0.300
Dataset: 28, ΔCC½: 0.336
Dataset: 1, ΔCC½: 0.336
Dataset: 18, ΔCC½: 0.426
Dataset: 14, ΔCC½: 0.453
Dataset: 11, ΔCC½: 0.464
Dataset: 13, ΔCC½: 0.468
Dataset: 19, ΔCC½: 0.489
Dataset: 7, ΔCC½: 0.551
Dataset: 24, ΔCC½: 0.561
Dataset: 9, ΔCC½: 0.649
Dataset: 8, ΔCC½: 1.018
Dataset: 3, ΔCC½: 1.133
Dataset: 12, ΔCC½: 1.211
mean delta_cc_half: -0.420
stddev delta_cc_half: 2.938
cutoff value: -9.233
Removing image range (16, 24) from experiment 4
Removing ['4:25:25'] due to scaling outlier group.
Cycle 1 of filtering, n_reflections removed this cycle: 1028
Detected existence of a multi-dataset reflection table
containing 7 datasets.
Scaling models have been initialised for all experiments.
================================================================================
The experiment id for this dataset is 0.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16972/20029 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16522
criterion: excluded for scaling, reflections: 10411
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 1.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16887/19947 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16455
criterion: excluded for scaling, reflections: 10358
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 2.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16958/20027 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16499
criterion: excluded for scaling, reflections: 10432
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 3.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16950/19995 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16514
criterion: excluded for scaling, reflections: 10388
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 4.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 18256/20081 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 17978
criterion: excluded for scaling, reflections: 10706
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 3 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 5.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 17044/20111 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16585
criterion: excluded for scaling, reflections: 10473
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 6.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16843/19930 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16380
criterion: excluded for scaling, reflections: 10246
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
Configuring a MultiScaler to handle the individual Scalers.
Determining symmetry equivalent reflections across datasets.
Using quasi-random reflection selection. Selecting from 3125 symmetry groups
with <I/sI> > 1.0 (6841 reflections)). Selection target of 78.01 reflections
from each dataset, with a total number between 2184.28 and 2621.14.
Summary of cross-dataset reflection groups chosen (799 groups, 2073 reflections):
+----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----+
| d-range | n_groups | n_refl | 0 | 1 | 2 | 3 | 4 | 5 | 6 |
|----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----|
| 35.439 - 8.685 | 38 | 83 | 12 | 13 | 12 | 13 | 12 | 9 | 12 |
| 8.685 - 6.236 | 50 | 110 | 15 | 17 | 15 | 18 | 14 | 16 | 15 |
| 6.236 - 5.118 | 49 | 114 | 22 | 15 | 16 | 17 | 14 | 15 | 15 |
| 5.118 - 4.444 | 41 | 100 | 15 | 14 | 12 | 23 | 12 | 12 | 12 |
| 4.444 - 3.981 | 52 | 121 | 17 | 18 | 15 | 26 | 15 | 15 | 15 |
| 3.981 - 3.638 | 47 | 118 | 16 | 15 | 17 | 25 | 15 | 15 | 15 |
| 3.638 - 3.37 | 33 | 94 | 13 | 17 | 14 | 14 | 12 | 12 | 12 |
| 3.37 - 3.155 | 47 | 117 | 17 | 22 | 15 | 18 | 15 | 15 | 15 |
| 3.155 - 2.975 | 47 | 115 | 16 | 15 | 15 | 23 | 15 | 15 | 16 |
| 2.975 - 2.824 | 33 | 90 | 12 | 13 | 16 | 13 | 12 | 12 | 12 |
| 2.824 - 2.693 | 30 | 88 | 13 | 13 | 12 | 13 | 12 | 12 | 13 |
| 2.693 - 2.579 | 31 | 92 | 12 | 12 | 14 | 14 | 12 | 12 | 16 |
| 2.579 - 2.478 | 38 | 113 | 15 | 15 | 19 | 17 | 15 | 15 | 17 |
| 2.478 - 2.389 | 34 | 97 | 13 | 17 | 18 | 13 | 12 | 12 | 12 |
| 2.389 - 2.308 | 31 | 95 | 15 | 17 | 12 | 15 | 12 | 12 | 12 |
| 2.308 - 2.235 | 42 | 113 | 15 | 18 | 16 | 19 | 15 | 15 | 15 |
| 2.235 - 2.169 | 44 | 114 | 19 | 18 | 15 | 15 | 15 | 15 | 17 |
| 2.169 - 2.108 | 35 | 95 | 13 | 13 | 12 | 18 | 12 | 12 | 15 |
| 2.108 - 2.052 | 34 | 92 | 12 | 12 | 16 | 13 | 12 | 12 | 15 |
| 2.052 - 2.0 | 43 | 112 | 15 | 16 | 15 | 18 | 15 | 15 | 18 |
+----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----+
Summary of reflections chosen for minimisation from each dataset (6841 total):
+--------------+------------------+----------------------+----------------------+--------------------+
| Dataset id | reflections | randomly selected | randomly selected | combined number |
| | connected to | reflections | reflections | of reflections |
| | other datasets | within dataset | across datasets | |
|--------------+------------------+----------------------+----------------------+--------------------|
| 0 | 297 | 84 | 1064 | 1064 |
| 1 | 310 | 70 | 1084 | 1084 |
| 2 | 296 | 98 | 1114 | 1114 |
| 3 | 345 | 153 | 987 | 987 |
| 4 | 268 | 70 | 586 | 586 |
| 5 | 268 | 150 | 930 | 930 |
| 6 | 289 | 96 | 1076 | 1076 |
| total | 2073 | 721 | 6841 | 6841 |
+--------------+------------------+----------------------+----------------------+--------------------+
Completed configuration of MultiScaler.
================================================================================
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 0.89
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 6841 | 1.706 |
| 1 | 6841 | 1.6941 |
| 2 | 6841 | 1.6923 |
| 3 | 6841 | 1.6902 |
| 4 | 6841 | 1.6858 |
| 5 | 6841 | 1.6828 |
| 6 | 6841 | 1.68 |
| 7 | 6841 | 1.6761 |
| 8 | 6841 | 1.675 |
| 9 | 6841 | 1.6728 |
| 10 | 6841 | 1.6719 |
| 11 | 6841 | 1.6706 |
| 12 | 6841 | 1.6662 |
| 13 | 6841 | 1.6606 |
| 14 | 6841 | 1.6573 |
| 15 | 6841 | 1.6563 |
| 16 | 6841 | 1.6556 |
| 17 | 6841 | 1.6548 |
| 18 | 6841 | 1.6537 |
| 19 | 6841 | 1.6517 |
| 20 | 6841 | 1.6477 |
| 21 | 6841 | 1.6462 |
| 22 | 6841 | 1.642 |
| 23 | 6841 | 1.6378 |
| 24 | 6841 | 1.6348 |
| 25 | 6841 | 1.6342 |
| 26 | 6841 | 1.6336 |
| 27 | 6841 | 1.6333 |
| 28 | 6841 | 1.6327 |
| 29 | 6841 | 1.6319 |
| 30 | 6841 | 1.6307 |
| 31 | 6841 | 1.6295 |
| 32 | 6841 | 1.6286 |
| 33 | 6841 | 1.6281 |
| 34 | 6841 | 1.6278 |
| 35 | 6841 | 1.6275 |
| 36 | 6841 | 1.627 |
| 37 | 6841 | 1.6266 |
| 38 | 6841 | 1.6253 |
| 39 | 6841 | 1.6232 |
| 40 | 6841 | 1.6224 |
| 41 | 6841 | 1.6195 |
| 42 | 6841 | 1.6158 |
| 43 | 6841 | 1.6141 |
| 44 | 6841 | 1.6134 |
| 45 | 6841 | 1.6121 |
| 46 | 6841 | 1.6102 |
| 47 | 6841 | 1.6079 |
| 48 | 6841 | 1.6074 |
| 49 | 6841 | 1.6062 |
| 50 | 6841 | 1.6052 |
| 51 | 6841 | 1.6049 |
| 52 | 6841 | 1.6042 |
| 53 | 6841 | 1.6037 |
| 54 | 6841 | 1.6033 |
| 55 | 6841 | 1.6027 |
| 56 | 6841 | 1.6019 |
| 57 | 6841 | 1.6013 |
| 58 | 6841 | 1.6002 |
| 59 | 6841 | 1.5995 |
| 60 | 6841 | 1.5992 |
| 61 | 6841 | 1.5986 |
| 62 | 6841 | 1.5983 |
| 63 | 6841 | 1.5981 |
| 64 | 6841 | 1.5974 |
| 65 | 6841 | 1.5968 |
| 66 | 6841 | 1.5962 |
| 67 | 6841 | 1.5953 |
| 68 | 6841 | 1.5947 |
| 69 | 6841 | 1.5942 |
| 70 | 6841 | 1.5939 |
| 71 | 6841 | 1.5938 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
16 outliers have been identified.
Performing multi-dataset profile/summation intensity optimisation.
+-----------------+---------+---------+
| Combination | CC1/2 | Rmeas |
|-----------------+---------+---------|
| prf only | 0.94557 | 0.15977 |
| sum only | 0.85556 | 0.23881 |
| Imid = 313.67 | 0.94546 | 0.18377 |
| Imid = 29830.43 | 0.95937 | 0.25634 |
| Imid = 2983.04 | 0.95506 | 0.23447 |
| Imid = 298.3 | 0.94284 | 0.18385 |
+-----------------+---------+---------+
Profile intensities determined to be best for scaling.
Combined outlier rejection has been performed across multiple datasets,
16 outliers have been identified.
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 0.32
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 6825 | 1.4942 |
| 1 | 6825 | 1.4937 |
| 2 | 6825 | 1.4934 |
| 3 | 6825 | 1.4916 |
| 4 | 6825 | 1.4911 |
| 5 | 6825 | 1.4892 |
| 6 | 6825 | 1.4879 |
| 7 | 6825 | 1.4865 |
| 8 | 6825 | 1.4862 |
| 9 | 6825 | 1.4858 |
| 10 | 6825 | 1.4857 |
| 11 | 6825 | 1.4855 |
| 12 | 6825 | 1.4852 |
| 13 | 6825 | 1.4845 |
| 14 | 6825 | 1.4834 |
| 15 | 6825 | 1.4832 |
| 16 | 6825 | 1.4823 |
| 17 | 6825 | 1.4817 |
| 18 | 6825 | 1.4814 |
| 19 | 6825 | 1.4812 |
| 20 | 6825 | 1.4809 |
| 21 | 6825 | 1.4806 |
| 22 | 6825 | 1.4806 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
19 outliers have been identified.
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 0.04
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 6822 | 1.4752 |
| 1 | 6822 | 1.4751 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Calculating error estimates of inverse scale factors.
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
19 outliers have been identified.
The reflection table variances have been adjusted to account for the
uncertainty in the scaling models for all datasets
211 reflections excluded: scale factor < 0.001
Using median unit cell across experiments : (68.5017, 68.5017, 103.944, 90, 90, 90)
Performed cycle of scaling.
Performing a round of filtering.
Read 140120 predicted reflections
Selected 20185 scaled reflections
Combined 1 partial reflections with other partial reflections
Removed 2386 reflections below partiality threshold
Resolution bins
0: 48.437, 6.277
1: 6.277, 4.457
2: 4.457, 3.644
3: 3.644, 3.158
4: 3.158, 2.826
5: 2.826, 2.581
6: 2.581, 2.390
7: 2.390, 2.236
8: 2.236, 2.108
9: 2.108, 2.000
Summary of input data:
# Datasets: 7
# Groups: 29
# Reflections: 17798
# Unique reflections: 14112
CC 1/2 mean: 89.943
CC 1/2 excluding group 0: 89.986
CC 1/2 excluding group 1: 89.881
CC 1/2 excluding group 2: 90.045
CC 1/2 excluding group 3: 89.041
CC 1/2 excluding group 4: 90.612
CC 1/2 excluding group 5: 89.838
CC 1/2 excluding group 6: 89.867
CC 1/2 excluding group 7: 89.448
CC 1/2 excluding group 8: 88.996
CC 1/2 excluding group 9: 89.515
CC 1/2 excluding group 10: 89.956
CC 1/2 excluding group 11: 89.840
CC 1/2 excluding group 12: 90.007
CC 1/2 excluding group 13: 89.624
CC 1/2 excluding group 14: 89.738
CC 1/2 excluding group 15: 89.646
CC 1/2 excluding group 16: 91.424
CC 1/2 excluding group 17: 90.107
CC 1/2 excluding group 18: 89.696
CC 1/2 excluding group 19: 89.967
CC 1/2 excluding group 20: 89.891
CC 1/2 excluding group 21: 90.009
CC 1/2 excluding group 22: 90.013
CC 1/2 excluding group 23: 89.759
CC 1/2 excluding group 24: 91.234
CC 1/2 excluding group 25: 89.816
CC 1/2 excluding group 26: 89.873
CC 1/2 excluding group 27: 89.650
CC 1/2 excluding group 28: 90.141
Dataset: 16, ΔCC½: -1.480
Dataset: 24, ΔCC½: -1.291
Dataset: 4, ΔCC½: -0.669
Dataset: 28, ΔCC½: -0.198
Dataset: 17, ΔCC½: -0.164
Dataset: 2, ΔCC½: -0.102
Dataset: 22, ΔCC½: -0.070
Dataset: 21, ΔCC½: -0.065
Dataset: 12, ΔCC½: -0.064
Dataset: 0, ΔCC½: -0.043
Dataset: 19, ΔCC½: -0.023
Dataset: 10, ΔCC½: -0.012
Dataset: 20, ΔCC½: 0.053
Dataset: 1, ΔCC½: 0.063
Dataset: 26, ΔCC½: 0.070
Dataset: 6, ΔCC½: 0.076
Dataset: 11, ΔCC½: 0.103
Dataset: 5, ΔCC½: 0.105
Dataset: 25, ΔCC½: 0.127
Dataset: 23, ΔCC½: 0.184
Dataset: 14, ΔCC½: 0.205
Dataset: 18, ΔCC½: 0.247
Dataset: 27, ΔCC½: 0.293
Dataset: 15, ΔCC½: 0.297
Dataset: 13, ΔCC½: 0.319
Dataset: 9, ΔCC½: 0.429
Dataset: 7, ΔCC½: 0.496
Dataset: 3, ΔCC½: 0.902
Dataset: 8, ΔCC½: 0.947
mean delta_cc_half: 0.025
stddev delta_cc_half: 0.491
cutoff value: -1.447
Removing image range (17, 24) from experiment 3
Removing ['3:25:25'] due to scaling outlier group.
Cycle 2 of filtering, n_reflections removed this cycle: 1099
Detected existence of a multi-dataset reflection table
containing 7 datasets.
Finishing scale and filtering as have now removed more than the limit.
Scaling models have been initialised for all experiments.
================================================================================
The experiment id for this dataset is 0.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16972/20029 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16522
criterion: excluded for scaling, reflections: 10411
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 1.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16887/19947 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16455
criterion: excluded for scaling, reflections: 10358
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 2.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16958/20027 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16499
criterion: excluded for scaling, reflections: 10432
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 3.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 18051/19995 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 17769
criterion: excluded for scaling, reflections: 10388
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 3 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 4.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 18256/20081 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 17978
criterion: excluded for scaling, reflections: 10706
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 3 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 5.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 17044/20111 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16585
criterion: excluded for scaling, reflections: 10473
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
The experiment id for this dataset is 6.
The scaling model type being applied is physical.
Applying filter of min_isigi > -5.0, partiality > 0.4
Excluding 16849/19930 reflections
Reflections passing individual criteria:
criterion: user excluded, reflections: 16380
criterion: excluded for scaling, reflections: 10252
The following corrections will be applied to this dataset:
+--------------+----------------+
| correction | n_parameters |
|--------------+----------------|
| scale | 5 |
| decay | 3 |
+--------------+----------------+
Using previously determined optimal intensity choice: profile intensities
A round of outlier rejection has been performed,
0 outliers have been identified.
Completed preprocessing and initialisation for this dataset.
================================================================================
Configuring a MultiScaler to handle the individual Scalers.
Determining symmetry equivalent reflections across datasets.
Using quasi-random reflection selection. Selecting from 2889 symmetry groups
with <I/sI> > 1.0 (6287 reflections)). Selection target of 77.53 reflections
from each dataset, with a total number between 2170.80 and 2604.96.
Summary of cross-dataset reflection groups chosen (812 groups, 2038 reflections):
+----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----+
| d-range | n_groups | n_refl | 0 | 1 | 2 | 3 | 4 | 5 | 6 |
|----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----|
| 35.439 - 8.685 | 47 | 100 | 15 | 17 | 15 | 15 | 14 | 9 | 15 |
| 8.685 - 6.236 | 51 | 110 | 15 | 19 | 15 | 16 | 14 | 15 | 16 |
| 6.236 - 5.118 | 49 | 112 | 22 | 15 | 16 | 15 | 14 | 15 | 15 |
| 5.118 - 4.444 | 38 | 90 | 14 | 13 | 13 | 14 | 12 | 12 | 12 |
| 4.444 - 3.981 | 49 | 113 | 16 | 18 | 15 | 19 | 15 | 15 | 15 |
| 3.981 - 3.638 | 46 | 112 | 16 | 15 | 18 | 18 | 15 | 15 | 15 |
| 3.638 - 3.37 | 36 | 97 | 14 | 21 | 14 | 12 | 12 | 12 | 12 |
| 3.37 - 3.155 | 49 | 114 | 18 | 20 | 15 | 16 | 15 | 15 | 15 |
| 3.155 - 2.975 | 47 | 113 | 17 | 15 | 15 | 18 | 15 | 15 | 18 |
| 2.975 - 2.824 | 36 | 92 | 13 | 13 | 17 | 12 | 12 | 13 | 12 |
| 2.824 - 2.693 | 31 | 89 | 13 | 13 | 13 | 12 | 12 | 12 | 14 |
| 2.693 - 2.579 | 33 | 95 | 13 | 14 | 13 | 14 | 12 | 12 | 17 |
| 2.579 - 2.478 | 33 | 92 | 12 | 12 | 14 | 13 | 12 | 13 | 16 |
| 2.478 - 2.389 | 35 | 96 | 12 | 17 | 17 | 13 | 12 | 13 | 12 |
| 2.389 - 2.308 | 32 | 95 | 16 | 17 | 13 | 13 | 12 | 12 | 12 |
| 2.308 - 2.235 | 43 | 113 | 15 | 18 | 17 | 17 | 15 | 16 | 15 |
| 2.235 - 2.169 | 44 | 115 | 19 | 18 | 15 | 15 | 15 | 17 | 16 |
| 2.169 - 2.108 | 45 | 110 | 15 | 15 | 15 | 17 | 15 | 15 | 18 |
| 2.108 - 2.052 | 36 | 93 | 13 | 12 | 15 | 12 | 12 | 15 | 14 |
| 2.052 - 2.0 | 32 | 87 | 12 | 12 | 12 | 12 | 12 | 13 | 14 |
+----------------+------------+----------+-----+-----+-----+-----+-----+-----+-----+
Summary of reflections chosen for minimisation from each dataset (6287 total):
+--------------+------------------+----------------------+----------------------+--------------------+
| Dataset id | reflections | randomly selected | randomly selected | combined number |
| | connected to | reflections | reflections | of reflections |
| | other datasets | within dataset | across datasets | |
|--------------+------------------+----------------------+----------------------+--------------------|
| 0 | 300 | 84 | 1038 | 1038 |
| 1 | 314 | 70 | 1053 | 1053 |
| 2 | 297 | 98 | 1084 | 1084 |
| 3 | 293 | 72 | 635 | 635 |
| 4 | 267 | 70 | 569 | 569 |
| 5 | 274 | 150 | 872 | 872 |
| 6 | 293 | 96 | 1036 | 1036 |
| total | 2038 | 640 | 6287 | 6287 |
+--------------+------------------+----------------------+----------------------+--------------------+
Completed configuration of MultiScaler.
================================================================================
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 1.02
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 6287 | 1.7803 |
| 1 | 6287 | 1.7631 |
| 2 | 6287 | 1.7524 |
| 3 | 6287 | 1.7432 |
| 4 | 6287 | 1.7356 |
| 5 | 6287 | 1.731 |
| 6 | 6287 | 1.7235 |
| 7 | 6287 | 1.7226 |
| 8 | 6287 | 1.7193 |
| 9 | 6287 | 1.7172 |
| 10 | 6287 | 1.7147 |
| 11 | 6287 | 1.7101 |
| 12 | 6287 | 1.7019 |
| 13 | 6287 | 1.6989 |
| 14 | 6287 | 1.6913 |
| 15 | 6287 | 1.6857 |
| 16 | 6287 | 1.6822 |
| 17 | 6287 | 1.676 |
| 18 | 6287 | 1.6688 |
| 19 | 6287 | 1.6649 |
| 20 | 6287 | 1.6601 |
| 21 | 6287 | 1.6569 |
| 22 | 6287 | 1.6516 |
| 23 | 6287 | 1.6487 |
| 24 | 6287 | 1.6436 |
| 25 | 6287 | 1.6385 |
| 26 | 6287 | 1.6338 |
| 27 | 6287 | 1.6302 |
| 28 | 6287 | 1.6283 |
| 29 | 6287 | 1.6259 |
| 30 | 6287 | 1.6203 |
| 31 | 6287 | 1.6195 |
| 32 | 6287 | 1.6182 |
| 33 | 6287 | 1.6171 |
| 34 | 6287 | 1.6158 |
| 35 | 6287 | 1.6156 |
| 36 | 6287 | 1.6143 |
| 37 | 6287 | 1.612 |
| 38 | 6287 | 1.6114 |
| 39 | 6287 | 1.6097 |
| 40 | 6287 | 1.6081 |
| 41 | 6287 | 1.6077 |
| 42 | 6287 | 1.6062 |
| 43 | 6287 | 1.606 |
| 44 | 6287 | 1.6055 |
| 45 | 6287 | 1.6046 |
| 46 | 6287 | 1.6042 |
| 47 | 6287 | 1.6032 |
| 48 | 6287 | 1.602 |
| 49 | 6287 | 1.601 |
| 50 | 6287 | 1.6 |
| 51 | 6287 | 1.5988 |
| 52 | 6287 | 1.5981 |
| 53 | 6287 | 1.5967 |
| 54 | 6287 | 1.5956 |
| 55 | 6287 | 1.5949 |
| 56 | 6287 | 1.5927 |
| 57 | 6287 | 1.5917 |
| 58 | 6287 | 1.5911 |
| 59 | 6287 | 1.5904 |
| 60 | 6287 | 1.5897 |
| 61 | 6287 | 1.5887 |
| 62 | 6287 | 1.5872 |
| 63 | 6287 | 1.5851 |
| 64 | 6287 | 1.5825 |
| 65 | 6287 | 1.5818 |
| 66 | 6287 | 1.5807 |
| 67 | 6287 | 1.5803 |
| 68 | 6287 | 1.5799 |
| 69 | 6287 | 1.5796 |
| 70 | 6287 | 1.579 |
| 71 | 6287 | 1.5788 |
| 72 | 6287 | 1.5784 |
| 73 | 6287 | 1.5782 |
| 74 | 6287 | 1.5779 |
| 75 | 6287 | 1.5777 |
| 76 | 6287 | 1.5775 |
| 77 | 6287 | 1.5771 |
| 78 | 6287 | 1.5767 |
| 79 | 6287 | 1.5763 |
| 80 | 6287 | 1.576 |
| 81 | 6287 | 1.5758 |
| 82 | 6287 | 1.5752 |
| 83 | 6287 | 1.575 |
| 84 | 6287 | 1.5746 |
| 85 | 6287 | 1.5745 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
9 outliers have been identified.
Performing multi-dataset profile/summation intensity optimisation.
+-----------------+---------+---------+
| Combination | CC1/2 | Rmeas |
|-----------------+---------+---------|
| prf only | 0.94793 | 0.15289 |
| sum only | 0.89277 | 0.21916 |
| Imid = 320.19 | 0.95189 | 0.17534 |
| Imid = 29830.43 | 0.96997 | 0.25033 |
| Imid = 2983.04 | 0.95951 | 0.23052 |
| Imid = 298.3 | 0.95145 | 0.17574 |
+-----------------+---------+---------+
Profile intensities determined to be best for scaling.
Combined outlier rejection has been performed across multiple datasets,
9 outliers have been identified.
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with an LBFGS minimizer.
Time taken for refinement 0.25
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 6278 | 1.4898 |
| 1 | 6278 | 1.4893 |
| 2 | 6278 | 1.4888 |
| 3 | 6278 | 1.4876 |
| 4 | 6278 | 1.4874 |
| 5 | 6278 | 1.4862 |
| 6 | 6278 | 1.4848 |
| 7 | 6278 | 1.4826 |
| 8 | 6278 | 1.4819 |
| 9 | 6278 | 1.4813 |
| 10 | 6278 | 1.4809 |
| 11 | 6278 | 1.4807 |
| 12 | 6278 | 1.4805 |
| 13 | 6278 | 1.4802 |
| 14 | 6278 | 1.4798 |
| 15 | 6278 | 1.4793 |
| 16 | 6278 | 1.479 |
| 17 | 6278 | 1.4786 |
| 18 | 6278 | 1.4784 |
| 19 | 6278 | 1.4782 |
+--------+--------+----------+
RMSD no longer decreasing
lbfgs minimizer stop: callback_after_step is True
================================================================================
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
10 outliers have been identified.
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with a Levenberg-Marquardt minimizer.
Time taken for refinement 0.18
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 6277 | 1.4763 |
| 1 | 6277 | 1.4753 |
| 2 | 6277 | 1.4744 |
| 3 | 6277 | 1.474 |
| 4 | 6277 | 1.4738 |
+--------+--------+----------+
RMSD no longer decreasing
================================================================================
Components to be refined in this cycle for all datasets: scale, decay
Performing a round of scaling with a Levenberg-Marquardt minimizer.
Time taken for refinement 0.07
Refinement steps:
+--------+--------+----------+
| Step | Nref | RMSD_I |
| | | (a.u) |
|--------+--------+----------|
| 0 | 6277 | 1.4738 |
| 1 | 6277 | 1.4738 |
+--------+--------+----------+
RMSD no longer decreasing
================================================================================
Calculating error estimates of inverse scale factors.
Scale factors determined during minimisation have now been
applied to all datasets.
Combined outlier rejection has been performed across multiple datasets,
10 outliers have been identified.
The reflection table variances have been adjusted to account for the
uncertainty in the scaling models for all datasets
211 reflections excluded: scale factor < 0.001
Using median unit cell across experiments : (68.5017, 68.5017, 103.944, 90, 90, 90)
Total time taken: 2.7325s
================================================================================
5.77% of model parameters have significant uncertainty
(sigma/abs(parameter) > 0.5)
Summary of image ranges removed:
Excluded images for experiment id: 3, image range: [1, 25], limited range: [(1, 16)]
Excluded images for experiment id: 4, image range: [1, 25], limited range: [(1, 15)]
Summary of dataset partialities
+------------------+----------+
| Partiality (p) | n_refl |
|------------------+----------|
| all reflections | 140120 |
| p > 0.99 | 67901 |
| 0.5 < p < 0.99 | 8331 |
| 0.01 < p < 0.5 | 8393 |
| p < 0.01 | 55495 |
+------------------+----------+
Reflections below a partiality_cutoff of 0.4 are not considered for any
part of the scaling analysis or for the reporting of merging statistics.
Additionally, if applicable, only reflections with a min_partiality > 0.95
were considered for use when refining the scaling model.
----------Merging statistics by resolution bin----------
d_max d_min #obs #uniq mult. %comp <I> <I/sI> r_mrg r_meas r_pim r_anom cc1/2 cc_ano
57.22 5.43 1007 771 1.31 46.22 29.2 22.2 0.080 0.112 0.077 0.148 0.990* 0.000
5.43 4.31 947 738 1.28 45.39 35.5 14.6 0.108 0.148 0.100 0.183 0.952* 0.000
4.31 3.76 921 725 1.27 44.18 36.4 16.6 0.122 0.170 0.118 0.229 0.863* -1.000
3.76 3.42 976 770 1.27 47.30 33.2 15.2 0.144 0.200 0.138 0.221 0.677* 0.000
3.42 3.17 1006 783 1.28 48.04 24.7 11.9 0.124 0.170 0.117 0.251 0.907* 0.996
3.17 2.99 917 736 1.25 45.80 16.9 8.5 0.130 0.178 0.121 0.229 0.950* -1.000
2.99 2.84 976 765 1.28 47.55 14.9 7.4 0.142 0.194 0.132 0.240 0.928* -1.000
2.84 2.71 992 766 1.30 47.08 11.8 6.3 0.156 0.214 0.146 0.290 0.873* 0.000
2.71 2.61 888 697 1.27 43.43 11.2 6.0 0.122 0.162 0.106 0.175 0.946* 0.774
2.61 2.52 987 760 1.30 46.63 9.6 5.1 0.145 0.198 0.134 0.314 0.906* 0.782
2.52 2.44 978 756 1.29 47.10 9.3 4.9 0.165 0.229 0.157 0.327 0.878* -1.000
2.44 2.37 961 739 1.30 46.04 7.4 3.9 0.170 0.229 0.153 0.291 0.871* 0.000
2.37 2.31 990 763 1.30 46.92 7.7 3.9 0.171 0.231 0.155 0.332 0.857* 0.000
2.31 2.25 862 689 1.25 43.09 7.6 3.8 0.181 0.246 0.165 0.391 0.882* 0.000
2.25 2.20 995 763 1.30 46.78 6.7 3.3 0.245 0.327 0.215 0.482 0.824* 0.000
2.20 2.15 938 721 1.30 45.01 6.2 3.1 0.235 0.321 0.218 0.504 0.745* 0.000
2.15 2.11 1025 777 1.32 47.55 5.3 2.7 0.289 0.391 0.261 0.505 0.626* -0.233
2.11 2.07 932 739 1.26 46.83 5.1 2.5 0.222 0.305 0.208 0.448 0.828* 0.000
2.07 2.03 897 701 1.28 43.03 4.8 2.4 0.239 0.331 0.227 0.470 0.676* 0.000
2.03 2.00 898 694 1.29 42.50 4.5 2.2 0.329 0.450 0.304 0.601 0.517* -1.000
57.20 2.00 19093 14853 1.29 45.80 14.5 7.4 0.143 0.196 0.134 0.261 0.894* -0.046
-------------Summary of merging statistics--------------
Overall Low High
High resolution limit 2.00 5.43 2.00
Low resolution limit 57.20 57.22 2.03
Completeness 45.8 46.2 42.5
Multiplicity 1.3 1.3 1.3
I/sigma 7.4 22.2 2.2
Rmerge(I) 0.143 0.080 0.329
Rmerge(I+/-) 0.143 0.083 0.310
Rmeas(I) 0.196 0.112 0.450
Rmeas(I+/-) 0.199 0.116 0.431
Rpim(I) 0.134 0.077 0.304
Rpim(I+/-) 0.139 0.081 0.299
CC half 0.894 0.990 0.517
Anomalous completeness 6.6 6.9 6.3
Anomalous multiplicity 1.1 1.1 1.1
Anomalous correlation -0.046 0.000 -1.000
Anomalous slope 0.786
dF/F 0.192
dI/s(dI) 0.907
Total observations 19093 1007 898
Total unique 14853 771 694
Writing html report to dials.scale.html
Summary of data removed:
Cycle number: 1
Removed image ranges:
(16, 24), dataset 4 cumulative % of reflections removed: 4.845
Cycle number: 2
Removed image ranges:
(17, 24), dataset 3 cumulative % of reflections removed: 10.025
Total time taken: 17.9806s
================================================================================
Writing html report to dials.scale.html
Saving the scaled experiments to scaled.expt
Saving the scaled reflections to scaled.refl
Saving output to an unmerged mtz file to unmerged.mtz.
Rotating experiments about axis (-0.7071067811865475, -4.329780281177466e-17, 0.7071067811865476) by 180.00°
Rotating experiments about axis (-0.7071067811865475, -4.329780281177466e-17, 0.7071067811865476) by 180.00°
Rotating experiments about axis (-0.7071067811865475, -4.329780281177466e-17, 0.7071067811865476) by 180.00°
Rotating experiments about axis (-0.7071067811865475, -4.329780281177466e-17, 0.7071067811865476) by 180.00°
Rotating experiments about axis (-0.7071067811865475, -4.329780281177466e-17, 0.7071067811865476) by 180.00°
Rotating experiments about axis (-0.7071067811865475, -4.329780281177466e-17, 0.7071067811865476) by 180.00°
Rotating experiments about axis (-0.7071067811865475, -4.329780281177466e-17, 0.7071067811865476) by 180.00°
Experiment crystals differ. Using first experiment crystal for file-level data.
Using median unit cell across experiments : (68.5017, 68.5017, 103.944, 90, 90, 90)
Read 140120 predicted reflections
Selected 19093 scaled reflections
Removed 0 reflections with d <= 2.00
Combined 1 partial reflections with other partial reflections
Removed 0 intensity.scale.value reflections with I/Sig(I) < -5.0
Saving 19092 integrated reflections to unmerged.mtz
Title: From dials.scale
Space group symbol from file: P4
Space group number from file: 75
Space group from matrices: P 4 (No. 75)
Point group symbol from file: 4
Number of batches: 175
Number of crystals: 1
Number of Miller indices: 19092
Resolution range: 57.1978 2.00006
History:
From DIALS 3.8, run on 2023-01-12 at 01:47:09 UTC (2023-01-12 at 01:47:09 GMT)
Crystal 1:
Name: XTAL
Project: DIALS
Id: 1
Unit cell: (68.5017, 68.5017, 103.944, 90, 90, 90)
Number of datasets: 1
Dataset 1:
Name: FROMDIALS
Id: 1
Wavelength: 1.03936
Number of columns: 17
label #valid %valid min max type
H 19092 100.00% 0.00 34.00 H: index h,k,l
K 19092 100.00% 0.00 34.00 H: index h,k,l
L 19092 100.00% 0.00 51.00 H: index h,k,l
M_ISYM 19092 100.00% 1.00 8.00 Y: M/ISYM, packed partial/reject flag and symmetry number
BATCH 19092 100.00% 1.00 625.00 B: BATCH number
I 19092 100.00% -74.22 864.12 J: intensity
SIGI 19092 100.00% 0.07 141.65 Q: standard deviation
SCALEUSED 19092 100.00% 0.16 7.51 R: real
SIGSCALEUSED 19092 100.00% 0.03 0.35 R: real
BG 19092 100.00% 126.70 11804.26 R: real
SIGBG 19092 100.00% 12.33 123.93 R: real
FRACTIONCALC 19092 100.00% 0.40 1.00 R: real
XDET 19092 100.00% 449.72 2078.61 R: real
YDET 19092 100.00% 499.60 2138.19 R: real
ROT 19092 100.00% -0.06 2.57 R: real
LP 19092 100.00% 0.01 0.50 R: real
QE 19092 100.00% 0.88 0.91 R: real
See dials.github.io/dials_scale_user_guide.html for more info on scaling options