The two ICDD PDF entries 43-1748 and 46-1964 are used (see UPPW11 and UPPW11-2)
for 8 tests of these benchmarks.
(in all cases using the 20 first lines, or the 20 first lines with I
>5%(I/Imax) :
A- indexing the raw data. A(1) for ICDD PDF
entry 43-1748 and A(2) for the 46-1964.
No permission to find the zeropoint except with an internal system.
B- indexing the data with I >5% (I/Imax).
B(1) and B(2) as above.
Most experienced powder diffractionist try to index on data obtained after
removing the small intensity peaks.
C- indexing the data corrected from zeropoint
(-0.10°(2-theta)). C(1) and C(2) as above.
D- indexing the data corrected from the zeropoint
and on selected peaks with I > 5% (I/Imax). D(1) and D(2) as above.
To these tests are added two more ones, much easier :
E indexing new high quality conventional
X-ray data
F- indexing synchrotron data (X3B1 beamline
- thanks to Peter Stephens).
The indexing programs should be applied to the
following series of 10 datasets in two modes :
- automated (using default values), and
- manual.
The total number of tests is thus equal to 20.
Indexing benchmarks data (couples of 2-theta and Intensity values) :
A(1) A(2) B(1) B(2) C(1) C(2) D(1) D(2) E(3) F(4)
2-thet I 2-thet I 2-thet I 2-thet I 2-thet I 2-thet I 2-thet I 2-thet I 2-thet I 2-thet I
6.238 2 6.690 2 6.712 10 13.601 5 6.138 2 6.590 2 6.612 10 13.501 5 10.765 58 4.887 6
6.712 10 9.417 1 13.171 10 14.757 6 6.612 10 9.317 1 13.071 10 14.657 6 13.522 27 6.139 33
9.403 3 10.849 1 13.584 16 15.492 29 9.303 3 10.749 1 13.484 16 15.392 29 14.690 122 6.664 26
13.171 10 13.135 2 14.882 10 16.463 24 13.071 10 13.035 2 14.782 10 16.363 24 15.398 101 6.989 101
13.584 16 13.601 5 15.498 40 17.419 20 13.484 16 13.501 5 15.398 40 17.319 20 16.336 65 7.403 56
14.483 4 14.757 6 16.528 37 18.925 58 14.383 4 14.657 6 16.428 37 18.825 58 16.453 91 7.460 79
14.882 10 15.492 29 17.430 34 19.730 11 14.782 10 15.392 29 17.330 34 19.630 11 17.312 106 7.850 82
15.498 40 16.463 24 18.928 100 20.131 63 15.398 40 16.363 24 18.828 100 20.031 63 18.828 405 8.531 225
16.528 37 17.419 20 19.808 12 20.841 100 16.428 37 17.319 20 19.708 12 20.741 100 19.699 60 8.921 30
17.430 34 18.925 58 20.148 60 22.508 23 17.330 34 18.825 58 20.048 60 22.408 23 20.031 500 9.062 47
18.534 4 19.730 11 20.852 90 23.224 6 18.434 4 19.630 11 20.752 90 23.124 6 20.752 500 9.251 180
18.928 100 20.131 63 22.539 22 23.614 13 18.828 100 20.031 63 22.439 22 23.514 13 21.641 141 9.386 304
19.808 12 20.841 100 23.680 13 23.979 20 19.708 12 20.741 100 23.580 13 23.879 20 22.414 173 9.781 11
20.148 60 21.722 3 23.982 18 25.085 56 20.048 60 21.622 3 23.882 18 24.985 56 22.788 34 10.136 93
20.852 90 22.508 23 25.088 56 25.650 12 29.752 90 22.408 23 24.988 56 25.550 12 23.152 40 10.300 8
21.852 2 23.224 6 25.844 20 26.796 15 21.752 2 23.124 6 25.744 20 26.696 15 23.528 72 10.468 21
22.331 3 23.614 13 26.556 12 27.407 75 22.231 3 23.514 13 26.456 12 27.307 75 23.872 354 10.634 28
22.539 22 23.979 20 27.412 78 28.262 19 22.439 22 23.879 20 27.312 78 28.162 19 24.970 327 10.783 62
23.252 4 25.085 56 28.272 24 29.724 19 23.152 4 24.985 56 28.172 24 29.624 19 25.365 34 11.280 160
23.680 13 25.650 12 29.786 14 31.220 20 23.580 13 25.550 12 29.686 14 31.120 20 25.549 47 11.450 16
(1) ICDD PDF entry 43-1748, use wavelength = 1.5418 Å
(2) ICDD PDF entry 46-1964, use wavelength = 1.5418 Å
(3) conventional X-ray data, use wavelength = 1.54056 Å
(4) synchrotron data, use wavelength = 0.6995 Å (thanks to Peter Stephens)
You may either download individual test files (click on the A(1), A(2) etc above andComments :
you will obtain a .txt ASCII file), or download the ten tests in benchmarks.zip.
Do not forget to apply these ten tests in both default and manual modes - if possible...
Why 20 lines ? Because "There are few exceptions from the rule that, if
all of the first 20 lines are indexed and M20 >10, the indexing
is physically reliable."
See: Werner, P.-E.: Autoindexing. in: Structure Determination from powder
diffraction data (Eds. W.I.F. David, K. Shank-land, L.B. McCusker, Ch. Baerlocher),
p.118-135. Oxford Science Publications 2002.
Some indexing programs expect 20 lines to be the minimum.
There are 8 impurity lines among the first 26 lines in the PDF entry 43-1748, and 3 impurity lines among the first 35 lines in the PDF entry 46-1964. Moreover, both patterns have a surprising large zeropoint error close to -0.10(2-theta)°. The difficulty level is thus decreasing from tests A to F. The application with default values should be preferably done in all crystal symmetries, or at least with maximum cell parameters being 20Å and Vmax = 2000Å3 in monoclinic symmetry. These conditions correspond probably to more than 50% of the crystal structures stored in the ICSD and CSD databases). The application in manual mode should be restricted to a monoclinic search in the 800-1200 Å3 volume range, and 5-20 Å cell parameters.
If a program expects to be used successfully by unexperienced users, it is clear that it should offer an automated/default/black-box mode.
The best FoM was reported from the use of the synchrotron data : M(20)
= 197, F20 = 1080 (0.0006, 32), the cell being monoclinic with a =
8.875 (Å), b = 16.408 (Å), c = 7.137 (Å), and beta = 93.84
(°), V = 1036.9 (Å3), space group P21/n.
Answers to these benchmarks are to be given in a simple way. Send the input and output files. Notation is either -1 or 0 or 1 :
The 1 point note to the A, B, C, D, E or F tests means that the correct cell was found in first FoM position among the proposals. Such a 1 point note means that the program can produce sometimes good results in unexperienced hands. A 1V note means that the correct solution is found in first position in a list of cells sorted by increasing volume.
The zero point note means that the correct cell is mixed with uncorrect ones, not at the head of the list. In that case, an expert may have still chances to locate the correct cell among the garbage, if it is listed among the first ten, but this requires much more additional work. And sometimes, the correct cell is only listed among the first 100 or 1000...! The order of the true solution in the list is given as a subscript : 06 means that it was the sixth cell proposal. But the order has to be < 10, otherwise, there is a -1 point note given.
The -1 point note means that the correct cell was not found at all, or at a position larger than 10 in the lists.
The results are reported below. A problem however is that programs using the raw powder diffraction pattern (like EFLECH/INDEX, GAIN) cannot be tested against these benchmarks.
Though there is no flat-cell problem in these selected benchmarks, some program may encounter also difficulties due to the fact that the bethanechol chloride b parameter is quite larger than a and c.
The global note is obtained by adding the 20 results (1, 0 or -1), so that
it will be in the range -20 to +20. Having a global note > 0 is desirable.
You may also propose other benchmarks.
Hyperlinks are given (click on the note 0 or +1 or +1V, no link for -1) to text files containing
the operator name, the input data file and the output file.
def = default run (or automated mode, or black-box mode, whatever);
man = manual run (the user selects more appropriate conditions than the default ones).
____________________________________________________________________________________________________________________
A(1) A(2) B(1) B(2) C(1) C(2) D(1) D(2) E(3) F(4) Note Global Time for
Program def-man def-man def-man def-man def-man def-man def-man def-man def-man def-man def-man note one test
____________________________________________________________________________________________________________________
ITO13 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 +1 +1 +1 -8 -6 -14 VF
DICVOL91 -1 -1 -1 -1 -1 -1 -1 +1 -1 -1 -1 -1 -1 -1 -1 +1 +1 +1 +1 +1 -6 -2 -8 S
TREOR90 -1 -1 -1 -1 -1 -1 +1 +1 -1 -1 -1 -1 -1 -1 +1 +1 +1 +1 +1 +1 -2 -2 -4 F
McMaille -1 06 +1V +1 +1V +1 -1 +1 -1 -1 -1 +1 -1 +1 -1 +1 +1 +1 +1 +1 -2 +7 +5 VS
N-TREOR
DICVOL04 ?
CRYSFIRE(*)
FZON
LOSH
KOHL
TAUP
Mmap
AUTOX
SVD-Index
X-Cell
GAIN
SCANIX
Powder49
Etc
___________________________________________________________________________________________________________________
(1) ICDD PDF entry 43-1748Special conditions of the benchmark runs with the various programs :
(2) ICDD PDF entry 46-1964
A raw data
B weak intensity lines removed
C zeropoint corrected
D weak intensity lines removed and zeropoint corrected
E(3) high resolution new conventional X-ray data
F(4) Synchrotron data
(*) for CRYSFIRE, this is the best result for one or several of the individual programs inside of this package.
Speed : VF = Very Fast (< second)
F = Fast (< few seconds)
S = Slow (< minutes or few minutes)
VS = Very Slow (< hour or few hours)
TREOR90
Default mode: typical conditions:
MONO=130.0,WAVE=1.5418,
CHOICE=3,
END*
0.0
Manual mode: typical conditions:
NIX=0,IDIV=0,MONOSET=7,MONO=130,WAVE=1.5418,
CEM=20.0,D2=0.0006,D1=0.0003,
( or D2=0.0002,D1=0.0001 for the synchrotron data
)
CHOICE=3,
END*
0.0
Comment: local TREOR90 version, the last line is a zeropoint.
ITO13
Default mode: typical conditions:
all default values (blank line except for the wavelenght and zeropoint when
known):
0.6995
Manual mode: typical conditions:
9009 1 1 1
0.6995
3.00 (TOLG tried with values 1, 2, 3,
4, 5, and 6)
Comment: the successfull test E(2) in manual mode is obtained only
with TOLG=1.00, not with 2,3,4,5,6 !
DICVOL91
Default mode: typical first 5 lines in the entry data :
ICDD 43-1748 Benchmark A-1
20 2 1 1 1 1 1 0
0. 0. 0. 0. 0. 0. 0.
1.5418 0. 0. 0.
0. 0. 0.
Manual mode: typical first 5 lines in the entry data :
ICDD 43-1748 Benchmark A-1
20 2 0 0 0 0 1 0
20. 20. 20. 800. 1200. 90. 120.
1.5418 0. 0. 0.
0.05 0. 0.
Comment: The peak position error tolerance is increased to 0.05 instead
of the default value, exploration is made only in monoclinic in a restricted
cell volume range. That version has no tolerance
at all for impurity lines, this explains the negative global note.
McMaille
Default mode: 3 unindexed lines maximum are tolerated (yes it was
only 2 in he previous version of McMaille, but benchmarks are made for improving
software, don't you think so ?). All symmetries examined (in fact stopped
before examining triclinic which was not examined with TREOR and DICVOL as
well). Maximum cell parameters and volumes are respectively 20Å, 2000
Å3 (monoclinic) and 1000 Å3 (triclinic).
No internal system able to find the zeropoint. Typical data preceeding the
list of 2-theta and I values is reduced to the 2 following lines (first line
: title; second line : the wavelength, the zeroshift and the code=3 corresponding
to the automated black-box mode) :
ICDD 46-1964 Indexing Benchmark A-2
1.5418 0.000 3
Manual mode: 8 unindexed lines tolerated, successive runs, the solution
being found in a monoclinic run with volume range 800-1200 Å3,
and 5-20 Å cell parameters, W = either 0.3 or 0.5 (line Width). Typical
conditions for a run in manual mode :
ICDD 43-1748 Benchmark A-1 Manual mode
! Wavelength, zeropoint, Ngrid
1.541800 0.0000 0
! Codes for symmetry
0 0 0 0 1 0
! W, Nind
0.300 8
!Pmin, Pmax, Vmin, Vmax, Rmin, Rmax, Rmaxref
5. 20. 800. 1200. 0.02 0.25 0.50
! Ntests, Nruns
2000000 20
! 2-theta Intensity
6.238000 2.000000
6.712000 10.00000
etc
Comment: why not using these manual mode conditions as the default
mode ? Because the calculation time is really much longer ! Problem with Monte
Carlo programs : different runs may not give the results in the same order.
You should possibly also make several runs for obtaining similar results...
As soon as computer speed will increase by a factor 1000 (will take the next
15 years ??), this McMaille program will be more comfortable ;-)
As expected, with extremely good data (high accuracy, no impurity line,
benchmarks E and F) most programs produce the solution. But when the data
are presenting some problems, then the indexing programs are clearly not equals.