*****************************************************************************
0. Contents of this file
*****************************************************************************
0. Contents of this file
1. Summary of test results
2. Full results
3. Details of the results format
4. Tests that were not included

This file documents the results of the various test suites on various RNGs.  
The test suites used include PractRand, TestU01, a not-yet-releasd version 
of RaBiGeTe, and Dieharder.  The RNGs include over over 100 RNGs I selected 
from a wide variety of categories.  Some were RNGs actually in use, some 
were obscure RNGs, some were RNGs I made up just to have a wider variety.  


*****************************************************************************
1. Summary of results
*****************************************************************************


number of RNGs bias was found in VS test battery VS time
                   10 secs  1 min   5 mins   30 mins   1 hour    4 hours   8 hours   1 day     4 days
PractRand std      43       65      83       104       112       130       136+      142+      147+
TestU01 *Crush     20       --      --       58        --        68        --        --        --
RaBiGeTe ext       19       26      31       40        45        --        --        --        --
gjrand mcp         52       67      90       --        102       --        124       --        134+
Dieharder (custom) 10       14      27       --        --        --        --        --        --
Dieharder (-a)     --       --      --       --        17        --        --        --        --
Times are very rough figures based upon modern desktop x86 CPUs, but using only one core (no multithreading).  
(times were fairly similar on a variety of such CPUs)
All results are based upon a set of 190 RNGs, listed below.  
Some categories of RNGs were omitted - candidate RNGs were not counted, 
nor were "Transforms applied to various low quality RNGs".  

total RNGs per category
category  ALL     recom.  simple  nonLCG  LCGish  cbuf    indirect
number    190     19      50      19      33      29      40

broken down by category
                         recom.  simple  non-LCG LCGish  cbuff  indirect
-- (10 seconds)                                                 
PractRand std 512 MB     0       14      5       7       11     6
TestU01 SmallCrush       0       8       3       7       2      0
RaBiGeTe ext 2 Mb        0       6       4       7       2      0
gjrand mcp --small       0       14      7       8       15     8
Dieharder (custom 10sec) 0       4       4       1       2      0
-- (1 minute)                                                   
PractRand std 4 GB       0       21      5       13      15     11
RaBiGeTe ext 16 Mb       0       7       4       11      4      0
gjrand mcp --standard    0       18      10      12      16     11
Dieharder (custom 1min)  0       6       5       2       2      0
-- (5 minutes)                                                  
PractRand std 16 GB      0       27      6       14      20     16
RaBiGeTe ext 64 Mb       0       9       4       14      4      0
gjrand mcp --big         0       24      11      15      22     18
Dieharder (custom 5min)  0       8       6       2       8      5
-- (30 minutes)                                                 
PractRand std 128 GB     0       35      9       19      21     20
TestU01 Crush            1       22      8       16      6      5
RaBiGeTe ext 256 Mb      0       9       5       19      6      1
-- (1 hour)                                                     
PractRand std 256 GB     1       35      15      20      22     24
RaBiGeTe ext 512 Mb      0       9       8       21      8      2
gjrand mcp --huge        1       32      15      16      23     19
Dieharder -a             0       7       2       4       2      2
-- (4 hours)                                                    
PractRand std 1 TB       1       38      19      24      24     29
TestU01 BigCrush         1       24      13      18      8      7
-- (8 hours)                                                    
PractRand std 2 TB       1       40      20      24+     25     31
gjrand mcp --tera        1       36      16      22      27     27
-- (1 day)                              
PractRand std 4 TB       1       42      20      24+     26     34
-- (4 days)                              
PractRand std 16 TB      1       46      22      24+     26     34
gjrand mcp --ten-tera    1       39      19      23      27     31+

test suites vs features
name                       PractRand    TestU01    RaBiGeTe   gjrand   NIST STS   Dieharder
unlimited test length      YES          no         no         YES?     NO         no
interim results            YES          no         no         no       no         no
multithreading             YES          no         YES        YES?     no         no
scriptable interface       YES          YES        no         YES      no?        YES
minimal false positives    YES          YES        ~almost    YES      NO         NO
cross-platform             YES          ~almost    no         ?        ?          NO
useful & concise output    YES          YES        NO         YES-ish  NO         ~almost

PractRand std:
	The standard test battery of PractRand.  
	The only test suite to allow functionally unlimited test lengths.  
		update: gjrand allows such, though requires custom battery setup past 10 TB
	The only test suite to offer interim results.  
	caveats:
		Cheats slightly on all RNGs by looking at the metadata - it treats 8 bit RNGs slightly differently than 32 bit RNGs, etc
		Requires more bits to find bias than any other test suite (of those listed).  
		May be cheating slightly on the LCGs - it often runs out the periods on those instead of finding bias the normal way.  
		Multithreading is supported, but max speedup tends to be limited to about 3x.  


TestU01 *Crush:
	Meaning SmallCrush, Crush, and BigCrush from TestU01.  
	The only test suite with a real academics in the field behind it.  
	The only test to guarantee (on default settings anyway) that all subtest results are 100% independant.  
		Although I think that only applies to subtest results with distinct numbers.
	caveats:
		Does not support multithreading.  
		More sensitive to RNG speed than the other test suites (of those listed) at default settings.  

RaBiGeTe ext:
	The "extended" preset of RaBiGeTe, but with DFT disabled.  
	(DFT wasn't adding anything to the results and was preventing longer tests from working)
	Can find bias in many RNGs using fewer bits than the other test suites.  
	Supports multithreading better than the other test suites (of those listed).  
	caveats:
		Tended to crash a bit.  
		I was using a pre-release version - the released versions at the time had some issues.  
		I had to ignore the authors instructions and make up my own to get clear & correct results.  
		The recent versions seem to be GUI-only, making it harder to automate.  
		The "30 minute" results actually took about 40 minutes... I was kinda generous there.  


Dieharder (-a):
	Dieharder used with the -a command line option.  
		(the only predefined set of tests in Dieharder, so far as I can tell)
	Very few biases found
	only a few tests showed significant ability to detect bias:
		dab_monobit2 (the best test in Dieharder by far)
		dab_dct (the next best test, I think)
		dab_filltree
		dab_filltree2
		diehard_opso
		diehard_oqso
		dab_bytedistrib
	and many tests had problems:
		Test name             Problems
		marsaglia_tsang_gcd   FALSE POSITIVES, very slow
		sts_serial            FALSE POSITIVES
		rgb_lagged_sum        FALSE POSITIVES
		diehard_dna           very slow
		diehard_sums          broken for metatests
		diehard_runs          broken for metatests
		rgb_bitdist           very slow
		rgb_minimum_distance  very slow
		rgb_kstest            broken for metatests
		("very slow" refers to as parameterized in the "-a" set of tests)
	takes about 40 minutes on my computer on a fast RNG

Dieharder (custom):
	Used some shell scripts to run Dieharder with a custom arrangement of tests
	Used several sets of tests & parameters for various time increments:
		tests:     monobit2    dct       filltree2 filltree bytedistrib
		10 second: 100000000   100000    --        --       --
		1 minute:  500000000   500000    5000000   15000000 51200000
		5 minutes: 2000000000  3000000   48000000  48000000 512000000
	(all of those test names have the prefix "dab_" omitted)
	Unfortunately, Dieharder does not permit test lengths longer than about 
	2 billion, so dab_monobit2 cannot increase beyond what's used in the 5 
	minute version there.  Increasing the rest of the tests lengths produces 
	very little return on the investment, so there's not much point to a run 
	longer than 5 minutes.  
	As an example, the command lines I use for the 10 second test on the RNG 
	named ranrot32 is:
		./RNG_output ranrot32 name
		./RNG_output ranrot32 123456789000000 | dieharder -g 200 -D 504 -D 16384 -d dab_monobit2 -t 100000000
		./RNG_output ranrot32 123456789000000 | dieharder -g 200 -D 504 -D 16384 -d dab_dct -t 100000

NIST STS (Statistical Test Suite)
	I've made a few modifications to improve performance and allow it to 
	work on larger samples than it could normally.  
	It's producing false positives for me on the non-overlapping template 
	test, when using larger sample sizes and larger numbers of samples.  At 
	128 samples the largest sample size I can safely use on that test is 8 
	megabits.  
	It seems to strongly encourage the use of large numbers of small 
	samples - its summary report will actually avoid listing individual 
	test p-values even if only one sample was run, instead showing only 
	number of test p-values within the normal range and a meta-test p-value.  
	Possibly it doesn't trust its basic p-value quality enough to show them, 
	or possibly it's just a brain-damaged interface.  
	It is extremely inefficient.  It likes to unpack binary data to 1 bit 
	per byte, among other inefficiencies.  It is also extremely weak, with 
	very poor efficiency, poor sensivity, and poor or very poor breadth.  

gjrand mcp / pmcp
	When limited to 10 seconds, I use mcp --small, for 1 minute I use 
	mcp --standard, for 5 minutes mcp --big, for half an hour... there's no 
	good option for that one, for 1 hour I use mcgp --huge, for 4 hours...
	there's no good option for that, for 8 hours I use --tera (I added the 
	8 hour slot primarily for gjrand) and there's also --ten-tera but that 
	takes half a week and --tiny which takes ~3 seconds, neither of which 
	match any of my time slots well.  (he used factors of 10 between each 
	test level, where I used factors of 2).  I think it supports custom 
	test lengths, but since there were sufficient variety in the named 
	test lengths I stuck to those.  pmcp is functionally multithreaded out 
	of the box, I think.  The author recommends using the overall p-value 
	for primary evaluation, but with my current skills with grep it's 
	easier for me to use the "grade (XX) failure (SERIOUSLY BAD)" messages 
	instead, which work about as well.  Plus there is the extra information 
	content that way of the number of tests failed.  
	This seems to be overall the best PRNG test suite out there aside from 
	PractRand.  In fact, it's better than many versions of PractRand, and 
	part of the reason PractRand has improved is because of gjrand - the 
	idea used for PractRand's 'mod3n' test was based upon the mod3 test in 
	gjrand.  
	In terms of interface, IMHO it's not as good as PractRand or TestU01, 
	but better than anything else.  The final summary is extremely readable 
	but missing the identity of which tests it failed (a relativeely high 
	priority datum I think).  The individual test results are too verbose, 
	printing out numbers that have no meaning to those who don't know how 
	the tests work internally.  The documentation fails to cover a few too 
	many areas.  
	The "slightly bad" result evaluation shows false positives occasionally, 
	but the "SERIOUSLY BAD" never has so far, though I suspect it will 
	sooner or later based upon some of the p-values I've seen it flunk.  
	Overall it's an awesome set of tests with a reasonable interface.  My 
	biggest complaint is that it silently fails, producing garbage results, 
	if multiple instances of the test suite are run simultaneously.  

	From its output, I'm currently counting "grade 10 failures" and up as 
	failure, and grade 4 to grade 9 failures as suspicious.  I run gjrand 
	mcp or pmcp once for each test length from --small to --tera, in 
	order, and if there are no failures or suspicious results I put a - 
	in the gjrand results column (from left to right, matching the 
	test length).  If there are no failures, but there are suspicious 
	results, then I put a ~ in the results column.  If there are 
	failures, I count them and put the number in the results column (in 
	hexadecimal, so that I can record numbers beyond 9 without throwing 
	off the formating).  Thus a gjrand result listed as ---~3 means that 
	mcp or pmcp reported nothing suspicious at --small, --standard, 
	or --big, zero failing results but one or more suspicious results at 
	--huge, and three failing results at --tera.  If there were no 
	failures at --tera, and I have spare CPU time, I then run it at 
	--ten-tera to add a final result.  



*****************************************************************************
2. Full results
*****************************************************************************


recommended RNGs
	RNG             Quality   PractRand   TestU01    RaBiGeTe   gjrand      others
	                rating    standard    *Crush     Extended   mcp
	hc256           5         > 1 TB      pass       pass       -----       -
	trivium         5         > 4 TB      pass       pass       -----       -
	efiix64x384     5         > 1 TB      pass?      pass?      -----       -
	efiix32x384     5         > 2 TB      pass       pass       -----       -
	efiix16x384     5         > 1 TB      pass       pass       -----       -
	efiix8x384      5         > 2 TB      pass       pass       -----       -
	isaac64x256     5         > 1 TB      pass?      pass?      -----       -
	isaac32x256     5         > 4 TB      pass       pass       -----       -
	chacha(8+)      5         > 4 TB      pass?      pass?      -----       -
	salsa(8+)       5+        > 4 TB      pass?      pass?      -----       -
	arbee           4         > 1 TB      pass?      pass?      -----       -
	xsm64           4         > 16 TB     pass?      pass?      -----       -
	xsm32           3         > 8 TB      pass       pass       ------      -
	jsf64           3         > 1 TB      pass?      pass?      -----       -
	jsf32           3         > 1 TB      pass       pass       ------      -
	sfc64           3         > 1 TB      pass?      pass?      -----       -
	sfc32           3         > 1 TB      pass       pass       ------      -
	sfc16           2         512 TB      pass       pass       ------      -
	rarns16         3         ?
	mt19937         2         256 GB      0/2/2      pass       ---111      -
(usually "pass?" means that a lower quality variant passed so I didn't fully test the higher quality variant)

other RNGs - simple (49)
	RNG               Quality     PractRand   TestU01     RaBiGeTe    gjrand      others
	                  subscores   standard    *Crush      Extended*1  mcp
*	xorshift32        0/0/0/0     64 KB       5/53/?      64 Kb       579CC       DHA0, STS
*	xorshift64        0/4/2/0     64 KB       1/25/?      128 Kb      55556       DHA0, STS
	xorshift32of128   0/5/3/1     512 KB      0/6/?       1 Mb        11111       -
	xoroshiro128plus  0/5/3/1     32 MB       pass        pass        --111       -
*	xorwow32of96      0/5/3/1     2 MB        0/3/5       64 Mb       11344       -
	xorwow32x6        1/5/4/1     16 MB       0/1/2       pass        -1112       -
	xsalta16x3        2/1/1/1     1 MB        0/1/3       pass        11111       -
	xsaltb16x3        2/1/1/1     2 GB        pass        pass        --135       -
*	xsaltc16x3        1/1/1/1     2 GB        0/1/1       pass        --124       -
*	sapparot          0/2/2/1     16 MB       0/0/1       64 Mb       22345       -
*	sap16of48         1/1/1/1     512 KB      0/3/2       1 Mb        11124       -
	sap32of96         4/3/3/1     2 MB        pass        4 Mb        11112       -
*	flea32x1          0/4/3/2     1 MB        pass        pass        22245       -
	jsf16             4/2/2/2     16 TB       pass        pass        ------      -
*	sfc_v1_16         1/1/1/1     8 GB        1/7/?       pass        -1124       DHA1, STS
*	sfc_v1_32         1/3/2/1     8 TB        0/2/7       pass        ---12       -
*	sfc_v2_16         0/1/1/1     2 GB        0/1/0       pass        ---23       DH2
.	sfc_v2_32         2/3/3/2     4 TB        0/0/1       pass        ----1       -
	sfc_v3_16         2/1/1/1     512 GB      pass        pass        -----~      -
	sfc_v3_32         5/3/3/2     > 16 TB     pass        pass        ------      -
*	simpleA           1/3/3/1     2 GB        pass        pass        12334       DH2
*	simpleB           0/1/1/0     128 MB      1/6/2       pass        ---33       DH1
	simpleC           0/1/1/1     16 GB       1/6/2       pass        ---~1       -
*	simpleD           2/3/3/2     128 GB      pass        pass        -1233       -
*	simpleE           3/3/3/2     1 TB        pass        pass        -~113       -
*	simpleF           2/2/2/1     16 GB       pass        pass        ---12       -
*	simpleG           1/3/3/2     128 GB      0/18/?      pass        --255       DHA
	trivium_weakenedA 4/4/3/3     > 16 TB     pass        pass        ------      -
	trivium_weakenedB 3/3/3/3     1 TB        pass        pass        ----4       -
*	mo_Lesr32         0/0/0/0     64 KB       7/?/?       256 Kb      888AD       DHA0, STS1
*	mo_ResrRers32     4/2/2/2     2 TB        pass        pass        ----~1      -
*	mo_Rers32of64     0/2/2/0     8 MB        0/10/?      16 Kb       44455       DHA0
*	mo_Resdra32of64   1/2/2/1     4 GB        0/2/6       pass        -~-5C       -
*	murmlacish        3/3/3/2     4 TB        0/1/0       pass        ----2       -
	gjishA            ?/?/?/?     32 GB       pass        pass        ---12       -
	gjishB            ?/?/?/?     > 16 TB     pass        pass        ------      -
	gjishC            ?/?/?/?     64 GB       0/1/4       pass        --~11       DHA
	gjishD            ?/?/?/?     128 GB      pass        pass        --~12       -
	ara16             2/1/1/1     1 GB        1/3/2       pass        11111       -
	ara32             4/3/3/1     2 GB        pass        pass        11111       -
	arx16             1/1/1/1     16 GB       1/3/2       pass        -1122       -
	arx32             4/3/3/2     2 TB        pass        pass        ------      -
	hara16            2/1/1/2     64 GB       pass        pass        ~-----      -
	harx16            2/1/1/2     128 GB      pass        pass        ----~2      -
	learx16           2/1/1/2     128 GB      pass        pass        --1112      -
	hlearx16          1/1/1/2     16 GB       pass        pass        ---~~2      -
	alearx16          4/1/1/3     16 TB       pass        pass        ------      -
	arac16            1/2/2/2     8 GB        0/1/1       pass        ------      -
	arxc16            5/2/2/3     > 16 TB     pass        pass        ------      -
	*** 10 sec                    14 (512 MB) 8 (SC)      6 (2 Mb)    14 (small)
	*** 1 minute                  21 (4 GB)   --          7 (16 Mb)   18 (standard)
	*** 5 minutes                 27 (16 GB)  --          9 (64 Mb)   24 (big)
	*** 30 minutes                35 (128 GB) 22 (C)      9 (256 Mb)  --
	*** 1 hour                    35 (256 GB) --          9 (512 Mb)  32 (huge)
	*** 4 hours                   38 (1 TB)   24 (BC)     --          --
	*** 8 hours                   40 (2 TB)   --          --          36 (tera)
	*** 1 day                     42 (4 TB)   --          --          --
	*** 4 days                    45 (16 TB)  --          --          39 (ten-tera)

other RNGs - non-LCG multiplication based
	RNG             Quality     PractRand   TestU01     RaBiGeTe    gjrand      others
	                subscores   standard    *Crush      Extended*1  mcp
	garthy16        0/1/1/1     64 MB       0/15/?      256 Mb      128CC       DH0
	garthy32        2/3/3/1     1 TB        pass        pass        --133       -
	binarymult16    0/3/3/1     512 KB      1/31/2      1 Mb        2248D       DHA0
	binarymult32    2/4/3/2     2 MB        pass        1 Mb        11224       DH1
	rxmult16        0/2/2/1     128 GB      ~pass       pass        ------      -
	multish3x32     1/3/3/2     16 GB       0/0/1       pass        11111       -
	multish4x16     1/2/2/1     512 KB      0/2/1       1 Mb        11111       -
	mwrca16         0/0/0/0     16 MB       1/4/10      128 Mb      ~23AC       DH2
	mwrca32         4/2/2/1     8 TB        pass        pass        -----1      -
	mwrcc16         0/1/1/1     128 MB      0/2/7       32 Mb       12447       DH0
	mwrcc32         2/3/3/2     64 GB       0/0/1       16 Mb       13345       -
	mwrcca16        0/1/1/1     512 MB      pass        pass        ---~4       -
	mwrcca32        4/3/3/2     > 32 TB     pass        pass        -----1      -
	old_mwlac16     2/2/2/2     256 GB      pass        pass        ------      -
	mwlac_varA      ?/1/1/?     32 GB       0/1/1       pass        -111~4      -
	mwlac_varB      ?/1/1/?     1 TB        0/2/2       pass        -----4      -
	mwlac_varC      ?/1/1/?     64 GB       ~pass       pass        -----2      -
	mwlac_varD      ?/1/1/?     16 TB       1/3/4       pass        11111       -
	mwlac_varE      ?/1/1/?     2 TB        pass        pass        ------      -
	mo_Cmfr32       0/0/0/0     64 KB       6/58/?      64 Kb       58ABD       DHA0, STS
	mo_Cmr32of64    3/2/2/2     512 GB      0/0/1       pass        -1111       -
	mulcr16         ?/1/1/2     512 GB      ~pass       pass        -11--4      -
	mulcr32         ?/3/3/?     > 32 TB     pass        ~pass       ------      -
	mmr16           ?/1/1/1     32 GB       0/1/1       pass        -----3      -
	mmr32           ?/3/3/3     > 16 TB     pass        pass        11111       -
	*** 10 sec                  8 (512 MB)  4 (SC)      4 (2 Mb)    10 (small)
	*** 1 minute                8 (4 GB)    --          5 (16 Mb)   14 (standard)
	*** 5 minutes               9 (16 GB)   --          6 (64 Mb)   15 (big)
	*** 30 minutes              14 (128 GB) 10 (C)      8 (256 Mb)  --
	*** 1 hour                  15 (256 GB) --          8 (512 Mb)  ~15 (huge)
	*** 4 hours                 19 (1 TB)   13 (BC)     --          --
	*** 8 hours                 20 (2 TB)   --          --          ~16 (tera)
	*** 1 day                   20 (4 TB)   --          --          --
	*** 4 days                  22 (16 TB)  --          --          21 (ten-tera)

other RNGs - LCG-based RNGs
	RNG               Quality     PractRand   TestU01     RaBiGeTe    gjrand      others
	                  subscores   standard    *Crush      Extended*1  mcp
	lcg(16,32)        0/0/0/0     512 KB      6/48/??     32 Kb       7ACBB       DHA1,STS
	lcg(16,40)        0/1/1/0     32 MB       1/29/??     128 Kb      3559B       -
	lcg(16,48)        0/1/1/1     32 MB       1/7/??      512 Kb      23334       -
	lcg(16,56)        0/2/1/1     128 MB      1/6/??      2 Mb        11223       -
	lcg(16,64)        0/4/2/2     1 GB        0/3/4       4 Mb        11112       -
	lcg(16,72)        1/4/2/2     4 GB        pass        8 Mb        -1111       -
	lcg(16,80)        2/5/2/2     32 GB       pass?       16 Mb       --111       -
	lcg(16,96)        4/5/3/2     1 TB        pass?       256 Mb      ------      -
	lcg(16,112)       4/5/3/2     > 1 TB      pass?       pass        -----_      -
	xlcg(16,32)       0/0/0/0     512 KB      10/?/?      2 Mb        59BBB       DHA0,STS
	xlcg(16,40)       0/1/1/0     64 MB       1/11/?      128 Mb      -238B       -
	xlcg(16,48)       1/1/1/1     8 GB        0/2/3       pass        ---24       -
	xlcg(16,56)       2/2/1/1     64 GB       pass        pass?       ----2       -
	xlcg(16,64)       3/4/2/2     128 GB      pass        pass?       ----1       -
	xlcg(16,72)       3/4/2/2     256 GB      pass?       pass?       ------      -
	xlcg(16,80)       ?/5/2/2     > 1 TB      pass?       pass?       ______      -
	clcg(16,64)       1/3/2/1     1 GB        0/4/?       64 Mb       -1323       -
	clcg(16,68)       1/4/2/1     32 GB       0/2/3       128 Mb      --1-2       -
	clcg(16,72)       1/4/2/2     256 GB      0/1/1       256 Mb      ---11       -
	clcg(16,76)       1/4/2/2     1 TB        pass        512 Mb      ----1       -
	clcg(16,80)       4/4/2/2     > 1 TB      pass        512 Mb      ----1       -
	clcg(16,84)       ?/4/2/2     > 16 TB?    pass?       pass        ------      -
	cxlcg(16,64)      0/3/2/1     1 GB        0/1/2       64 Mb       --122       -
	cxlcg(16,68)      2/4/2/1     64 GB       pass        256 Mb      ----2       -
	cxlcg(16,72)      4/4/2/2     1 TB        pass        pass        -----1      -
	bblcg(32,160,32)  ?/5/3/0     1 MB        0/21/??     2 Mb        44555       DHA
	bblcg(32,192,32)  ?/5/4/1     2 GB        0/11/??     4 Mb        -1144       DHA
	bblcg(32,224,32)  ?/5/4/1     1 TB        0/2/5       64 Mb       ------      -
?	bblcg(32,256,32)  ?/5/5/2     > 1 TB      0/0/1       ~pass       -----_      -
?	bblcg(32,288,32)  ?/5/5/2     > 16 TB?    pass        pass        -----_      -
	pcg32_norot       1/4/2/2     2 GB        0/1/2       8 Mb        ----1       -
?	pcg32             5/4/2/3     > 4 TB      pass        pass        ------      -
?	cmrg32of192       5/5/4/3     > 1 TB      pass        pass        ------      -
?	xsh_lcg_bad       4/5/5/2     > 1 TB      pass        32 Mb       ------      -
	*** 10 sec                    7 (512 MB)  7 (SC)      7 (2 Mb)    8 (small)
	*** 1 minute                  13 (4 GB)   --          12 (16 Mb)  12 (standard)
	*** 5 minutes                 14 (16 GB)  --          16 (64 Mb)  15 (big)
	*** 30 minutes                19 (128 GB) 17 (C)      21 (256 Mb) --
	*** 1 hour                    20 (256 GB) --          23 (512 Mb) 16 (huge)
	*** 4 hours                   24 (1 TB)   18 (BC)     --          --
	*** 8 hours                   24+(2 TB)   --          --          22 (tera)
	*** 1 day                     24+(4 TB)   --          --          --
	*** 4 days                    24+(16 TB)  --          --          23 (ten-tera)
	notes on the meaning of *lcg names:
		lcg(X,Y) means an LCG with modulus 2^Y, discarding (Y-X) low bits to produce X bit output
		clcg(X,Y) means lcg(X,Y-32) combined with a 32 bit non-power-of-2-modulus LCG
		xlcg(X,Y) is similar to lcg(X,Y), but xor is used in place of addition
			(i.e. equivalent to suppresing the carries on the addition in binary)
		cxlcg(X,Y) means xlcg(X,Y-32) combined with a 32 bit non-power-of-2-modulus LCG
		bigbadlcg(X,Y,Z) aka bblcg(X,Y,Z) means an LCG with modulus 2^Y, discarding (Y-X) low 
			bits to produce X bit output, with a low-quality multiplier of (1 + 2^Z)

other RNGs - cyclic buffer based (or other static structures)
	RNG               Quality     PractRand   TestU01     RaBiGeTe    gjrand      others
	                  subscores   standard    *Crush      Extended*1  mcp
	mm32              0/5/5/0     2 MB        1/8/12      pass        45678       DH2
	mm32_awc          0/5/5/1     256 MB      0/8/11      pass?       34567       DH2
	mm16of32          0/5/5/0     1 GB        0/0/2       pass?       44555       DH2
	mm16of32_awc      0/5/5/1     2 GB        0/0/1       pass?       44555       DH2
	mwc4691           1/5/5/1     2 GB        pass        pass        -1111       -
	cbuf_accum        0/5/5/2     128 MB      pass        pass        14444       -
	cbuf_accum_big    0/5/5/2     8 GB        ~pass       pass?       --214       -
	cbuf_2accum_small 0/5/4/2     512 GB      pass        pass        --~22       -
	cbuf_2accum       0/5/5/2     64 TB       pass        pass?       ------      -
	dual_cbuf_small   0/5/5/2     4 MB        pass        16 Mb       12333       -
	dual_cbuf         0/5/5/3     128 MB      pass?       pass        --112       -
	dual_cbufa_small  0/5/5/2     4 MB        pass        16 Mb       11112       -
	dual_cbuf_accum   0/5/5/2     128 MB      pass?       pass        --1112      -
	fibmul16of32      1/5/5/1     16 GB       pass        pass        --124       -
	fibmul32of64      3/5/5/2     512 GB      pass        pass        ----1       -
	fibmulmix16       ?/5/5/2     2 GB        pass        pass        11345       -
	fibmulmix32       ?/5/5/3     64 GB       pass?       pass?       1111        -
	ranrot32small     ?/5/5/2     256 MB      0/1/0       256 Mb      44445       DHA0
	ranrot32          2/5/5/2     2 GB        pass        pass        12222       -
	ranrot32big       3/5/5/2     16 GB       pass?       pass?       ----2       -
	ranrot3tap32small ?/5/5/2     512 MB      pass        pass        22444       -
	ranrot3tap32      3/5/5/3     256 GB      pass        pass        --222       -
	ranrot3tap32big   4/5/5/3     4 TB        pass?       pass?       --~-2       -
	ranrot32hetsmall  4/5/5/3     8 GB        pass        pass        11234       -
	ranrot32het       4/5/5/3     2 TB        pass        pass?       -~123       -
	ranrot32hetbig    4/5/5/3     64 TB       pass?       pass?       ----1       -
	mt19937_unhashed  2/5/5/1     32 GB       0/2/2       pass        11122       -
	salsa(3 rounds)   0/5/5/0     4 KB        13/??/??    16 Kb       BBBCC       DHA0, STS
	chacha(3 rounds)  0/5/5/1     32 MB       0/6/28      2 Mb        36999       DH2, STS
	salsa(4 rounds)   1/5/5/2     16 GB       ~pass       256 Mb      11257       DH2
	chacha(4 rounds)  5/5/5/3     > 16 TB     pass        pass        ------      -
	*** 10 sec                    11 (512 MB) 2 (SC)      2 (2 Mb)    17 (small)
	*** 1 minute                  16 (4 GB)   --          4 (16 Mb)   18 (standard)
	*** 5 minutes                 21 (16 GB)  --          4 (64 MB)   24 (big)
	*** 30 minutes                23 (128 GB) 6 (C)       6 (256 Mb)  --
	*** 1 hour                    24 (256 GB) --          6 (512 Mb)  25 (huge)
	*** 4 hours                   26 (1 TB)   8 (BC)      --          --
	*** 8 hours                   27 (2 TB)   --          --          29 (tera)
	*** 1 day                     28 (4 TB)   --          --          --
	*** 4 days                    28 (16 TB)  --          --          29 (ten-tera)

other RNGs - indirection based
	RNG             Quality     PractRand   TestU01    RaBiGeTe    gjrand      others
	                subscores   standard    *Crush     Extended*1  mcp
	ibaa8x2         0/0/0/1     512 MB      0/8/19     512 Mb      -~8CC       DH2
	ibaa8x4         1/1/1/2     8 GB        pass       pass        --~12       -
	ibaa8x8         4/3/2/3     1 TB        pass       pass        -----1      -
	ibaa16x2        2/2/2/2     2 GB        pass       pass        -1123       -
	ibaa16x4        4/3/3/2     256 GB      pass?      pass        ----1       -
	ibaa16x8        5/3/3/3     > 16 TB     pass?      pass?       ------      -
	ibaa32x2        4/3/3/2     2 GB        pass       pass        11111       -
	ibaa32x4        4/3/3/2     256 GB      pass?      pass?       --111       -
	ibaa32x8        4/5/5/3     > 16 TB?    pass?      pass?       ----1       -
	rc4_weakenedA   1/5/5/2     8 GB        0/1/1      pass        12226       -
	rc4_weakenedB   2/5/5/2     256 GB      pass       pass        ---~1       -
	rc4_weakenedC   1/5/5/2     16 MB       0/27/?     128 Mb      15579       DHA
	rc4_weakenedD   1/5/5/2     64 GB       pass       pass        ----~4      -
	rc4             4/5/5/3     1 TB        pass       pass        -----2      -
	isaac16x4       0/4/3/2     4 MB        0/0/1      pass        ---14       -
	isaac16x8       0/5/4/3     1 GB        pass       pass?       -----1      -
	isaac16x16      4/5/5/4     2 TB        pass?      pass?       ------      -
	isaac32x4       0/5/4/2     8 MB        pass       pass        11112       -
	isaac32x8       0/5/5/3     256 MB      pass       pass?       --111       -
	isaac32x16      4/5/5/4     4 TB        pass?      pass?       -----1      -
	genindA(5)      0/5/5/1     128 MB      0/0/1      pass        25567       -
	genindA(7)      3/5/5/2     256 GB      pass?      pass?       12334       -
	genindA(9)      4/5/5/3     64 TB       pass?      pass?       ------      -
	genindB(1)      0/0/2/0     1 GB        0/52/?     pass        ~~CCC       DHA2
	genindB(2)      1/1/3/1     16 GB       pass       pass?       ---4C       -
	genindB(3)      1/3/4/2     512 GB      pass?      pass?       ----2       -
	genindB(4)      4/4/5/2     4 TB        pass?      pass?       -----2      -
	genindC(2)      1/3/3/1     2 GB        0/2/2      pass        -3444       DH2
	genindC(3)      2/4/3/2     64 GB       pass       pass?       11344       -
	genindC(4)      4/5/5/2     4 TB        pass?      pass?       ----1       -
	genindC(5)      4/5/5/3     > 16 TB     pass?      pass?       -----1      -
	genindD(6)      1/5/5/2     32 GB       pass       pass        -1123       DH2
	genindD(9)      3/5/5/3     2 TB        pass?      pass?       ~~-~1       -
	genindE(1)      2/1/3/1     8 GB        pass       pass        ---~3       -
	genindE(2)      3/2/3/2     512 GB      pass?      pass?       -~--1       -
	genindE(3)      4/5/5/3     64 TB       pass?      pass?       -~----      -
	genindF(2)      1/2/3/2     16 GB       *pass      pass        --247       DH2
	genindF(3)      2/5/5/2     128 GB      pass       pass?       --~24       -
	genindF(4)      3/5/5/3     1 TB        pass?      pass?       ----~3      -
	genindF(5)      4/5/5/3     32 TB       pass?      pass?       -----~      -
	*** 10 sec                  6 (512 MB)  0 (SC)     0 (2 Mb)    7 (small)
	*** 1 minute                11 (4 GB)   --         0 (16 Mb)   10 (standard)
	*** 5 minutes               16 (16 GB)  --         0 (64 Mb)   15 (big)
	*** 30 minutes              20 (128 GB) 5 (C)      1 (256 Mb)  --
	*** 1 hour                  24 (256 GB) --         2 (512 Mb)  19 (huge)
	*** 4 hours                 29 (1 TB)   7 (BC)     --          --
	*** 8 hours                 31 (2 TB)   --         --          27 (tera)
	*** 1 day                   34 (4 TB)   --         --          --
	*** 4 days                  34 (16 TB)  --         --          34 (ten-tera)
	note: I wrote a few generic indirection-based PRNGs so there's more variety here.
	They're generally pretty mediocre.  To keep quality low enough to find flaws 
	in I mostly kept their accumulator / mixing pool down to a single word, and 
	limited their word size to 16 bits.  To determine the size of their indirection 
	arrays I gave them each a parameter that is the log2 of that size, and started 
	it at the lowest value to fail at least 3 different test batteries (usually 
	PractRand and gjrand make up two of those three, and the third is either 
	TestU01 or Dieharder).  If that resulted in a size too small for indirection 
	to occur then the PRNG was discarded.  From there I increased the 
	log2-of-array-size parameter by 50% of the lowest value of that parameter, 
	rounded down to a minimum of 1, and kept increasing it by that amount until it 
	looked likely to pass all test batteries at the next size.  


*****************************************************************************
3. Details of the results format
*****************************************************************************

columns:
	RNG
		The name of the RNG algorithm used;
		The name used is the name it is called in PractRand, not necessarily 
		its real full name - for instance, hc256 instead of HC-256.  
	PractRand standard
		The PractRand standard tests are those returned by 
		PractRand::Tests::Batteries::get_standard_tests().  
		quirks:
			test set is (slightly) dependent upon RNG metadata
		speed on a fast RNG on my computer (in single-threaded mode):
			1 GB: ~15 seconds
			1 TB: ~4 hours
		format:
			The length of sequence required for the PractRand standard tests 
			to detect bias.  If multiple seeds were tested, and bias was 
			found in them at different sequence lengths, then the median 
			result is used.  If bias was not found then the length of the 
			longest sequence tested is recorded, with a ">" prefixed in 
			front of it.  
			Bias is considered to have been found when the standard tool 
			rates the result as "FAIL" (with or without exclamation marks).  
			(p-value threshold vary with circumstances but are typically 
			near 1e-10 on the low side and 1 - 1e-10 on the high side)
			The length is recorded as a number followed by KB, MB, GB, or 
			TB.  Those stand for kilobyte, megabyte, gigabyte, or terabyte 
			respectively.  
			All lengths are rounded up to the nearest power of 2.  
	TestU01 *Crush
		This refers to the batteries "SmallCrush", "Crush", and "BigCrush" 
		in TestU01.
		quirks:
			does not accept a sequence length parameter
			works on RNGs outputing 31 bits per call:
				on 32 bit RNGs it works but disregards the lowest bit?!?
				on 8/16 bit RNGs, I concatonate multiple outputs together to produce a 32 bit output
				on 64 bit RNGs... I am currently discarding the upper 32 bits to produce a 32 bit output
					might, in the future, change that to returning half and buffering half for the next call
			does not print any results until all are ready
		speed on a fast RNG on my computer:
			SmallCrush: ~8 seconds
			Crush:      ~30 minutes
			BigCrush:   ~4 hours
		format:
			If all 3 are passed then the result is recorded as "pass".  
			If all 3 are passed, but only barely, the result is recorded as "~pass".
			If all 3 are passed, but it is noticed that bit-reversed versions of the PRNG fail one or more, 
				the result is recorded as "*pass" (note that this case is not checked systematically)
			Otherwise, the result is recorded as X/Y/Z, where X is the 
			number of tests failed in SmallCrush, Y is the number failed in 
			Crush, and Z is the number failed in BigCrush.  
			Failure is defined as a p-value <= 1.0e-10 or >= (1 - 1.0e-10)
	RaBiGeTe extended
		This refers to the preset testing parameters named "extended", but see the quirks section.  
		quirks:
			In order to get 512 Mb samples to run on extended I had to disable the DFT test
			I used a more recent version of RaBiGeTe that has not been publicly released at this time.  
			I used RaBiGeTe in a slightly non-standard way, testing single long samples instead of many short samples.  
				In my testing this method worked much better than the recommended way.  
			It crashed several times during testing.  
		speed on a fast RNG on my computer:
			16 Mb: just over 1 minute
			512 Mb: a little under 40 minutes
		format:
			The same format was used as for PractRand standard, with three exceptions.  
			A result was considered a failure if any p-value was <= 1e-10, or if 5 or 
			more p-values were <= 0.000001 or >= 0.999999, or if any p-values were 
			omitted from the results.  I never consider a single p-value close to one  
			a failure in RaBiGeTe even if it is exactly 1.0.  
			The second exception is that lengths are listed in bits instead of bytes.  
			The third exception is that if an RNG passed 512 megabits of testing then it was listed as "pass".  
				... because RaBiGeTe was unable to test sequences longer than 512 Mb on the settings I used.  

	If a result is listed as "?" that means that that test has not been run 
	on that RNG.  If a result has a normal value followed by a "?" that 
	means that either that result is strongly implied by tests on nearly 
	equivalent RNGs, or that the test was run and is believed to have 
	produced that result but the record-keeping involved was incomplete and 
	I can't be sure without re-running the test.  


*****************************************************************************
4. Tests that were not included
*****************************************************************************

tests that were not included:
	PractRand expanded standard
		This test battery was not used because:
			A. It significantly underperforms PractRand standard on fast RNGs.  
			B. If I listed its test results here I'd have to update them 
				every time I added a new test to it.  
	TestU01 Rabbit
		This test battery was not used because:
			A. It consistently crashes if used on a long sequence
			B. Its time to complete a test does not increase linearly with 
				sequence length - in some cases it actually decreases with 
				length.  This appears to be partially caused by the test 
				set being a function of the testing length.  
			C. Its results do not seem to improve in power with sequence 
				length consistently enough.  
	TestU01 Pseudo-diehard
		This test battery was not used because its quality is too low.  
	NIST RNG tests
		This test battery was not used because:
			A. Severe difficulty with longer test runs due to not only 
				keeping the whole sequence in memory, but also 
				inflating the sequence to store only one bit per byte.  
			B. Poor quality relative to speed, poor maximum quality 
				given the short maximum sample lengths.  


*****************************************************************************
