This notebook explores some simple one - dimensional examples cases of the general procedure described in: https://arxiv.org/abs/2111.05442

First we create some models (our name for toy MCs) which either do or don' t generate data which is translation invariant on the torus. Histograms of data produced with these models will shortly be displayed.

unifModel[]:=RandomReal[]

lowModel[]:=RandomReal[]^1.3

sineModel[]:=Module[{x=RandomReal[],δ=0.1},
(While[RandomReal[]>(δ Cos[3 2π x]+1)/(1+δ),x=RandomReal[]];x)
]

genUnfilteredDatum[model_]:=model[]

Create some filters (inefficiencies in the detector, etc)

testFilter[x_]:=Which[x>0.2 && x<0.4, 0/10, x>0.8 && x<0.9, 2/10, True, 1]

nullFilter[x_]:=1

genFilteredDatum[model_,fil_]:=Module[{x=genUnfilteredDatum[model]},
(While[
RandomReal[]>fil[x],
x=genUnfilteredDatum[model]
];x)
]

(* Here is a function that randomly transforms an object in a way such that p(a->b) = p(b->a) and that would leave the uniform distibution on [0,1] invariant *) 
wideTransform[x_]:=Mod[x+RandomReal[],1];
narrowTransform[x_]:=Mod[x+RandomReal[{-1/4,1/4}],1];
veryNarrowTransform[x_]:=Mod[x+RandomReal[{-1/100,1/100}],1];

Some infrastructure ...

genData[n_,model_,fil_]:=Table[genFilteredDatum[model,fil],{i,1,n}]

genFilteredDatumPair[model_,fil_,tfm_]:=Module[{x=genFilteredDatum[model,fil],y},
(y=tfm[x];
While[
RandomReal[]>fil[y],
( x=genFilteredDatum[model,fil] ; y=tfm[x])
];{x,y})
]

genFilteredDatumPairs[n_,model_,fil_,tfm_]:=Table[genFilteredDatumPair[model,fil,tfm],{i,1,n}]

Finally: here is some data generated from the "uniform" model :

Histogram[genData[10000,unifModel,nullFilter]]

Here is some data generated from a non-uniform model called the “lowModel” (because it is biased low) :

Histogram[genData[10000,lowModel,nullFilter]]

Here is some data generated from a non-uniform model called the “sineModel” (because it has a sine or cos-like variation) :

Histogram[genData[100000,sineModel,nullFilter],50]

Here is some data generated from the "uniform" model but taking our filter into account:

Histogram[genData[10000,unifModel,testFilter]]

Here is some data generated from the "lowModel” taking our filter into account:

Histogram[genData[10000,lowModel,testFilter]]

Here is some data generated from the "sineModel” taking our filter into account:

Histogram[genData[10000,sineModel,testFilter]]

"filtered datum pairs" are what you get by generating a data point x, filtering it, then transforming x to y,  then applying the filter to the transformed point y . Here we  throw the entire pair (x,y) away if the transformed event “y” fails the filter . In a more nuanced approach the event - pair could be retained but weighted according to the filter probability. Note that “y” is called “ x’ ” in the paper.

Check that uniform model’s  filtered (x) and filtered-transformed-filtered (y) distributions are THE SAME AS EACH OTHER (which is the sign of symmetry preservation in the original) when we use (say) the narrow transform:

Histogram[(genFilteredDatumPairs[100000,unifModel,testFilter,narrowTransform]//Transpose),50]

Check that uniform model’s x and y distributions are also THE SAME AS EACH OTHER (which is the sign of symmetry preservation in the original) when we use (say) the wide transform:

Histogram[(genFilteredDatumPairs[100000,unifModel,testFilter,wideTransform]//Transpose),50]

Check that low model x and y distributions are DIFFERENT TO EACH OTHER (which is the sign of symmetry violation in the original) when we use (say) the wide transform:

Histogram[(genFilteredDatumPairs[100000,lowModel,testFilter,wideTransform]//Transpose),50]

Let' s take the (log of the) ratio of the yellow histogram above to the blue histogram above and call it "betterZetaForLowWide" . This is only something we are generating for illustrative purposes in this notebook. In reality, no histogram ratio needs to be taken. Instead a neural net (or similar) would learn a good zeta for any given problem. Here we are sidestepping those Net shenanigans by creating a function that makes good zetas for us from some histograms of the data:

(
removeZeroData[x_]:={x[[1]],  x[[2]]/.{0->1}};

extractGoodZeta[filteredDatumPairs_,bins_]:=Module[{n=Length[filteredDatumPairs],l1=removeZeroData[HistogramList[(filteredDatumPairs//Transpose)[[1]],bins]],
l2=removeZeroData[HistogramList[(filteredDatumPairs//Transpose)[[2]],bins]]
},
Interpolation[ {  
Table[ ( l1[[1]][[i]]+l1[[1]][[i+1]])/2 , {i,1,bins}],
    Log[l1[[2]]/l2[[2]]]
}//Transpose]
]
)

betterZetaForLowWide=extractGoodZeta[genFilteredDatumPairs[100000,lowModel,testFilter,wideTransform],50];Plot[betterZetaForLowWide[x],{x,0,1}]

Check that sine model x and x’ distributions are DIFFERENT TO EACH OTHER (which is the sign of symmetry violation in the original) when we use (say) the wide transform:

Histogram[(genFilteredDatumPairs[100000,sineModel,testFilter,wideTransform]//Transpose),50]

betterZetaForSineWide=extractGoodZeta[genFilteredDatumPairs[1000000,sineModel,testFilter,wideTransform],50];Plot[betterZetaForSineWide[x],{x,0,1}]

Check that low model x and x’ distributions are DIFFERENT TO EACH OTHER (which is the sign of symmetry violation in the original) when we use (say) the narrow transform:

Histogram[(genFilteredDatumPairs[100000,lowModel,testFilter,narrowTransform]//Transpose),50]

Check that low model x and x’ distributions are DIFFERENT TO EACH OTHER (which is the sign of symmetry violation in the original) when we use (say) the very narrow transform:

Histogram[(genFilteredDatumPairs[100000,lowModel,testFilter,veryNarrowTransform]//Transpose),50]

Could compare histograms, but that requires binning, etc . Better to work on the pairs themselves

controlZeta[event_]:=1

dumbZeta[event_]:=If[event<0.3,1,-1] (* In reality one should get a neural net (or similar) to optimise Zeta to make it super-terrific at testing for symmetry violatioin. The example here is not trained at all but (by eye) has some hope ... *)

whichIsRealForOneEventPair[eventPair_,zeta_]:=zeta[eventPair[[1]]]-zeta[eventPair[[2]]]

whichIsRealForEventPairs[eventPairs_,zeta_]:=Table[whichIsRealForOneEventPair[eventPairs[[i]],zeta],{i,1,Length[eventPairs]}]

stats[x_]:=Module[{m=Mean[x], s=StandardDeviation[x],n=Length[x]},
{
{"Standard Deviation",s},
{"Mean",m},
{"Mean uncertainty",s/Sqrt[n]},
{"Sigmas mean is from zero", m/(s/Sqrt[n])}
}
]

Check mean of zeta is NOT significantly different from zero when underlying model is UNIFORM:

whichIsRealForEventPairs[genFilteredDatumPairs[100000,unifModel,testFilter,wideTransform],dumbZeta]//stats//N//TableForm

Repeating same check for a different zeta is NOT significantly different from zero when underlying model is UNIFORM:

whichIsRealForEventPairs[genFilteredDatumPairs[100000,unifModel,testFilter,wideTransform],betterZetaForLowWide]//stats//N//TableForm

Check mean of zeta IS significantly different from zero when underlying model is NON-UNIFORM :  (Note that as zeta is not properly trained, the real evidence for symm violation should be even greater)

whichIsRealForEventPairs[genFilteredDatumPairs[100000,lowModel,testFilter,wideTransform],betterZetaForLowWide]//stats//N//TableForm

Check mean of zeta IS significantly different from zero when underlying model is NON-UNIFORM :  (Note that as testZeta is not properly trained -- zeta here is cruder than previous guessr)

whichIsRealForEventPairs[genFilteredDatumPairs[100000,lowModel,testFilter,wideTransform],dumbZeta]//stats//N//TableForm

Similar check to above, but using the narrow transform rather than the wide transform. Note that the evidence for symmetry violation is smaller:

whichIsRealForEventPairs[genFilteredDatumPairs[100000,lowModel,testFilter,narrowTransform],dumbZeta]//stats//N//TableForm

And for a VERY narrow transform the evidence is smaller still ....

whichIsRealForEventPairs[genFilteredDatumPairs[100000,lowModel,testFilter,veryNarrowTransform],dumbZeta]//stats//N//TableForm

... so wide transforms are good .

Can we see non - uniformity in the Sine model?

Control: should see nothing at all using the control zeta:

whichIsRealForEventPairs[genFilteredDatumPairs[100000,sineModel,testFilter,wideTransform],controlZeta]//stats//N//TableForm

Evidence is very weak using the dumb zeta: (this is expected)

whichIsRealForEventPairs[genFilteredDatumPairs[100000,sineModel,testFilter,wideTransform],dumbZeta]//stats//N//TableForm

But (as expected) evidence is much stronger using a better trained zeta (albeit here created by cheat rather than by Neural Net or actual training process):

whichIsRealForEventPairs[genFilteredDatumPairs[100000,sineModel,testFilter,wideTransform],betterZetaForSineWide]//stats//N//TableForm