Talk:F-divergence

	Mathematics portal This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.MathematicsWikipedia:WikiProject MathematicsTemplate:WikiProject Mathematicsmathematics
Mid	This article has been rated as Mid-priority on the project's priority scale.

Statistics Mid‑importance

	This article is within the scope of WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics
Mid	This article has been rated as Mid-importance on the importance scale.

Instances of f-divergences

In the table, what is t? I guess t=P(x)-Q(x) but I couldn't tell for sure. Metaxal (talk) 12:14, 4 July 2013 (UTC)[reply]

Incosistent with KL-divergence article

Article on KL-divergence defines it as:

D_{\mathrm {KL} }(P\|Q)=\sum _{i}\ln \left({\frac {P(i)}{Q(i)}}\right)P(i)

while based on this article one would derive the KL to be:

D_{\mathrm {KL} }(P\|Q)=D_{f=-\ln }(P\|Q)=\sum _{i}-\ln \left({\frac {P(i)}{Q(i)}}\right)Q(i)=\sum _{i}\ln \left({\frac {Q(i)}{P(i)}}\right)Q(i)

I guess both of them cannot be right because the divergence is not symmetric. Can somebody with the necessary knowledge decide which is the right formulation? Tomash (talk) 10:34, 16 August 2014 (UTC)[reply]

Instead of taking f(t) = -ln (t), take f(t) = t ln t:

$KL(P\|Q)=\sum _{i}P_{i}\ln(P_{i}/Q_{i})=\sum _{i}Q_{i}(P_{i}/Q_{i})\ln(P_{i}/Q_{i})=\sum _{i}f(P_{i}/Q_{i})Q_{i}=D_{f}(P\|Q)$

--David Pal (talk) —Preceding undated comment added 12:23, 16 May 2016 (UTC)[reply]

Add visual examples

This would be much easier to understand if there were a few examples of distributions, with the distances given between pairs of distributions via various measures. ★NealMcB★ (talk) 16:33, 1 March 2021 (UTC)[reply]

Jensen-Shannon imposter

Whatever this is, it does NOT generate the Jensen-Shannon divergence! I don't know what it generates. It certainly generates a symmetric f-divergence, but it is NOT the Jensen-Shannon divergence!

$f(t)={\frac {1}{2}}[(t+1)\log {\big (}{\frac {2}{t+1}}{\big )}+t\log t]$

I explicitly calculated the $D_{f}$ between the two coin-toss distributions with heads probability 1/3 and 2/3. I got

$D_{f}(1/3\|2/3)={\frac {2}{3}}\log {\frac {32}{27}}$

which is not equal to

$D_{JS}(1/3\|2/3)={\frac {1}{6}}\log 4$

And there is no way to fix this. There is no way to find some $a,c$ such that $a(f(t)+c(t-1))=f_{JS}(t)$ .

Whoever knows what it actually generates please add it back with the proper name attached.

pony in a strange land (talk) 01:18, 24 May 2022 (UTC)[reply]

This is in fact Jensen-Shannon divergence. Elestrophe (talk) 18:40, 6 October 2024 (UTC)[reply]

Conheça Walt Disney World

Instances of f-divergences

Incosistent with KL-divergence article

Add visual examples

Jensen-Shannon imposter