P value just tells you how likely the difference between two data sets is just random chance. Having said that, there are many ways to p hack to achieve significant values.

[–] panda_abyss@lemmy.ca 1 points 1 week ago

I’m probably being too pedantic, but that’s exactly correct, though it also captures the spirit and practically most common cases.

Just change “two datasets” to “a statistic”, because hypothesis tests can be pretty general.

I don’t love P values, they’re not quite an intuitive probability, they bake in a lot of assumptions, they can be deceptively over or under sensitive, and they’re hard to explain. As a tool they’re useful but they always need context and planning.

[–] village604@adultswim.fan 1 points 1 week ago

That's why it annoys me when people complain about what they think is a small sample size.

Large sample sizes can be used to p hack