Thread: Testing a conventional approach to load development

For those interested in understanding further whether the testing methodology is valid -


Does shooting 20 round groups allow us to see and measure differences in precision?


Here is an example from a more precise rifle, with 2 loads that DO demonstrate a statistically significant difference in precision.


This is a 20 round group with handloads using the 80gr Targex (incidentally a load randomly selected using the approach described in post 1).

The shooting methodology was - prone, bipod & rear bag, short break between each 5rd group. The mean radius is 0.71cm.

Wind was light (<1ms).





Next we have a 20 round group with Hornady 55gr SP "training ammo".

The shooting methodology was the same, conditions more or less the same.

The mean radius is 1.18cm.





I have photos, for each of these groups, of each of the 4x rd groups that makes up the larger 20rd group. In neither case did the last shots "blow it out"; adding more shots over time simply fills in a better picture as you go.

These groups represent a reasonable picture of the precision of the system with these loads. Even with 20rd the statistical power is limited - in the world of statistics, 20 samples is a small amount when you are trying to detect small differences between means with relatively large standard deviations. However with 20rd groups and the difference in mean between these loads we do have power to detect the difference, and it is a significant difference at the 95% confidence level.






(groups are .9 vs 1.6 MOA)

Either of those is of course a wonderful hunting load, from a precision perspective, and now I have a lot of information about where the zero for the 55gr is, and a really accurate average speed to use for ballistics.

On pressure: I think if there is concern about the safety of the data due to uncertainty or variation, then a single shot ladder at reasonable charge weight spacings (e.g. 1gr) shot for a rough idea of velocity and pressure should help inform the loader, then follow up with a 10rd group at the charge weight area selected to give the required velocity to establish an idea of precision and a good average velocity.

Originally Posted by STC

..Parallax dialled out will minimize parallax, not eleminate it completely...

Have no idea of the basis for your statement. Can you expand please.

Bloody hell! Detailed and interesting! Thanks!

The more you know the less you know.
That is reloading!

Sent from my SM-A556E using Tapatalk

Originally Posted by andyanimal31

The more you know the less you know.
That is reloading!

Sent from my SM-A556E using Tapatalk

Applies to pretty much everything eh.

Originally Posted by Sidetrack

Ignore this, can’t seem to edit. GPM raised the 260 load data issue and you’re gone part way to explaining your thinking. The chamber differences still bug me but if they’re still within tolerances can we assume we’re still ok? I ask this as I was getting bolt lift issues at the bottom end of powder charge when loading for my 270. It was my first handload so I was being conservative but it stuck with me, hence the questions.

Two 270s get loaded for from my bench.one is near top of book load,the other sits between start and midpoint. No way would I load for the second rifle and hotter. Possible a short throat/ jump till pill hits rifling is the difference. If you look at Weatherby rifles of old( not sure of new ones) they had quite a spacewalk for projectiles which allowed a longer burn and higher charge perhaps? The new hypersonic shotgun rounds work on similar principle. By allowing payload to start moving forward it sort of increases the volume of the case....hopefully that makes sense??? It's why seating deep increases pressure,smaller volume.

Originally Posted by gimp

These groups represent a reasonable picture of the precision of the system with these loads. Even with 20rd the statistical power is limited - in the world of statistics, 20 samples is a small amount when you are trying to detect small differences between means with relatively large standard deviations. However with 20rd groups and the difference in mean between these loads we do have power to detect the difference, and it is a significant difference at the 95% confidence level.

@gimp I don't doubt your results but out of interest.

Questions;
How were you able to determine the location of each shot to obtain each shot location and radius? Looks difficult to do for the targex group unless this is was a group derived from multiple overlapping targets.
To assess significance are you visually comparing mean radius and 95% CI error bars or running a statistical test?

Are you able to provide standard deviations or raw data? I'd be interested in having a play and work out statistical power and effect size to give us an estimate of what your sample size would have to be to confidently detect a group that is over 50% bigger than another? I suspect very few shots when the difference is >50%. I do warn though that detecting the difference between two groups is different from understanding the precision of a single load.

Originally Posted by Makros

@gimp I don't doubt your results but out of interest.

Questions;
How were you able to determine the location of each shot to obtain each shot location and radius? Looks difficult to do for the targex group unless this is was a group derived from multiple overlapping targets.
To assess significance are you visually comparing mean radius and 95% CI error bars or running a statistical test?

Are you able to provide standard deviations or raw data? I'd be interested in having a play and work out statistical power and effect size to give us an estimate of what your sample size would have to be to confidently detect a group that is over 50% bigger than another? I suspect very few shots when the difference is >50%. I do warn though that detecting the difference between two groups is different from understanding the precision of a single load.

The targex group I have the 4x 5rd component groups available on separate targets (that were precisely overlaid over the 1 that contains the composite group) allowing me to precisely measure X & Y for each shot. I have then corrected X/Y by the MPOI offset from point of aim to obtain radii from MPOI.

The standard deviation for the mean radius of the targex load is 0.38cm and the SD for the hornady load mean radius is .58

a T test (paired, 2 tailed) in excel gives me .002 for the Targex vs Hornady and .53 for the 2 .260 loads - it's clearer visually to show the CIs overlapping for the audience

Happy to share raw data - I'm not an expert statistician by any means and peer-review of my work is always welcome

Gotta like science and scientists - bloody useful when they are doing such practical shit!

Funnily enough, with these SD for each group, for an 80% power (80/100 times you test you will detect the statistically significant difference in mean group radius and reject the null hypothesis they are the same), you need at minimum 19 shot groups.

This is rather revealing when one group is 77% bigger and has a 66% larger mean radius than the other, it still takes at least 19 shots to be sure they're different, because of the quite large variance (SD) of the samples.

Just playing here:
Assuming consistent SD of group 1 = 0.38cm and SD of group 2 = 0.58. Required sample size to detect a few differences in mean radius are below.
0.1cm difference in mean radius = 379 shot groups
0.2cm = 96
0.3cm = 44
0.4cm = 25
0.5cm = 17
0.6cm = 12
0.7cm = 9
0.8cm = 8
0.9cm = 6
1.0cm = 6
1.1cm = 5
.
.
.
1.6cm = 3

Post script; nothing wrong with your stats assuming you're picking the t-test for unequal variances. It's the appropriate test.

Also, nothing wrong with your stats assuming you're picking the t-test for unequal variances. It's the appropriate test. Also I'm not an expert statistician but use enough of these sort of tests in my field that I can do this without running to an actual statistician. Agree with the visual depiction of the 95% CIs means more to most and arguably are more meaningful that a statistical test to derive a p-value.

Despite me providing some samples sizes you would need to detect "statistically significant" difference between groups of different sizes. This is mostly an incorrect usage of p-values and power and is borderline meaningless other than an academic curiosity.

I warn that the importance of statistical significance should be tempered with an understanding of the data and what a statistical test, and p-value means (note it's not actually what was taught in high school textbooks, nor many a 101 stats textbook).
Also the fact that "significance" when we're talking stats doesn't mean "meaningful or important" nor my result is "correct".
It's a bloody hard concept to understand and I'm not certain I understand p-values fully. However, If you are interested in understanding it more and learning how misunderstood and how pervasive the incorrect usage of p-values is I would suggest reading the 13 misconceptions about p values here: https://www.ohri.ca/newsroom/seminar...03,%202014.pdf
And the delightfully named paper: The insignificance of statistical significance testing: https://core.ac.uk/download/pdf/188120111.pdf
Or a simple run down here: https://en.wikipedia.org/wiki/Misuse_of_p-values

In short make decisions on the observed effect, it is the most meaningful thing in your data. But understand with less samples the observed effect may be an extreme result on the basis of chance because of the variability of the data.
Do not dismiss results because they are not "significantly different" nor should you accept results simply because they are "significant".

From a practical perspective it becomes impractical to assess the difference between loads sensibly, hence my approach that it is best to take the shortest route to find one where you can have high confidence that it meets your requirements, and move on.

From another practical perspective the question is "when buying a rifle, how do I assess the actual quality of the barrel" because marketing hype aside, barrels do vary in quality.

Wether it's a modestly priced factory rifle like a Tikka, expensive factory like a Sako, or custom like a Trueflite or Bartelin, you'd generally expect to get what you pay for (the occasional complete dud that any manufacturing can produce from time to time accepted).

The fact that this statistical approach better informs us of a barrels accuracy (versus the old "you need to tune the load to the barrel and them I can rely on poor statistical analysis to provide a warranty/guarantee of sub MOA or whatever) might provide some challenges to gunmakers if it catches on.

This has so far been one of the more useful recent threads on here