Why YouTube "Gun Tests" Are Irrelevant.

[GardoneVT]

The cheesy into music ends.

Next, a tall dude with a pistol dunks it into some Filthy Stuff after a 45 minute monologue on handgun toughness. Said pistol then mailfunctions. He tests another brand and that gun runs like a top.

What's this mean? Entertaining TV, period.
Problem One: population vs sample size. Without rehashing Stats 101,sample refers to a subset of a given number of what you're measuring, and population refers to the entire enchilada.

Now a population study on firearm reliability would be a very expensive proposition,as well as legally and logistically unfeasable: you'd have to test the crime guns and the PD duty pistols too. So a sample size is the only reasonable way to test if Brand X pistol is reliable.

But how big should the sample size be? Ideally a proportion of the total production in a given time frame; which means we are conservatively talking about hundreds of individual pistols. For high volume products like a VP9 or Beretta 92 ,thousands. Even if a YouTube producer was enterprising enough to source hundreds of different guns and tested them consistently,the sample size compared to a population of guns in the tens thousands would be a laughably small portion of the whole.

Then there's the other ,nastier variables. Each shooter has different hand and finger sizes, so either you use one shooter for the entire evaluation (better get some pain meds for their trigger finger!) or use multiple shooters and risk ergonomic data contamination. Ammo must also be standardized, and at this level even a 1% unreliability rate in the ammo can have consequences. Lubrication, the type of test medium used(did you use Hoosier dirt or Nevada desert sand?) ,all of it matters.

While such a rigourous study would be a good thing to see,it would be so cost prohibitive only governments could afford that bill. While abusing guns is entertaining video ,don't confuse it with a valid test of reliability....even if the video uses "test" in the title.

[Gray222]

All that and the simple fact that their "tests" or "reviews" are horseshit.

[RJ]

Here's where you went wrong. :cool:

Your argument is only applicable to objective functional, or physical requirements.

Things that can be measured against a specification. Length. Hardness. Spring rate. Physical characteristics. These can certainly be measured, and applicable sampling theory applied to select the appropriate and meaningful sample size for a arbitrary group N.

I agree with everything you said, for these types of requirements. For example, to have an internet celebrity produce a video claiming spectacular reliability results after "100 flawless rounds" would only appeal to Cleetus and Lurline.

Operational requirements, or how the user employs a product, howwver, can't be measured with a yardstick. They have to be, well, used. This attracts a certain level of expectation that the stochastic nature of operational use will produce unprecitable results.

In particular, even when the Operational Concept Document is followed precisely, and to the letter, in operational test design, when confronted by the actual product, users interact with it in oddly unpredictable ways. This occurs even in what appear to be fairly tightly controlled test regimens.

So, in sum, while I would not base a firearm purchase myself purely on a Sterilite tray of river mud, a consistent operational evaluation of it in an online video is worth paying attention to. :)

I mean, besides generating butthurt amongst us owners of the pistols that have gone through such tests. :cool:

[GardoneVT]

Here's where you went wrong. :cool:

Your argument is only applicable to objective functional, or physical requirements.

Things that can be measured against a specification. Length. Hardness. Spring rate. Physical characteristics. These can certainly be measured, and applicable sampling theory applied to select the appropriate and meaningful sample size for a arbitrary group N.

I agree with everything you said, for these types of requirements. For example, to have an internet celebrity produce a video claiming spectacular reliability results after "100 flawless rounds" would only appeal to Cleetus and Lurline.

Operational requirements, or how the user employs a product, howwver, can't be measured with a yardstick. They have to be, well, used. This attracts a certain level of expectation that the stochastic nature of operational use will produce unprecitable results.

In particular, even when the Operational Concept Document is followed precisely, and to the letter, in operational test design, when confronted by the actual product, users interact with it in oddly unpredictable ways. This occurs even in what appear to be fairly tightly controlled test regimens.

So, in sum, while I would not base a firearm purchase myself purely on a Sterilite tray of river mud, a consistent operational evaluation of it in an online video is worth paying attention to. :)

I mean, besides generating butthurt amongst us owners of the pistols that have gone through such tests. :cool:

If one abuses any weapon enough, it's going to fail. Marines do this as a hobby. ;)

Just because MrYoutube can shoot a singular Glock 17 after tossing it out of an airplane doesn't factually mean yours will do the same.

[Al T.]

One of "those guys" tends to show the bullets hitting paper, then cut away to him shooting, back to holes in paper. At no time does he give you an idea as to actually how far away he is from said paper. And IIRC, saw some video clip of him at shot show in a booth... Holeeee Cow......

[orionz06]

I'd agree with the OP but he's just flat wrong. They are relevant, just not to people who know better. They're entertaining as hell for some people, just not us.

[abu fitna]

I will accept entertaining as one part of the answer to why we would be poorer as a community of practitioners for the loss of a certain segment of the well executed examples among such videos. (I can even accept laughs when not so well executed - like the peanut butter examples and the like.)

I will also add to this the idea of confidence. Once can have all of the abstract engineering one wants, but to have a random example pulled from the rack and subject to things and ordinary user would not find a stretch to far (for whatever values of ordinary one cares to posit for mission, audience, or even just aspiration) is something indeed. The fact that almost no one celebrity or otherwise has a lock on such videos - and the proliferation of various flavours of design test tortures atests - means that sample sizes may be small, but variables are otherwise immensely confounding. There is something to be said for performance against all odds, and under any circumstances - realistic or otherwise. Does failure then count heavily in one's thinking about a particular platform? Perhaps not - but perhaps so when otherwise one can find many successive examples where "works as expected" is the same answer every time, even if outlandish circumstances are presented.

It is not stats, it is aggregate experience. While it is not engineering or elegance, there is something in us - since the time of edge chipping, rock throwing, and spear chucking that is without a doubt able to consider all of these things in a different manner upon watching a demonstration or test. That we look to experienced warriors for such a thing is something that may be manipulated by the marketing drones that would leverage such men and the many imitators that may come out of the woodwork claiming to be themselves such a thing does not mean that the underlying thing itself is without some value. But this brings us back to many individuals making hard choices, and documenting their experiences with the product of those choices. And in the end, this is I feel what the best practitioners thrive on - forthright presentation of facts based on what one has done with one's own hands, and seen with one's own eyes (whether accompanied by an electronic eye or not). And from the abundance of such examples I personally have benefited, here and elsewhere, and would be loathe to see us back to the age of paid shills and third hand samzidat passed hand to hand until it echoes only into rumor and fiction.

[Kyle Reese]

I'm going to pour gravy, lard or hot caramel sauce all over my new VP 9, just to prove it'll git me kilt in the event of a mass gravy attack, or if I have to fight after dropping my carry gun in a vat of decadent, molten caramel goodness. Conversely, I'm sure that the Arex Rex One Zero (or whatever the hell it's called) would survive the cara-pocalypse with aplomb.


*joking*

[MistWolf]

Operational Concept Document

That's Concept Document, Operational. Must keep things in the proper alphabetical order

[RJ]

That's Concept Document, Operational. Must keep things in the proper alphabetical order

LOL.

Well, when I was teaching SE 101 we used ConOps or OCD somewhat interchangeably. INCOSE has a good reference in the SEBoK:

http://sebokwiki.org/wiki/Operational_Concept_(glossary)

I will say the noun names varied greatly with customer. In fact, sometimes with startling results.

True story: Sitting in a lengthy meeting in London with our MoD customer, my level 7 boss got up, stretched, and said his suspenders were killing him and he needed a break.

A murmur swirled around the British side and several eyebrows shot up as my boss walked out to stretch his legs.

I leaned over quizzically to my in-country fellow cell-phone carrier, who, like me, was there to take notes. He whispered: "Rich, in the UK, suspenders are what women wear to hold up their stockings - I think you yanks call it a 'garter belt'."

In Systems Engineering, words matter.

:cool:

[Greg]

I suspect this trend of questionable review videos extends beyond guns.

The "everybody gets a trophy" crowd seems extroverted and cameras are cheap.

[RevolverRob]

My favorite Youtube Reviews are by fat guys who talk about how great "Gun X" is for the "Teams". Or basically...just about every Youtube video with Larry Vickers in it.

I only watch Youtube Previews of incoming products, suppressors tests (because I just like to hear what they sound like), and ammo shot into bonafide gel. Everything else is people goofing off on camera and might as well come with a disclaimer that says, "Prior to filming this, I said, "Hold ma' beer and watch this!""

[Hot Sauce]

I suspect this trend of questionable review videos extends beyond guns.

The "everybody gets a trophy" crowd seems extroverted and cameras are cheap. I should point out the the most popular gun YouTube reviewers are definitely not of "everybody gets a trophy" generation.

Turns out there were plenty enough trophies to go around for everyone. Once those trophies have passed being water/mud dunked, that is. You need trophy that you can bet your life on.

[busdriver]

Sample size problems aside, I find Pat Kelley's youtube channel informative.

[Clobbersaurus]

Youtube gun test video's are almost as irrelevant as forum based opinion threads.

I will say that I do tend to pay more attention to the videos and forum posts of folks who are shooters (the Pistol Forum definition of "shooter") than those of Cletus McCleti the youtube/forum poster.

I also now put any new pistols through the 2000 round challenge, and base my final opinion on my own personal sample of one.

[txdpd]

I like the Hickok45 videos. While I don't think he does true "gun test", if he's shooting something and has a hard time loading or operating it, it's definitely on the do not buy list.

[karmapolice]

You mean the guy who pins the trigger on every shot and has to readjust his grip as well during the video review on a new pistol being hot shit might not be worth the camera equipment it was filmed on?

Get outta here... my reviews are at least closer to their value which is my two cents :cool::p

[P.E. Kelley]

Beyond the YouTuber...are "we" trusting" the Paper Rags glowing reviews of nearly every new toy?

I have had a few articles published in the glossies over the years and I know I tell it like it is...

But unless we know who the scribe is (as in what qualifications they have beyond a journalism degree) how
can we trust a gunwriters, writings?

[ReverendMeat]

I guess I shouldn't be basing any opinions on my own experience either, given how statistically insignificant it is.

[Greg]

I should point out the the most popular gun YouTube reviewers are definitely not of "everybody gets a trophy" generation.

Agreed, I was merely speculating that if one searched YouTube for DSLR, motorcycle, gas oven or drone reviews you'd find a lot of bad info on those items as well.

There are gun related channels I like.

[RJ]

Beyond the YouTuber...are "we" trusting" the Paper Rags glowing reviews of nearly every new toy?

I have had a few articles published in the glossies over the years and I know I tell it like it is...

But unless we know who the scribe is (as in what qualifications they have beyond a journalism degree) how
can we trust a gunwriters, writings?

Mr Kelley: I like your videos because you, you know, actually 'shoot' the pistols. They are also upbeat, and fun. Plus you don't take yourself too seriously.

[P.E. Kelley]

Mr Kelley: I like your videos because you, you know, actually 'shoot' the pistols. They are also upbeat, and fun. Plus you don't take yourself too seriously.

I can't take myself too seriously...I ain't a serious kinda guy.

[RJ]

I like the Hickok45 videos. While I don't think he does true "gun test", if he's shooting something and has a hard time loading or operating it, it's definitely on the do not buy list.

Me too. Dang that gong is far away. :)

I also like the fact that he shows, really shows, what the pistol looks like. I find the slow close ups pretty interesting. It odd other videographers don't seem to do that.

Plus, he is interesting to listen too, and his voice is relaxing. Personally I could do with more 'review' and less 'here's me shooting stuff' but I like him overall.

[Greg]

I like the Hickok45 videos. While I don't think he does true "gun test", if he's shooting something and has a hard time loading or operating it, it's definitely on the do not buy list.

I believe Hickok45 is 6'8" - he's got to have some giant sized hands. A lot of guns have to feel like the controls and trigger reach are all horked up to him.

[Al T.]

how can we trust a gunwriters, writings?

Not just the writer, but the editor. One of the things that makes SWAT magazine great is Denny Hansen and one of the things that infuriated a buddy of mine who wrote for a big hunting magazine was an editor changing his data.

[jetfire]

I've never had an editor change my data before, and I'm pretty sure I'd lose my god given mind if something like that happened. Holly crap.


Sent from my iPhone using Tapatalk

[BaiHu]

I'm going to pour gravy, lard or hot caramel sauce all over my new VP 9, just to prove it'll git me kilt in the event of a mass gravy attack, or if I have to fight after dropping my carry gun in a vat of decadent, molten caramel goodness. Conversely, I'm sure that the Arex Rex One Zero (or whatever the hell it's called) would survive the cara-pocalypse with aplomb.


*joking*
Well, I think we should ask Lord of the Food Court, @Mr_White, to see if he's ever had this problem before we just throw it out willy nilly [emoji12]

Sent from my SM-G930V using Tapatalk

[jetfire]

I had someone tell me that shooting a gun for 2000 rounds over the course of a month or so wasn't relevant because "no one shoots that much."

I died a little.


Sent from my iPhone using Tapatalk

[Lex Luthier]

I had someone tell me that shooting a gun for 2000 rounds over the course of a month or so wasn't relevant because "no one shoots that much."

I died a little.


Sent from my iPhone using Tapatalk

Who knew that Pauline Kael had a brother named Cletus?

[Joe in PNG]

I had someone tell me that shooting a gun for 2000 rounds over the course of a month or so wasn't relevant because "no one shoots that much."
I died a little.
Sent from my iPhone using Tapatalk

The best response is to point at their novelty zombie target- the one with scattered bullet holes at random locations despite being 5 yards away- and say "that is why you fail".