PDA

View Full Version : How about a real-world test?



bigalinwv
11-22-2007, 07:07 AM
Instead of "foam" and closed courses, how about a real-world GOL test? Set up the radar guns on real roads with real traffic, and make two or three runs for each detector. Yes, I know there will be infinitely more variables, but the last time I checked, I drive in the real world, not in a closed course.

Every detector test I have ever seen is not in any way representative of real-world driving.

happya$$
11-22-2007, 07:23 AM
Instead of "foam" and closed courses, how about a real-world GOL test? Set up the radar guns on real roads with real traffic, and make two or three runs for each detector. Yes, I know there will be infinitely more variables, but the last time I checked, I drive in the real world, not in a closed course.

Every detector test I have ever seen is not in any way representative of real-world driving.

THe detectors that do best in our variable control tests will do best in the real world. The key is to control the variables and our tests do just that. The best real real world test that can be done is the ones that you encounter in the real world. For example if we do a test like you want and on one run we have an RMR that gets lucky because the radar bounced off another vehicle object and beat the top dogs does this represent RMR's performance? Remember radar bounces off everything

bigalinwv
11-22-2007, 07:59 AM
I think you're missing my point. I don't drive on a closed course with no variables. I understand what you're saying, but it almost reminds me of the EPA mileage listed for vehicles, who the he** ever gets what is on the sticker? There again, in a closed environment, not real world.

thestaton
11-22-2007, 08:02 AM
I'm good with how we do it now, but is there really a need to ever test detectors for the next couple of years? Every test ends up exactly the same, with the V1 dominating, then the Belscorts fighting for anything else that matters.

Real world is just to dangerous for me.

JeepinJeff
11-22-2007, 08:17 AM
I think a real world test would be nice but it would need to be in addition to, not in place of, the other testing.

bigalinwv
11-22-2007, 08:23 AM
I agree with that, it would be interesting to see how the two tests would "pair" up.

happya$$
11-22-2007, 08:48 AM
I'm good with how we do it now, but is there really a need to ever test detectors for the next couple of years? Every test ends up exactly the same, with the V1 dominating, then the Belscorts fighting for anything else that matters.

Real world is just to dangerous for me.



:lol: :lol: :lol: :lol: :lol: :lol: :lol: :lol: :lol:

I agree 100%. No need to do another RD test for a long time not unless something new comes out with more SENSITIVITY not more features

Michael B
11-22-2007, 09:04 AM
I respectfully disagree with thestaton and happyass on this one.

While I agree that controlled tests are needed to determine the most sensitive unit and the hen picking order that follows, it is not the same as a real world test.

If all you looked at were test results, then all the units get 8 to 9 miles of range based upon the SML test and you all know this to not be true in the real world with buildings, cars, terrain, etc...etc...Staton, you were there.

Some people drive in autoscan mode. Does that mean they get the same level of real world performance as in Highway mode?

Many units require 600 or 800ms to 1 full second of radar signal in order to alert while others may only require half of that. In a controlled test at 20-30 MPH there is little difference in the distance as a result of the extra 1/2 second however when was the last time you wrere driving at 20-30 MPH and were worried about receiving a speeding ticket?

Driving at 70MPH and now that half a second difference can add up.

Yes it makes sense to do a real world test and yes the results may be inconclusive all on its own but how is anyone going to know if the test is not done.

I seem to recall back when chamber data was presented, everyone said it was not the same as the real world and they are correct.

The GOL did a straightaway eccosorb sensitivity test and the order of the results mimicked the order of finish in the chamber tests indicating that the chamber test and a controlled outdoor straightaway test can provide similar results however one still has to say that this is not the real world.

ahmadr
11-22-2007, 10:10 AM
Many units require 600 or 800ms to 1 full second of radar signal in order to alert while others may only require half of that. In a controlled test at 20-30 MPH there is little difference in the distance as a result of the extra 1/2 second however when was the last time you wrere driving at 20-30 MPH and were worried about receiving a speeding ticket?

Driving at 70MPH and now that half a second difference can add up.

Good point. To keep the test controlled, maybe GOL can rig up some sort of automatic beep to shutter press converter, to exactly measure the distance upon alert at higher speeds....

bigalinwv
11-22-2007, 07:03 PM
In order to make it more repeatable and have some credibility, three runs or more with each should cancel out any kind of anomaly. (like the RMR comment above)

thestaton
11-22-2007, 07:41 PM
I hate to break the news but I'm not risking my life, or time to go play in the busy streets of Ohio for something real world.

The cops already get called enough when we are out in the middle of no where, I couldn't imagine the headache of operating around the public.

However, feel free to get together with a group & do this test. I'd like to see it, but with the price of gas, my safety, & not wanting to draw unwanted attention I'll pass and keep it on the closed courses.

Holla
11-22-2007, 10:22 PM
I'm good with how we do it now, but is there really a need to ever test detectors for the next couple of years? Every test ends up exactly the same, with the V1 dominating, then the Belscorts fighting for anything else that matters.

Real world is just to dangerous for me.

Bingo!!! Until somebody/company states they can SMOKE the V-1 there is no need :lol:

happya$$
11-23-2007, 05:02 AM
I hate to break the news but I'm not risking my life, or time to go play in the busy streets of Ohio for something real world.

The cops already get called enough when we are out in the middle of no where, I couldn't imagine the headache of operating around the public.

However, feel free to get together with a group & do this test. I'd like to see it, but with the price of gas, my safety, & not wanting to draw unwanted attention I'll pass and keep it on the closed courses.

Amen the only thing a "real world" test will do is even out the play field. If you do not like little spread a real world test will make it even smaller. As the saying goes "there is more than one way to skin a cat" and the end result is a skinned car, in this case V1 comes out most sensitive no matter what test it just will not be as extreme

bigalinwv
11-23-2007, 08:20 AM
So it's life threatening to do a real world test, where we all drive every day??

BullyDawg19
11-23-2007, 08:42 AM
the only problem with this idea is you cant compare apples to apples here. everything will be different for every detector on every run. there can be no direct comparison because unless you have every car traveling at the exact speed and in the same position for each and every unit and every run then there can be no comparison between two units. you would have to use the same cars, going the same speed, in the same position, etc and even that wouldnt really show an exact comparison. not to mention the distance required to perform that test. theres a reason for controlled tests! to test the abilities of the units under the same conditions with no other outside influence. for now ill stick with Mike B.'s and GOL's tests.

happya$$
11-23-2007, 09:00 AM
So it's life threatening to do a real world test, where we all drive every day??

It is when you are stationed on the side of the road shooting radar and worried about cops ticketing you

thestaton
11-23-2007, 09:02 AM
or better yet have some drive there car into yours at 80MPH. happens to officers all the time.

happya$$
11-23-2007, 09:18 AM
or better yet have some drive there car into yours at 80MPH. happens to officers all the time.

I left that part out :lol: :lol: :lol: :lol: :lol:

BullyDawg19
11-23-2007, 10:55 AM
So it's life threatening to do a real world test, where we all drive every day??stop and ask a construction worker what he thinks of your driving?

thestaton
11-23-2007, 11:24 AM
I'm about to lock this, this thread has ran it's course.

You guys are so passive aggressive. It's not like if you ask on the 201st time I'm going to say yes we will do it.

JeepinJeff
11-23-2007, 11:39 AM
Might as well lock it. The people that want the test won't change their mind and neither will those that run them.

Hey it's not like you get paid for your time, might as well make the rules. It's not a democracy.

It would certainly be very difficult to control the situation and safety should always be an issue.

Personally I still think there are ways that it could be done safely. But that's just my opinon and I made it with my vote.

KnightHawk
11-23-2007, 05:20 PM
the only problem with this idea is you cant compare apples to apples here. everything will be different for every detector on every run. there can be no direct comparison because unless you have every car traveling at the exact speed and in the same position for each and every unit and every run then there can be no comparison between two units. you would have to use the same cars, going the same speed, in the same position, etc and even that wouldnt really show an exact comparison. not to mention the distance required to perform that test. theres a reason for controlled tests! to test the abilities of the units under the same conditions with no other outside influence. for now ill stick with Mike B.'s and GOL's tests.

Well stated.. You also have to cover a large area too when doing straight line test and think about the vehicles and objects in between the radar source and the test vehicle with the RD unit tested.

crazyVOLVOrob
11-23-2007, 08:23 PM
We would of had the best real word radar detector course but Craig Peterson from radartest.com sabotaged the test course!!! As for the way i have seen the radar courses set up I have seen cops sit the same way off axis shooting the rear of cars

As for the laser end I would like to see both the Worst Case senerio vehicle as well as a midsize car to test out single heads and the M20/M25 :wink:

o2bad455
11-24-2007, 10:19 AM
Honestly, "real world" means too many variables to be repeatable, much less meaningful for comparison purposes. The GOL tests have been excellent, and so have the lab tests (props to Michael B.). The radar foam used by GOL in the last test was the best method yet, IMHO. Please keep up the great work, and I look forward to more tests of the latest post-fix STi, the latest hardware rev V1, and the next gen Whistler, among others.

happya$$
11-24-2007, 12:12 PM
I have Mike B's lab test results from the units we tested in August that I will be posting soon so we can see how lab results compare to our test results

o2bad455
11-24-2007, 12:34 PM
I have Mike B's lab test results from the units we tested in August that I will be posting soon so we can see how lab results compare to our test results
Excellent. Can't wait to see them :D

zr1
12-13-2007, 09:45 PM
I'm a noob on the radar, but have some opinions on the scientific method aspect of it.

A controlled test can be done with precision, which is what we've seen with the GOL tests. However accuracy is another matter. We don't drive in a test environment, so it's an assumption that the detectors will perform in some sort of the same pecking order as the results of the controlled test.

A real world test would be interesting, and I'd definitely read it. But the methodology would not be science as there would be some uncontrolled variables, which was correctly pointed out above.

To alleviate these variables would be to do a quantity of tests where the results over time would begin to gravitate toward an accurate picture. However, this wouldn't be practical at all as I estimate the number of tests per detector would be somewhere around....a few hundred? Maybe as little as a hundred? Even then, with the length, time of day factors (rush hour), day of the week factors, seasonal factors, it would start to play in. Then it's likely the testers would need to selectively negate some of the test runs where the variables are grossly out of whack...a herd of trucks on the road at that moment, a lightning strike, dogs and cats living together, mass hysteria! Negating test runs would have to be based on specific criteria. For quantity tests, this is often a point of contention among scientists that their method of disqualifying results isn't reasonable.

A test using science is something where the methodology is published so that other people could follow the methodology and get the same results. Anything else is just screwing around.

But for the intrigue and entertainment value, I would love to see a real-world GOL "exercise" (with a big disclaimer on it of course).

amoney
12-13-2007, 10:20 PM
There are two things, accuracy and precision, accuracy is in this case how sensitive a detector is. Precision is the repeatability of getting that same result. Thus together equals your performance.

GOL has in the past tested in a controlled environment with a sweeping curve, and has tested another real world where the officer is shooting at an extreme angle into a wooded area.

The trend of the results have always been the same, and thus following Mike B sensitivity numbers, as well as GOLs easier to administer absorbent foam tests.

And we already have other real world results, just look at the threads, real experience etc. that has the same result as if you asking GOL to do real world tests. You have the same detectors doing extremly well AND extremly poor, depending upon the environment.

Its like beating a dead hoarse, how much do you want to beat on it, all the premium detectors are great. Be happy. Go argue over the question of a plane on a treadmill, will it take off?

:D

cell21633
12-13-2007, 10:36 PM
i have a solution for everyone.


if you want a test done differently, go ahead and run a test yourself.

GOL did it. they started their own group for getting data directly to the people, without restriction from companies. their tests are in a controlled manner, and they clearly state that their tests are meant to judge the RELATIVE performance of detectors. they also give you distance readings, so from there you can get the ABSOLUTE performance.

MEM-TEK
12-14-2007, 09:48 AM
Hi everyone,

There are a lot of things which I pay attention to when reviewing GOL's carefully executed tests. For example, GOL's straight-line sensitivity tests were very accurately executed, and even Michael B comments:


The GOL did a straightaway eccosorb sensitivity test and the order of the results mimicked the order of finish in the chamber tests indicating that the chamber test and a controlled outdoor straightaway test can provide similar results however one still has to say that this is not the real world.

Indeed, I calculated dBm sensitivities for each of the radar detectors which were very similar to Michael B's previously lab results.

Yet raw sensitivity is NOT everything. GOL's off-axis tests were interesting because those tests showed that there can be significant differences in the off-axis performance of various radar horn designs, and that occasionally a radar detector might experience particular difficulties picking up a particular off-axis radar frequency due to either phase cancellation issues within the horn or due the phase cancellation muddying up the received signal such that internal filtering is triggered. This latter reason is why I HATE Cobra's choices of LO frequencies and poorly controlled emissions. An Escort rep told me a while back that filtering out Cobra radar detectors really does present major design headaches for other radar detector manufacturers.

Aside from raw sensitivity, it is important to look at GOL's over-the-hill (not performed during their August test), around-the-curve, and forward-facing tests. I had no problem with the setups for each of these tests, and I feel that these tests did a good job of mimicking the real world for these types of encounters. It is also in these tests where a radar detector can either perform unexpectedly poorly or unexpectedly well compared to what would be expected based on the raw straight-line sensitivity tests. Why? Because these tests do mimic the real world rather well.

Some radar detectors performed better than expected in these tests, some performed as expected, and some performed worse than expected. The over-the-hill and forward-facing tests, for example, are particularly good tests to see how good each radar detector is at alerting to random blips of radar (caused by random reflections off of nearby objects) which are just above the radar detector's raw sensitivity threshold. Likewise, the around-the-curve tests can reveal off-axis detection problems caused by off-axis phase cancellation issues and/or filtering algorithm issues.

Like Michael B says:


While I agree that controlled tests are needed to determine the most sensitive unit and the hen picking order that follows, it is not the same as a real world test.

GOL's real world around-the-curve and forward-facing tests revealed some surprising results. For example, the Whistler Pro-78 performed surprisingly well on Ka in the forward-facing and around-the-curve tests -- better than would be expected based on the straight-line sensitivity tests. The Pro-78 had essentially the same straight-line 34.7 sensitivity as the Cobra and Solo 2, yet the Pro-78 performed better in the around-the-curve test. Likewise, two of the forward-facing 35.5 runs for the V1 version 3.863 were only marginally better than two of the runs for the Pro-78, were behind the performance of the V1 version 3.818 and significantly behind the performance of the V1 version 3.813. Interestingly, Cobras appear to be all but dead to radar in in the forward-facing and around-the-curve tests. The Cobra's display does make a nice nighttime map light though!

While the straight-line sensitivity tests are excellent for determining the inherent raw sensitivity of each radar detector when detecting a very weak yet steady radar signal, its really GOL's other tests such as over-the-hill tests, forward-facing tests, and around-the-curve tests which truly give one a better idea about how each radar detector will perform in the real world.

In conclusion, I think that GOL's current testing methodology is just fine as-is.

Best regards,

--Michael

Esoterica
01-08-2008, 06:44 PM
I'm brand new here so maybe I can offer an impartial perspective. However, if this is more a thing about some people here not getting a long or agreeing with others then my suggestion will be of no use because that would seem more that people are just arguing to argue.

Maybe I'm not completely impartial though, I've read all of the GOL reports and findings when I went into the market researching for a detector that I just recently bought. I found their information and Radar Roy's to be invaluable in helping me make a informed decision (I went with the 9500i by the way just for the record).

If this is a legitimate debate among all of you though the solution is very simple.

I do see the concern and points of real world testing, I also understand the absolute scientific importance in set controls for fair and accurate comparison.

What I don't see is why you can't have both. I'd start by asking the person who initiated the request to be very specific in just what factors you see needed in a real world test, like buildings, other vehicles, small children running out into the street chasing their ball?

Then there would be no reason why these exact conditions could not be reproduced in the test environment by the test control personnel, for example, other testers in other vehicles traveling in front of or behind the actual test vehicle. For buildings to be factored in I don't know about your test locations, but anywhere I've ever lived we have huge industrial parks full of streets and buildings that on a Sunday afternoon when all of them are closed the places are ghost towns. You could even set up an X band signal source off the side of the road to simulate a false signal during the tests, which leads into my next comment here anyhow...



We would of had the best real word radar detector course but Craig Peterson from [banned site] sabotaged the test course!!! As for the way i have seen the radar courses set up I have seen cops sit the same way off axis shooting the rear of cars


YouTube - Guys of Lidar Radar Detector Test Sabotaged!!! (http://www.youtube.com/watch?v=3tlh6LKI1fk)

You guys actually sound like your bothered by what this guy did, personally I think it was hilarious that some bone head did this. I mean what harm was actually done to any of you or your testing as a result of that, you actually ended up with all those free goodies that the guy just gave you like he was Santa Claus himself.

What I can't believe is the guy was dumb enough to use a box with his own name and shipping label still on it, know there was no way you wouldn't detect the equipment there, and just give it to you for free like that. It just seems pretty odd and and almost screams a frame job at play more than an act of actual sabotage.

SmaartAasSaabr
01-08-2008, 08:09 PM
You guys actually sound like your bothered by what this guy did, personally I think it was hilarious that some bone head did this. I mean what harm was actually done to any of you or your testing as a result of that, you actually ended up with all those free goodies that the guy just gave you like he was Santa Claus himself.

What I can't believe is the guy was dumb enough to use a box with his own name and shipping label still on it, know there was no way you wouldn't detect the equipment there, and just give it to you for free like that. It just seems pretty odd and and almost screams a frame job at play more than an act of actual sabotage.

I was actually there at the GOL test in Arizona then and I thought it was pretty cool too :lol: Unfortunately someone beat me to calling dibs on it :lol:

zr1
01-09-2008, 08:08 PM
There are two things, accuracy and precision, accuracy is in this case how sensitive a detector is. Precision is the repeatability of getting that same result.

Precision is straight measurement and getting the same measurements each time like you say. Accuracy is a function of how close the instrument is to the human goal. If the human goal is sensitivity, then so be it.

In another example, a rifle can be precise. And with precision, a rifle can repeatedly group 1" groups, but 1 foot left of the target and be precise. But to be accurate, it requires a human shooter that puts the bullets where the human desires. We still toss around the phrase "that thar' gun is accurate!" a lot. However for the scientific method, they have different meanings.

MEM-TEK
01-09-2008, 08:15 PM
Precision is straight measurement and getting the same measurements each time like you say. Accuracy is a function of how close the instrument is to the human goal. If the human goal is sensitivity, then so be it.

In another example, a rifle can be precise. And with precision, a rifle can repeatedly group 1" groups, but 1 foot left of the target and be precise. But to be accurate, it requires a human shooter that puts the bullets where the human desires. We still toss around the phrase "that thar' gun is accurate!" a lot. However for the scientific method, they have different meanings.

Hi zr1,

Welcome to the forum! (If you haven't been welcomed already.)

For one of your first posts, your above post is pretty darned good. A point well made if I may say so!

--Michael

specialchicken
01-09-2008, 10:15 PM
ill post some results after we have done our real world tests!!

Esoterica
01-17-2008, 03:27 AM
ill post some results after we have done our real world tests!!


I would like to see some real world test of any system being used out in a blinding snow storm.

Not because I so much care about what the results are, but just to see video of someone who isn't me out in the blowing cold suffering from it. I think Chicago in February would make for a great location for such a real world test.

steagall1000
01-28-2008, 06:10 PM
I agree with Happa$$ and others, there really is no need to do a new radar detector test unless a new brand of detector comes out claiming better sensitivity. Then again I also like the way they do their radar detector test at GOL. If your going to do a real world radar detector test, then factor in all the variables. radar bouncing off roads,cars,trucks etc. Just wanted to add my opinion to this discussion.

Georgia Boy :)

Ghost917
02-21-2008, 07:50 PM
Ok i said no to this BUT I would like to see a test on a really big hill with the radar gun shooting at the top of the hill and not over it. I would really like to see those results!!!

Obsidian
02-21-2008, 10:44 PM
This again? The most sensitive detector should always win the GOL test by design, and most sensitive does not necessarily equal best for everyone's situation. Testing them in LA would be an exercise in frustration.

My results testing the V1 (7 or 8 months) and X50 (a few years) in the real world is the X50 is better for me than the V1. I found the V1 slightly better on the highway (forward), much better in rear detection, and I found the arrows to be helpful. I found the X50 much better in the city, much better over hills and around corners on Ka, and much better filtering & muting falses. Laser never applied, but the V1 falsed Laser (and every radar band) much more than the X50. YMMV.

Testing them yourself and determining which one is best for you is the only way you can take other people's opinion's out of the equation.

You can basically test the V1 as long as you want, sell it and get your money back. I sold mine on eBay for *very* close to what a new V1 plus shipping goes for after using it for 7 months or so. The others all have at least a 30 day return policy.

-Obsidian-

Ovencleaner
02-21-2008, 11:01 PM
MEM-TEK, I have to agree and when I was looking for a detector, I didn't even look at the straight test because they are all similar. Off axis is really what defines detection.

djrams80
02-21-2008, 11:03 PM
I agree with Happa$$ and others, there really is no need to do a new radar detector test unless a new brand of detector comes out claiming better sensitivity.
Like the 9500ci(STi-R) and the ZR4?

MEM-TEK
02-21-2008, 11:25 PM
MEM-TEK, I have to agree and when I was looking for a detector, I didn't even look at the straight test because they are all similar. Off axis is really what defines detection.

Thanks for the compliment, but I simply was agreeing with zr1's post -- compliments are due to him.

Anyway, I too pay much more attention to off-axis detection capabilities -- but only within a +/- 30 degree angle relative to on-axis. Beyond 30 degrees (30 degrees already resulting in radar/laser gun readings which reflect just 87% of your true speed), I just don't care. In particular I pay attention to radar detection capabilities within +/- 20 degrees of on-axis, and what I look for within this fairly narrow off-axis range is excessive detection roll-off due to phase canceling within the radar horn design or due to software problems in the filtering algorithms.

I hope that GOL will do another off-axis detection test but instead simply concentrate on a range of 0 to 25 degrees off-axis with 5 degree increments. I think that this would most adequately cover the "field of view" associated with the vast majority of radar encounters which could result in a ticket.