Here’s an interesting test: measure your Risk Quotient. It’s a 50 question survey of a set of questions, some simple and some obscure, in which you estimate your confidence in providing an answer. You aren’t scored on just getting the right answer, but on whether you accurately assess your likelihood of being right — if you answer wrongly but with great confidence and certainly you’ll score poorly, but if you answer just as wrongly but with a more cautious appraisal of your certainty, you’ll score better. If you’ve got a serious case of the Dunning-Kruger effect, you might want to avoid the survey. It won’t help your self-esteem.
I scored an 83. I’m completely uncertain about whether that is good or bad.
You’re going to have to fix that link, PZ.
It takes us directly to YOUR test results!
Oops!
MikeM
You linked to your own test results.
You want to link to http://projectionpoint.com/test2.php instead.
Stranger, we are such PITAs.
MikeM
Hmmm, cool test. I scored an 80, which they tell me is ‘high’.
I think the fact that it shows up the answer to each question as you click the response is a bit problematic, because if someone makes a few early errors they may be ‘scared’ into being more conservative later in the test.
I got 90, and I think that’s good, but I’m really not sure. Your curve being on the diagonal is the best result, and mine seemed quite close, though I often underrated my knowledge, choosing 80% or 90% for true statements. Interesting. You can do it again with different facts.
Yep.
I got a 95! I’m about 90% pleased with that.
That’s a cool little piece of research.
90 for me.
Also, PZ, I noticed that your results had recorded several earlier completions of the test. If those are someone else’s — if you followed a link like the one you initially gave here — then your result of 83 might be skewed by others’ tests. If that’s the case, clearing your browser cookies and starting over will fix it.
I got 60. Average.
I don’t think i mind.. mayby i do..
*SPOILER*
I wasn’t impressed by their Ebola question. The mortality rate of Ebola varies significantly based on the strain. So their true/false definition on that question is faulty, despite the result it shows after you answer. Grrr.
Your RQ score is 82. Such a score is high.
Interesting. I didn’t think I had done all that well.
94, whatever that means.
77 here. Acceptable.
Well I’m darned if I can figure out what the results mean. I don’t appear to have done very well, but I can’t quite figure out why. Certainly the questions were very USA centric and so I didn’t know many of them.
A bit of experimenting shows that not knowing any of the answers and so giving them all 50% gives you an average score, which is essentially what I got. Oddly, my score is slightly lower than average even though I know that when I did venture away from the middle it was generally in the right direction. I am confused.
Ellie, I think maybe you were insufficiently confident about your correct answers, or too confident about your incorrect ones. That’s if I’m understanding the test correctly.
I got “Your RQ score is 47. Such a score is low.”
However, I note it seems rather US-centric, and that you can’t correct an answer when you misclick.
(And I’m not entirely sober!) ;)
I’m not sure if I’m convinced by the weighting method. For example, my curve was bulging up on the right and down on the left, which means that when I thought something was true but expressed doubt, it was likely to be true and vice versa.
So I was penalised underestimating myself! *shakes fist*
Lesson: express less doubt. Arrogance is a virtue, because I’m always right.
Aargh! I got 84, in part because I was certain that El Salvador is not in the Caribbean (my brothers used to surf there), so I said absolutely not, which is what the question asked!
So, a decent risk quotient despite spotty reading comprehension.
It does occur to me that psychologists have a long tradition of magic-style experiments – that is to say, when you think you’re watching the experiment, it has already been done.
After you’ve completed the test they ask you to give them your location and the answers to a couple of (more- or less-) risk-related questions, and it’s only if you agree to that that information privacy is mentioned. I’m fairly sure that this isn’t the case (say 90%?), but I wouldn’t rule out the main body of the test being bogus and the ending being the payload for them.
(Also, they have a strange view of the UK. In the “where I’m from” dropdown they allow you to be Northern Irish, Scottish or Welsh – but not English. If you’re English, you have to be British).
Did you notice they differentiate between “Dutchman” and “Dutchwoman?” And I’m surprised no Canadians have made shrill noises about US-ians having no option to choose but “American.”
Okay I got a RQ score of 1
This was deemed a very low score….it shadowed the diagonal…but I have not a clue as to what that is supposed to mean…
#21
“The RQ score ranges from 0 (low RQ) to 100 (high RQ). Your RQ score is 1. Such a score is very low.
Risk intelligence can be measured by calculating something called a “calibration curve”. The graph that is displayed above, is your calibration curve
A perfect calibration curve would lie exactly on the diagonal line, so the area between the curve and the diagonal would be zero. Nobody is perfectly calibrated, but people with high risk intelligence come very close to this ideal.”
My line was directly on the diagonal so apparently I have High risk intelligence…but low score…go figure…I tried have not got a clue what they are on about…I give them 0% for clarity.
I scored 87, which is apparently “very high”.
I clicked on the link and the machine told me I had a RQ of 1 – and I hadn’t even done anything. Oh well, I guess I should be happy I’m a fundamentalist creatard who knows everything ’cause the bahble tells me so.
If you’re getting a result of 1 without even taking the test yet, try using a different browser or a different computer, updating your browser, or updating Flash.
My RQ is probably the same as my IQ considering I got 63. The questionaire was very oriented to North America and so I just said 100 percertain to a few questions I had no idea about. If you’re gonna be wrong, be really wrong! :)
They lied about the test only taking five minutes. Googling all the answers took us nearly half an hour…
What?
Oh.
I got 77 but in my defence I’ve had 3 glasses of a very fine sauv blanc….Hic…
#25
Yes you are right SGBM it clicked but obviously did not register…I also think that maybe you have to allow the test results to be released to them…
“The RQ score ranges from 0 (low RQ) to 100 (high RQ). Your RQ score is 83. Such a score is high.
Risk intelligence can be measured by calculating something called a “calibration curve”. The graph that is displayed above, is your calibration curve
A perfect calibration curve would lie exactly on the diagonal line, so the area between the curve and the diagonal would be zero. Nobody is perfectly calibrated, but people with high risk intelligence come very close to this ideal.”
Still have not got a clue as to what it signifies!
Either extremely stupid or extremely ridiculous…no idea!
Interestingly, I recently re-took it and Googled the 5 answers to which I responded either 100 or 0; I had them all correct¹. Roughly half my answers were 50 (I had no idea — I mean, Bogart’s wives or Harry Potter or Cool J?! I didn’t even check those, I just don’t care).
I suspect Kludge @19 has the most insightful comment here, so far.
—
¹ Well, I didn’t check that Canberra is the Capital of Australia. Perhaps that one is wrong.
Hm, average (70). Average, me . . .?
Maybe I should have read the instructions first, but REAL MENTM don’t read instructions!
Went for 0% or 100% on all. That was maybe not the idea?
Now the LLCoolJ question I got right. I guessed though, I had no idea at all but the answer just seemed too ridiculous to NOT be real.
Guess I now got me some street cred amongst you geeky peeps in that I am fully now aware of the funky music ( said in best AliG voice)
I’ve never even heard of this “LLCoolJ” person, so I picked 50% for that one. Popular culture is, I’m afraid, not my forté.
Okay, I switched to a different browser to take the test again without screwing up the El Salvador question, and trying to replicate the answers I gave on the first pass (and having polished off the bottle of Tuscan wine and gone halfway through a bottle of Rhone) and got 83.
Perhaps on the first pass I wasn’t certain that Wilson wasn’t on the $100,000, whereas on the second pass, knowing that I’d gotten it wrong the first time, voted 0 when perhaps I only voted 20 before. I can’t really say.
I’m not going to take it again, but perhaps someone else could try it, first being dead certain about the “right” answers, and then with the same answers but less confidence.
84. I put 100 down for some I was only pretty sure of, I think that helped my score. This test is awesome!
I tried it again, with a third browser, never going outside the range of 30-70, trying to stick to the same answers I gave before, and dropped to 72. So, unless they privilege Firefox over Opera over Safari, it’s advantageous to be certain when you can. (Surely it’s not this last glass of Caves des Papes, $4.99 at TJ’s.)
wasn’t that the point?
—
Did you check the explanation page?
http://projectionpoint.com/calibration_curve.php
So, if you take all the answers where you were 100% certain, 100% of those should turn out to be true if your self-assessment is correct. Out of all the answers where you picked 90%, 90% should turn out to be true*. etc.
(*hmmm, sample size?)
Yay! 92, despite being “British” whatever that means.
#33
“Popular culture is, I’m afraid, not my forté.”
Neither is it the forté of the law profession in its entirety…so not surprising.
Lawyers seem to spend more time calculating their expenses rather then understanding the modern world!
Being able to advise is one thing, but not worth a jot of justice if they fail to grasp context on that which they pontificate on.
As for Judges they seem notoriously stuck in the 18th Century, some appear to have actually been born around then!, anyway whatever they have not a clue as to the actual contempt they engender in the populace in general.
A damning indictment of their relevance and competency in dispensing the Law.
British Law is supposed to be the system to aspire to, maybe it was, not so sure these days….I mark it as 30% NO!
92 also.
Bollocks. “user account suspended”
I suspect they couldn’t handle the traffic.
I think there may be a flaw in the test.
I wasn’t sure if there was a ‘trick’ element to some of the statements, in that an element of it might be given slightly wrong, although the general gist of it appeared to be right. Were we to suppose that the question setter was trying to trick us or not?
I guess i wasn’t the only one to experiment on this, while others probably went right in with the expectation that the given statements were 100% precise.
It probably means that i’m really flaky.
Site seems to be down now – I get a message about “Your connection to this server has been blocked in the firewall.
As a result of invalid password or access attempts, your local computer has been blocked by the LetsHost firewall.”
I did get in and answer a few questions, but this error appeared halfway through.
LL Cool J, 1987. I understand he danced like that willingly.
Yes, Canberra is indeed the capital of Australia. That was a very easy 100% for me :)
At least one answer given in the test is wrong. For instance, Puncak Jaya in New Guinea (an island) comes in at 5 030 m, whereas Mauna Kea in only 4 205 m.
This statement is actually false. Lions are mentioned throughout the Bible, and they are cats. I suppose they may mean domestic cats, but that’s not what they asked.
I scored a 77.
I’m 90% confident that 20% of their answers to their questions are wrong. I just don’t want to spend time finding out that I’m wrong…
At least one answer given in the test is wrong. For instance, Puncak Jaya in New Guinea (an island) comes in at 5 030 m, whereas Mauna Kea in only 4 205 m.
From the Ocean floor to 4205m or whatever above sea level gives a height greater than Everest. Perhaps that’s what they were getting at.
That’s exactly what I was thinking, and I answered with the understanding that lions are cats. :(
61% Reassurringly average.
I think that reflects my personality but I’m not sure I want to risk being wrong
I had the same issue. I have read that some people tend to do terribly on multiple choice and true/false tests because of their propensity to over-analyze and find multiple meanings in the parsing of the questions. Like the one with Putin – I know he’s the Prime Minister, but is the question saying “president” as in title, or as in generic leader of the country? Also with the lions, as well. I got a 77, and I’m still not sure what that means. Does that mean my confidence is tracked to my accuracy 77% of the time, or what exactly?
I should say it as I get the calibration curve (thanks to windy), but I don’t quite grok how the number corresponds to your fit to the curve.
Yes, but as I understand it, the geographic convention is that the “highest” mountain is measured from sea-level, whereas the “tallest” mountain is measured from the mountain’s base. So Mauna Kea is the “tallest” mountain in the world whereas Everest is the “highest”.
I would guess it means that 23% of the area (of the whole graph) is in that space between your curve and the diagonal line. But I really don’t know.
vanharris@#42 I was the same: scared of being to definate unless they had sneaked a QI-style “well-known falsehood” or urban myth into the question.
The test does seem to be measuring a weird mixture of what you know and how accurately you predict certainty. The top scores are only available if you can to both, right? I suppose they could have tested against a “purer” measure of predicting one’s ability (which would probably have to be longer) and found it a decent proxy.
Ah yes, I see. Thanks, sgbm.
There is an email address there, if people would like to inform them of the ambiguous and misleadingly wrong nature of some of their statements. :)
I got 87 but I had to answer 50% for the great majority because I honestly didn’t know the answers. I tended to answer in a very polarised way – I either knew the answer with 100% certainty or I didn’t (50%) On just a few I answered 90% / 10% because I wasn’t quite sure, but sure enough.
By the way, one of the few questions I knew with 100% certainty was the LL Cool J one. I don’t like to think too hard about what that says about me.
Scored 87. Had to put a fair few questions as 50%, since they were too US-centric. Got a couple of 100% guesses completely wrong.
Hardest part was a few more minutes searching for my UK nationality…
I’m ashamed of many of you, how could you live through the 90’s and avoid one of the many interviews where LL Cool J explained the origins of his name?
Also, Bogart was amazing, and I’m kind of obsessed with old cinema, but somehow I wasn’t aware he’d actually married Bacall. Go figure.
51% – boringly average.
As with others, vanharris’es ‘trick elements’ and my ignorance of American culture and history probably gave me too many conservative 50%s to get a more exciting score.
Maybe social upbringing is a factor too as IMO there is no weakness in not knowing something but to make a false statement with the air of certainty is akin to dishonesty. The results might well be marred by the reluctance to make a prediction rather than the accuracy of the predictions themselves, nevertheless it was fun doing it.
Oh, btw, I got 83. I’m not sure that marking a lot of 50%s will lower your score, as admitting you don’t know is kind of the whole point, isn’t it?
74%, but even for a test on the internet questions were quite USA centric. might have to redo it when I am a bit more awake
#39:
I award you an SQ (starfart quotient) of 41%. Disappointing.
I like their concept, but agree with others here that the execution could have been better. The cats in the bible question was ambiguous, and I’m sure that some people could well refer to gout as “the royal disease” because it is associated with rich foods/affluence.
Likewise, natural gas has an odour after it has been artificially added as a safety measure, Putin is indisputably the Russian head of state, but is the correct term ‘president’ or ‘prime minister’? Does the San Andreas fault *form* a plate boundary, or does it lie on *part of* a plate boundary?
But anyway… the Dunning-Kruger is strong in this one still: only a 68 :-
I started taking the test. My first mistakes took some of my motivation away. Then the site started telling me “This statement is” when I answered. Nothing more. That one sucked away the rest of my motivation.
The people behind the survey should have answered one test for themselves:
Our server will be able to handle the load.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
This statement is.
:( can’t find mine out.
I got this in the middle of taking the test and now this is all I get from the whole site. Just me?
“Account Suspended”
They weren’t ready for the kind of traffic PZ can move.
The problem is that, being the most popular science blog on the whole web, Pharyngula has much the same effect as slashdot once it’s readers are directed to a site not geared to handling simultaneous users in the hundreds of thousands.
A site hosted on a single server (or even worse, sharing a server with others on a hosting service) is not likely to cope very well.
Too bad for them. They aimed too low. Now they not only missed an opportunity to collect loads of data, they botched the whole thing. That’s RQ for ya.
Putin runs the place. He used to be President, but was term limited so he is Prime Minister now. PM is head of government. Pres. is head of state.
I suspect this is not really test on RI, I think it is a test on how much faith people have in what they read on the internet.
I scored an 80 but I got more cautious as I went because they were telling me when I got a question right or wrong. They should not have told the answers.
I too missed the cat question thinking they meant any cat not domesticated cats.
1? That’s a poor RQ for a glass of water.
gah. I hate this shit. Ambiguities and exceptions abound. True or False? IT DEPENDS! The more you know, the more likely you are to know more than the clowns who devised the test. But they’ll score you wrong for being righter than them.
It reminds me of when my daughter was administered this general sort of test when she was very young. She was involved in somebody’s research project (on-campus daycare/”Child Development Laboratory”) and the one example I remember was she was shown a picture of two animals, one in front of the other. The question was “The cat [or whatever] is ______ the dog.” She got marked wrong for saying “chasing” instead of “behind”.
That pissed me off.
I got an 82, but I answered most of the questions 50% because I didn’t have a clue.
Right, so all 50%s gives a middling score even though it may represent a perfect description of the state of you knowledge. It seems like the proper test should be:
ask a lot of trivia questions plus “How sure are you?”;
see how close the proportion of answers to questions marked as N% certain is to N%;
combine those values in some sensible way.
(Probably worth making people answer even they think it is a 50-50 guess; they might have better instincts for what’s true than they thought/knowledge hidden somewhere out of conscious reach).
Also I think it would be better to have questions more “symmetrical” than “statement is true” and “statement is false”. Like” When did EVENT occur: a) DATE A or B) DATE B?” As it stands if you are the sort to over-think things you could be thrown off from giving sensible answers by the fact that there are vastly more false statements than true ones*. Unless you know something about the process used to choose the statements, it can be reasonably argued that you should assume them very likely to false unless you have evidence otherwise. (Like gods and space-teapots and such).
*Of less than some fixed length, at least. If we allow arbitrary length then “P maps to “P is false”” provides a 1-1 map from false statements into true statements. We are presumably only dealing with statements shorter than the number of bits our universe can support.
I know precisely nothing about RQ except what the test told me, but is it possible the ambiguity of the question is deliberate?
I thought the question about natural gas having an odor was ambiguous. As pointed out above, it doesn’t have an odor on its own but does have one added to it for safety. I answered that one at roughly 30% only because I wasn’t sure what they meant, but had a reasonable idea they meant natural gas that comes out of the ground, not out of the pipe.
Think this could be really interesting if it was done a bit more scientifically.
Like with Risk – Reward – Consequences type questions, not just a know-don’t know string.
As it was, there was not much risk or risk assessment involved.
82 for me as well… I’ve never been more completely ambivalent about a score in my life, I’m fairly sure…
Or maybe I’m only about 80% sure.
Damn.
I got a 1 but it appeared to crash between submission of my answers and the scoring. I’m clinging to that hope.
Or maybe #19 above is right and I got dinged for my habit of buying a single lotto ticket twice a week for sentimental reasons.
Scored an 81 mostly because I had no idea on most of them, and embarrassingly, I screwed up on the Paula Jones question. With gossipy facts such as the ones it tested us on, there is not much use in being right, not much risk in being wrong, and it is very easy to correct oneself.
pffft
“account suspended”
Now I’ll never know!
Ol’Greg – yeah – me too! I tried once in chrome and it basically died – went to FireFox and finished, had two resets then got Account Suspended.
WTF?
Have we busted their system?
JC
“Account Suspended” — They didn’t display the answers to many of the questions.
As Bogart was married three times before Bacall, technically he was married twice before Bacall. How did they score that?
Some sources say the U.S. federal government issued bank notes in 1862, some 1861.
Others have mentioned the cats in the bible issue, and the sea mountain issue, and the focus on North America.
I also believe they don’t account for people who have been told from authoritative sources something which happens to be wrong.
I give this an “A” for effort, but only a “C” for implementation.
Apparently, they put the site up at the start of the new year; so, about 34 days from start to smash. The site operators also have a Facebook app for the test… which, however, is also broken while the site has problems.
Poking at the Google Cache of the site still turns up some interesting background on the idea.
I suspect accurate probability estimation might have a link to logical-mathematical, but not a strong one; however, that’s just me guessing.
Possibly of more interest to Pharangula’s godless heathens is the tentative link between reduced ability with probabilities and increased acceptance of the “paranormal”. See “Paranormal belief and reasoning”, Dagnall et al (doi:10.1016/j.paid.2007.04.017); and “Paranormal belief and susceptibility to the conjunction fallacy”, Rogers et al (doi:10.1002/acp.1472).
Which possible connection between low “risk intelligence” and acceptance of the paranormal might (among other things) shed new light on the perennial popularity of Pascal’s Wager.
I would think ambiguous questions would mess it up. It’s rating your confidence in knowing the answer to a question, so if you don’t have confidence in the question itself, that would influence your confidence in knowing the answer because you aren’t sure what the question is.
gr8hands – I think they scored the Bogart question “False”. I am about 78% certain of that.
92% certain this is JC.
I couldn’t get past the captcha. I think I’m a robot : /
I must have typed in 10 different captchas, but it rejected all of them. I tried all lower case and all upper case, but nothing worked.
Is that the point of the test? Measuring how long you’ll type in new captchas until you get sick of it and close the browser?
Can’t get past the captcha either. Tried two different browsers, same results.
It appears to be busted.
I had an amazing Dogfish Head Peche last night. Low ABV but for an American-brewed beer it was a very good approximation of a lambic. Too bad I only got the one (in a build-your-own-sixpack). I’ll have to go back and get more this weekend.
In college I actually took exams based on this. It was a multiple choice test and next to each response they had us put the probability that it was correct. They then plugged in the probability into a log function and that was how many points you got for the question. (.25 = 0)
Hilariously if you put 0 next to the correct response you got negative infinity and failed the class. Even more hilariously, some people actually did this.
Now that the site has been busted, my experience concerning the slashdot effect says that it’s best to wait a few days before trying again (if you’re not already too pissed off with them, that is). If you’re still interested, drop by again on Sunday.
Same thing happened when Last FM launched.
I tried easily 10 times to enter the Captcha code in Opera. It simply refuses. It doesn’t even show an error message, but simply gives me a new code. I can read them just fine, thank you.
Then I tried it in Firefox. Same result.
WTF? Do I need to update my Flash player or something? (In that case, I’ll just try again from home…)
Before you get all smug, how many scientist nerds do you think know about such boring topics as rap music?
And it’s somewhat ridiculous to spell forte with an accent. The word is Italian, not French or something… Apparently the accent is just there to make English-speakers pronounce the e, like the dots of Emily Brontë and the silly accent of Pokémon.
Come on. How could “president” possibly mean “generic leader of the country”? Russia has a head of state with the title of president (Dmitriy Medvedev), and – officially – Putin has no power over him.
And “cats” is obvious, too – it’s obvious that domestic cats are meant. Calling lions cats is zoology.
WTF. That doesn’t even work outside of English!!!
And what does it even mean to talk about “the base” of a mountain?!? A bit of geology should disabuse anyone of that notion very quickly indeed.
<leaning back, smug grin on face>
Interviews? What interviews? There wasn’t any in the Austrian TV or radio news.
…but in fact it’s scrofulose, the illness that the touch of the king was believed to heal.
Seriously. I’m not making this up.
It is entirely disputable that Putin is the Russian head of state. By law, he’s not. He was for a long time (when he was the president), but now he’s the prime minister instead.
Wasn’t this in the news where you come from? Many people wondered if Putin would get the constitution to be changed so he could run for president a third time. He didn’t – instead he installed his sockpuppet as president and became prime minister ( = head of government).
This isn’t the USA, people. Head of state and head of government are different jobs!
It is part of the boundary between the North American and the Pacific plates. Other parts are the midocean ridge of the Gulf of California and the faults and subduction zones off Alaska.
Moronic.
To me it’s obvious that this is a typical nerdy trick question – natural gas has no odor, a mixture of natural gas with a stinking agent has an odor. The answer that natural gas has an odor is supposed to be the obvious one, and therefore wrong…
<sigh> No, he was married more than twice before Bacall.
according to the Smithsonian Press there was in 1934 a $100,000 bill (gold certificate) issued though never circulated to the general public, with Woodrow Wilson’s face on it [Viewable here, They even have one on display in the Smithsonian]. The test however claims that the sentence “The face on the $100,000 bill is Woodrow Wilson” is false.
Damn their captcha… Their logins are worse then the logins used to be here.
Yeah, me too…
(/Man, I don’t think there’s anything more embarassing than failing a Turing test.)
Has anyone figured out how to get around the captcha yet? I have javascript enabled, and enabled third-party cookies, but it still won’t work.
I know I’m kinda dumb so could someone explain to me how the curve actually works ?
I read the explanation page and to use windy’s recap, I understand that “if you take all the answers where you were 100% certain, 100% of those should turn out to be true if your self-assessment is correct. Out of all the answers where you picked 90%, 90% should turn out to be true*. etc.”
That’s why they write that to have a perfect curve, x=y.
But how do they plot the other values if you only use some of the confidence percentages ?
In which case the test answer is totally wrong. The bill, while rare and discontinued in 1940, is still considered legal tender and does in fact have the picture of Woodrow Wilson on it… so what am I missing?
I can’t get around the Captcha either, I wonder if the site is worn out :)
Well, I can’t get access to the test right now but I did it three times yesterday night and if I remember correctly, the answer was indeed “True”. But I may be mistaken.
When I took it, I answered true and the test concurred.
that’s odd when I took it this morning the test told me it was false.
I was very upset, as I knew this to be incorrect and I went to my email tab, sent them an email about it, then came back and tried to finish the test and it gave me some message about my test being voided or something.
do you guys suppose that the test may be designed to occasionally give you the wrong answer as you’re going, to throw you off of what you think you know to see how it impacts your percentage answers?
Or am I just being paranoid again?
I love how every commenter scored a very high score. Pharyngula is full of RQ geniuses!
Somethings weird about that… maybe all the studies that show that the average person has a fairly low RQ are wrong. Hmmmmm…..
Haha.
#97:
Interesting. You live in a universe where “obvious” and “zoology” are mutually exclusive?
Can’t get past the Captcha either :( (and I’ve tried IE, Chrome and Firefox)
Yup, Captcha just repeats ad nauseum. If they can’t get a simple thing like Captcha right, I don’t think I would trust the rest of the test anyway.
Following on from the incorrect answers issue. I was told that Wilson was truly on the 100k bill (got that one wrong). However when asked about something I do know – whether prostate is the most common cancer in men I – correctly – answered no. Squamous cell carcinoma of the skin is much more common, but usually not serious. Prostate is the most common non-skin cancer in men and the second most common cause of male cancer mortality in the U.S. (after lung – and that’s getting close). However it’s another example of them not knowing the answers and thus skewing their own data.
Overall not impressed – long lags and I only got their answers to some of the questions, leading me to wonder if the others were being tallied. Also their e-mail address comes back as undeliverable.
Screw this. I tried typing in “USUCK” and “FECKU” to the captcha, and while they didn’t allow me in to the site, they did change my response to “the code you entered was incorrect”. Heh.
I despise captchas. For some reason I often have a hard time reading them, and not every site that uses them has a sight-impaired alternative. I’m not sure what it is; I’m nearsighted and astigmatic, but nothing other than that, yet the captchas elude me. Perhaps it’s that I’m a robot in denial.
David M wrote:
Is this an RQ test? The English word “forte” *is* French in origin, not Italian, and it’s pronounced “fort.” And yes, it’s wrong to put an accent on the last letter.
I’m pretty sure about that.
Why would they put a captcha on a test anyway? I can understand preventing spambots from commenting or preventing a spammer from signing up for a million email accounts, but why stop spambots from taking intelligence tests?
ric.baker1:
But they’re doubly self-selected.
Pharyngula’s readership really is an unusual subset of the population, and of course, you’re only seeing the ones that want to report their score.
Captcha just repeats ad infinitum as others have reported.
My confidence in the correctness of my captcha entry was 100%, but apparently it was wrong. I guess my RQ is 0.
Bloody blockquote/paragraph break bug.
First paragraph after the blockquote ends should also be included.
I doubt it changes anything but could it be because U.S. is not the only country considered ?
Maybe they took it from Wikipedia :
“Prostate cancer develops primarily in men over fifty. It is the most common type of cancer in men in the United States, with 186,000 new cases in 2008 and 28,600 deaths.[122] Prostate cancer is also the most common cancer among Canadian men.[citation needed] It is the second leading cause of cancer death in U.S. men after lung cancer. In the United Kingdom it is also the second most common cause of cancer death after lung cancer, where around 35,000 cases are diagnosed every year and of which around 10,000 die of it.”
It’s actually a study to see how many times they can get people to submit a CAPTCHA form before they give up
@ 117 I was thinking the same thing.
If a spam bot is navigating this test and submitting results I want to know if it is actually accurately reporting how much information it has on the questions and estimating from that data because if so then that is a pretty awesome program some one wrote.
I mean, yes I have a record of that information or no I don’t isn’t that impressive, but recognizing a random batch of questions and then recognizing how much recorded data might pertain to each question is sweet.
I don’t think that would be a “spam” bot though, and I don’t know why you’d be so afraid of it getting in to take your quiz.
BdN #122
I think you may be right! Got to love a definitive answer that relies on Wikipedia. Just pulled up the 2008 US statistics, they don’t even count basal and squamous skin cancers, just note that there are over a million cases per year (both sexes so about half male). They come up with essentially the same numbers as you do for prostate, around 186k. To my mind that puts the number of prostate cancer CASES a distant second.
I bet the relative numbers are pretty much the same in the UK, and indeed pretty much worldwide, as in the US. Perhaps a little less skin cancer in the UK due to the persistent lack of sunshine, as long as you don’t hit the mediterranean beaches too hard!
Program won’t work for me, it just keeps wanting to verify I’m human.
They were having an SQL DB connection issue for a bit.
Just for clarification, the person who is posting as “waltonan” and/or “ahwalton” is not me.
Walton, given your recent heavy intake of “unusual Rwandan coffee”, can you be absolutely sure of that?
;-)
Am I wrong to think that a low score is what is “desirable” with this test? Doesn’t a high score indicate a strong discrepancy between how well you estimate your knowledge and what it actually is? It seems to me that all these high scores indicate a lot of overconfidence or underconfidnce (I know which I would guess) rather than accurate self-knowledge.
84, btw (very high)
“The RQ score ranges from 0 (low RQ) to 100 (high RQ). Your RQ score is 100. Such a score is very high.”
Woohoo! The first time I think I ever scored high on a test for all the things I don’t know. I started to feel like a real tool towards the end with all the things I don’t know.
It is phenomenal in our culture though, this desire to pretend to know.
Not that long ago in my office I got criticized for saying I wasn’t sure if one piece was complete and functioning. They said, but you were running it. And I said, yeah… it worked once. They said how sure are you that it’s ready, what percent? And I said, about 70% sure.
And they said… that’s it then, it’s ready.
NO! NO it isn’t! Dang ppl.
But whatever.
Follow up to my comment above, I think I was wrong that a low score is “desirable”. I just went through again and entered 0% for every question and got a 17%. Carry on.
Chris @ 130
“Am I wrong to think that a low score is what is “desirable” with this test”
No, a low score I think would mean you incorrectly estimated your likelihood of being right. For instance you said you were 0% sure which means you were sure it was false with not doubt. If that answer was in fact true then the 0% would lower your score more than the 50% would have if you’d said you were unsure.
So admitting to being less than sure would have raised your score.
I’m a newb poster here, but I really had to show off my score: “Your RQ score is 92. Such a score is very high.” I say this with gloating and some heavy chestthumping.
Not bad, considering I didn’t even know there was a 100’000USD bill ;) … are you guys in the US expecting so high inflation?
The test says:
The world’s highest island mountain is Mauna Kea
This statement is True
Uh, I’d have to disagree. It may be the tallest mountain, but it’s not the highest.
As of now, the program let me past the captcha. Either the site was pharyngulated for a while, or they fixed something.
My score was 75.
My RQ score is 97, which it says is very high. I answered the things I was certain about with either 0 or 100, and if I was at all uncertain, I put 50. To me, the point of the test seems to be that you should be confident about the things that you know, while not being afraid to admit that you know nothing.
Daemonward, you are right. I just tried it by answering the first question 100% and all the rest 50% and my score was 99.
Hmm. I got 54%. But that doesn’t surprise me; I knew I wouldn’t have much confidence going in, plus some of the questions were confusing, as mentioned above, and I got frustrated the more I got wrong. -_-;; Telling people the answers as they take the test is odd.
I got 83 also but I had no idea about a lot of the questions so answered 50% or sometimes 40 or 60 only one or two I knew to be absolutely true or false. It’s a bit US centric which does not help being Australian, except when it asked the capital of Australia of course :)
A rat in a (BTW, your handle never registered with my brain until it was finally explained to me when you got the OM. And I have that Smashing Pumpkins album. Go figure) cage, that proves the thing is just broken. Or at least the equations they use to analyse the results can’t handle statistical anomalies…
Carlin put it best:
The 100K bill was used for money transfers between banks. It was never put in general circulation. It was superseded by wire transfers in the 1930s and withdrawn by 1940.
-Yer friendly, neighborhood ex-Treasury official
Oh by the way, my score was 74, which is supposedly high.
I don’t know what an LL Cool J is.
After posting my comment earlier, I sent an email to the contact email address at the site. I’ve been busy today, so I didn’t check my mail until now. This is what I got back:
Hmmm. I don’t get a good feeling about this site. To those of you who got in, did you have to enter any personal information like an email address?
I scored 87.
A lot of people seem to be missing the point of this test. It doesn’t matter whether you get the answers right or not. It’s measuring how well you self assess your competency. It could be entirely US centric and it’d still produce somewhat useful results. Mind you I’m pretty sure a wide range general knowledge questions will produce more useful results.
Questions outside of the testee’s scope of knowledge would make the test more useful now that I think about it. The designers want to see how people rate their own knowledge when they really have no clue.
And isn’t some of the speculation here about the motives of the testers an argument from ignorance? Just because you don’t understand the purpose or methodology of this test doesn’t mean that it’s invalid.
MetzO’Magic, yeah (also, >;) on the nym). I tried other combinations and it does appear to penalize a lack of confidence, boosting your score into the “high” range only if you get the non-50% answers right and are totally sure about them (unless, like you say, it can’t handle outliers). It doesn’t appear to even count 50% answers.
Test results by answering:- Googol 100%, Africa 0%, all else 50% = 100
- (Googol 40%, Africa 60%) and vice versa, all else 50% = 52
- Googol 70%, Africa 30%, all else 50% = 52
- Googol 100%, Mauna Kea 60%, all else 50% = 98
- Googol 60%, Mauna Kea 60%, all else 50% = 50
- Googol 0% and all else 50% = 2
- Googol 10% and all else 50% = 51
- 3 of them at 50% and all else correctly at 100% or 0% = 100
- 7 of them at 50% and all else correctly at 60% or 40% = 39
- 6 of them at 50%, 1 correctly at 60%, all else incorrectly at 60% or 40% = 2
- 1 correctly at 10% and all else incorrectly at 10% or 90% = 1
@ FossilFishy #147
Theres a differnce though, between testing someones knowlege in something they arent sure about (e.g. they heard or learnt it ages ago but cant remember very well, or it is vaguely talked about by those around you) and something they have no idea about because they never learnt anything about it. Surely it cant help the statistics of the test when most of the answers are 50%.
semeisie #149.
But that’s the point. To you, and me for that matter, the answers in most cases are 50% because we know that we don’t know anything about that subject. To someone who displays the Dunning-Kruger effect they’re going to answer something other than 50% even though they don’t know anything about the subject. The point of this test is to show how many people don’t realize just how little they know.
The title of Dunning and Kruger’s paper was “Unskilled and Unaware of it.”
A good video about it:
I’ll also submit that MetzO’Magic and others who are saying that the thing doesn’t work are displaying the Dunning-Kruger effect quite nicely. It’s not a knowledge test, it’s a test of the accuracy one’s self-assessment.
I got a 92. Which means that I am quite smart and sure of myself? I 80% believe that. Haha.
I got a 76.
This test seems to penalize people greatly for luck. It seems to graph what percent of the answers for which you answered 0, 10, 20, etc. for were right, and then compare that to the diagonal graph. The trouble is, you are rarely going to be 90% certain of something, (usually you’re either 100% certain or much less certain) so if you only put 90% for one question and happen to get it wrong (a 10% chance, given completely accurate self assessment) you are going to get a huge penalty for it, just due to bad luck. I suspect someone with perfect self-assessment and average knowledge would get much less than 100 on this test, on average, purely because of this phenomenon.
seemeisie @149
Whether you can identify that you a) know something for sure or b) have no clue is one question, but not a very interesting one. Your ability to assess how sure you are about something you think you may or may not know the answer to is a lot more interesting, and being able to mark “50%” to a bunch of questions you have no clue about is exactly useless for that.
Yeah, but a lot of people who don’t display it still are not very good at assessing how much they know when they know more than nothing and less than everything. Which is way more interesting, really.
@FossilFishy
I think people who question the test understand very well that it’s not about knowledge but self-assessment. But they see two problems :
1- Some answers are wrong so it does have an impact if you get the answer or not. If you are sure of the answer, because it is the real answer, but the test gets it wrong because it was poorly written, you come out as having too much self-confidence when it is not the case.
2- There seems to be a problem in the way the curve and the final score are calculated as clearly shown by aratina cage @148. If only one answer doubles your whole score
well, there is something… fishy about this.
If you read the explanation page, that’s not exactly how they explain it.
BdN #155.
Thanks.
1. Agreed.
2. I didn’t look at aratina’s numbers carefully enough. Yup, I don’t understand how they’re coming up with those scores.
3. Hmmm, the explanation page(s) really do muddy the waters don’t they? I don’t see how they get from accuratly self-assessed confidence in personal knowledge to accurate assessment of probabilities.
I wasn’t trying to imply that everyone didn’t understand the quiz. Hell, I’m no longer sure I understand it. I was mostly trying to address those comments that complained about the American bias of the quiz and I did a poor job of it.
Whether it’s fishy: badly designed, or fishy: dishonest, I can’t tell but yup, fishy it is.
No problem !
As for the American bias, I think it mainly becomes a problem because of the way the RQ is evaluated. As far as I understand, which I’m not quite certain, even though it’s not about knowledge, it seems that it would be hard getting a high score if you don’t know most of the answers. Moreover, people may be panicking slightly when realizing they are hopeless, as in any quiz.
I must admit I’m not good enough at statistics nor knowledgeable enough in this kind of experiment to judge if it is badly designed or dishonest. I guess I would have to read more about the subject but quite frankly, I think my bed awaits.
The only thing I could come up with is the Wiki page about Calibrated probability assessment :
“are subjective probabilities assigned by individuals who have been trained to assess probabilities in a way that historically represents their uncertainty[1][2]. In other words, when a calibrated person says they are “80% confident” in each of 100 predictions they made, they will get about 80% of them correct. Likewise, they will be right 90% of the time they say they are 90% certain, and so on.”
So it seems that you obtain this percentage only after training, which is kind of hard when it’s… the first time you try. Furthermore, since questions in this particular test are either true or false, it leaves us with a strange situation since both sides of the 50% mark are evaluating your confidence.
For example, 80% in the case above is the level of confidence in your answer. But in this test, 80% is your confidence that it is true and 20% is your confidence that it is false. But both become positive assessments in the sense that contrary to the real test where 20% would mean that you really are unsure, here, it means that you are quite sure.
So it would make sense in the original test that you get only 20% of the answers for which you wrote you were 20% confident since you hesitate for all of them so it’s normal you fail most. But in the second case, it makes less sense. Why would I get only 10% of the answers right for which I am almost certain they are false ?
Sorry if I’m unclear. A few drinks and I need to go to bed.
I just realized that it’s written on the Wikipedia page that “[i]f a person has no idea whatsoever, they will say they are only 50% confident” and it stops at 50%, there are no lower scores.
So either the test is wrong or they calculate 20% as 80%, 10% as 90% and so on, in which case their explanation page really needs a clean-up.
Just to add to aratina cage’s experiment :
-Googol 100% and everything else at 50% = 99 with an almost perfect curve (little, almost imperceptible, bulge downward towards 50)
-Africa at 0% (which means I’m sure it’s false) and everything else at 50 % = 52 with a curve forming a triangle with the summit at 50 ^
Strange…
Okay, sorry to be kind of all over the place, but I just tried :
-Africa at 100% (so, incorrect) and everything else at 50% and…….. I got the same result as when I put it at 0% = 52 with a triangle.
What the hell ?
BdN, cf. Kludge @19.
But John Morales, that wouldn’t make sense in calculating the score.
Building on BdN’s experiment, I did another test:
All false at 0%, all true at 50% = 34 (very low) with a triangular shape bisected at the identity line’s midpoint.
Then I read up on their calibration curve. They only plot points (x, y) for percentage categories (0% to 100%) where x = a percentage category you marked and its corresponding y = the percent of statements you marked “x” that were true.
So by answering only 0% for all false statements and 50% for all true statements gives two points: (50%, 100.0) and (0%, 0.0). It must be automagically plotting the 100% category to get the triangle. Since I did not answer anything with 100%, they plot (100%, 0.0). The same works for a single 0% answer with all the rest at 50%.
Now we can see why a single 100% true statement with all the rest being 50% is an RQ of 99: it plots two points, (100%, 100.0) and (50%, 49.0) [assuming half are false and half are true without counting them leaves 24 true statements out of 49], and then automagically plots the 0% category that you did not answer, so (0%, 0.0). By answering 1 true statement 100% and one false statement 0%, it plots: (0%, 0.0), (50%, 50.0), (100%, 100.0)—a.k.a. the identity line.
Hmmm.. I also got a 1. I suppose that’s accurate though, I obviously miscalculated the risk of taking the test…
Finally got past the capcha, and got 95. I see Ol’Greg got 100 – there ought to be an award for that!
Occurs to me that so many sceptical Pharyngulistas could screw up their results. Just how confident are they that their methodology is sound ;-)
That’s most remarkable. It asked the only 50 questions to which I didn’t know the answer. What are the chances of that?
I have a problem with at least one question on this test:
According to the test makers:
“Question 8: Cats are not mentioned in the Bible
This statement is True”
“Cats” is not properly defined. I was sitting there thinking, “Do they mean domestic house cats? Or do they mean cats of any type, because I thought they mentioned large cats a couple times.”
I chose false, because lions are mentioned several times and tigers I think a time or so. Which turned out to be correct, lions are mentioned.
So this question needs to be reworded or thrown out. Seriously, test makers, check your damn facts first.
ElitistB #166:
As long as you didn’t answer 0% — which I assume you didn’t if you were wondering exactly what the question meant — it shouldn’t matter much, so get over it.
All the answers that people are harping on for being incorrect are those that have some ambiguity. The more ambiguous a question, the more likely it is that someone who’s not sure about the answer will get it wrong, but that ambiguity should also affect the degree of certainty in the answer, thus self-correcting the result. Someone else might be absolutely certain about the same question, seeing no ambiguity, but then it’s very unlikely that he/she will answer incorrectly.
Nobody’s complaining with a claim that the unambiguous questions (e.g., googol, Africa) are incorrect.
Woo I got 76! Although one of the questions I knew from someone on #pharyngula who took the test. (and apparently is wrong anyway).
Also the questions about US stuff I had no frikken clue, because I don’t keep up with news about the ex-colonies. Once you’re gone, you’re gone.