Stealth Questions - rubbish ?

jibbajabba

I just read another anti-**** article where Symantec claims to put "Stealth Questions" into their exams in order to see who used brain **** or not.

How is that suppose to work .. Even if a specific question is unrelated to the topic of the exam (I suppose this is what they mean), you may still get it right when you guess the answer ..

IMHO a bit dangerous, isn't it ....

dynamik

What if they're not looking for the correct answer?

undomiel

I'm guessing the stealth questions are questions extremely similar to other questions in the exam with a word tweak here or there to give it a completely different meaning thereby you would pick the opposite answer from what the regular question would give. The brain dumper would probably glance at it and think it was the regular question and give the regular answer.

Just a guess though. I can't see it being a terribly effective method. They probably look for consistency of answer in the stealth questions.

brad-

If its multiple choice, it still wouldn't be a definitive..."this guy used X"...maybe they might use it as their own metric on when to release a new test version or bank of questions, but there's no way that would be used against an individual.

shednik

undomiel wrote:

I'm guessing the stealth questions are questions extremely similar to other questions in the exam with a word tweak here or there to give it a completely different meaning thereby you would pick the opposite answer from what the regular question would give. The brain dumper would probably glance at it and think it was the regular question and give the regular answer.

Just a guess though. I can't see it being a terribly effective method. They probably look for consistency of answer in the stealth questions.

+1 my thoughts exactly

jibbajabba

undomiel wrote:

I'm guessing the stealth questions are questions extremely similar to other questions in the exam with a word tweak here or there to give it a completely different meaning thereby you would pick the opposite answer from what the regular question would give. The brain dumper would probably glance at it and think it was the regular question and give the regular answer.

Just a guess though. I can't see it being a terribly effective method. They probably look for consistency of answer in the stealth questions.

I am not native English though so I might trip over those questions as well ...

rbutturini

I'm in a dispute with a particular certification vendor over their purported use of "forensics" on one of my exams right now. I think you guys may be right on with the method they use, but they won't disclose any information to me. I'll let you know (within reason and the site rules) if I can find out some more specific info.

jibbajabba

rbutturini wrote:

I'm in a dispute with a particular certification vendor over their purported use of "forensics" on one of my exams right now. I think you guys may be right on with the method they use, but they won't disclose any information to me. I'll let you know (within reason and the site rules) if I can find out some more specific info.

Yea - please let us know ..

This is getting silly ... I always study exams to the death and normally quite quick and with a good score and I hate braindumps ..

I always said (as a joke really) that I receive my "you cheated you bastid" note from Microsoft any minute :P

This is really a joke .. In order to catch cheater - legit people who study their ass off getting punished ..

I would even go so far to stop doing exams once I finished my MCITP : EA 2008 .. tired of this bollocks ..

astorrs

Honestly, it's a heck of a lot more complex than what you guys are describing. Unfortunately, I think that's probably the limit of what I'm allowed to say...

dtlokee

I would doubt that they are using what score you received as the only measurement in their forensics. I would think it uses a blend of technologies and I think they will focus on people choosing answers which are wrong, but a **** may provide as the correct answer. If an exam shows a few of those then it may get flagged. They may keep track of how much time you spend on each question that could be a factor, how and where you move the mouse, how quickly you choose an answer, who knows.

I think it's a good thing, put the word out and get people to stop using them. The problem is that there are going to be exams that people take and do it honestly that get flagged, I hope they have a good system of what happens after they are flagged to review the exams.

jibbajabba

Microsoft claims that they false-positive rate is apparently 1 in a .. wait for it ... TRILLION ... World population is what .. 6.7 Billion .. But yea - what do they have .. ONLY a) time taken for the exam and b) score

I was talking to someone in the testing center while I was waiting for a friend of mine .. She was like "man that was quick and almost perfect score - this is normally a three hour exam"...

So what .. Most people might read the full questions, think about it and try to answer them ..
I skip half of the question and read the rest very fast .. let me explain ...

The structure of a Microsoft question is typically made of three parts ..

1. Explanation of the environment
2. Question
3. Selection of solutions ..

for example

---
1. Michael leaves his house and went to work where he was supposed to meet Jill. He realises that he forgot his documents and need to drive home in rushhour.

2. What is the fastest, not shortest way to get home

3. a) Bus
b) Taxi
c) Car
d) Beaming by Scotty

---

So part 1. is just waste of time and can mostly be skipped ... All I am interested in is part 2. and 3.

IF part 2. doesn't make any sense .. I can still go back to part 1. and read about the environment...

The best example is one Windows question I had in one of my Server 2008 exams...

Again - part 1. was mostly BS which nobody cares about .. part 2. was "you need to configuire DNS on Server 2008 Core".

Part 3 had four possible answers

a) netsh interface ipv4 add dnsbollocks
b) netsh interface ipv4 add dnsserver
c) netsh interface ipv4 add dnsbollocks
d) netsh interface ipv4 add dnsbollocks

So reading the question took 2 seconds ... figuring out that three commands out of four are plain wrong anther 2 seconds and ticking the right questions and hit 'next' another 2 ...

So this question, with fook knows how much text was answered correctly in under 10 seconds ...

Am I cheating now ?

Man that pisses me off ...

If they want to stop cheating - God dammit create more questions and not just 1,2 or 3 sets of questions .. I failed one exam and had 70% of the same questions on the re-take ..

You could almost use the "2nd shot" as "brain ****" .. go there without studying .. fail .. go home .. study targeted to the subjects asked ... job done ..

[/rant]

dtlokee

A google search turned up these

http://caveon.com/df_blog/?p=34

http://caveon.com/df_blog/?p=30

http://caveon.com/df_blog/?p=29

All interesting to read.

EDIT: really the whole site is a good read.

jibbajabba

dtlokee wrote:

A google search turned up these

http://caveon.com/df_blog/?p=34.

That makes perfectly sense .. thanks for finding this one

astorrs

Gomjaba wrote:

Microsoft claims that they false-positive rate is apparently 1 in a .. wait for it ... TRILLION ... World population is what .. 6.7 Billion .. But yea - what do they have .. ONLY a) time taken for the exam and b) score

As dtlokee's links show there is a lot more than that to go on. They aren't lying about the false-positive rate either, they require a very high deviation to presume cheating - their goal is to catch cheaters without impacting a single legitimate tester.