Redmond CSI: Anti-piracy and Microsoft Certification

astorrs

The webcast Microsoft did last week on combating braindumps, cheating, etc is now available for viewing at https://www.livemeeting.com/cc/lmevents/view?id=msft062508PMlm&pw=ATT11808.

The what's allowed (in Microsoft's opinion - the only one that counts) based on the NDA is interesting:

Can I Answer
- How many questions about "X" were on your exam? No
- How many questions are on the exam? Yes
- How long (in time) is the exam? Yes
- What specific can you give me about the exam? No
- Can I post or share questions if it is not word for word? No
- What is it like to take an exam? Yes
- How are they graded? No
- What is the passing score? Yes and No (your score and the passing score only)

I have seen many "violations" on the forums then related to the number of questions on a specific technology (I have been guilty myself on this by referring to "2 or 3 questions about this or that" - time to go correct any of those).

Exam retake policy is also changing slightly in Summer '08:
- 24 hour waiting period after the first fail
- 14 days waiting period after subsequent failures
- No more than 5 attempts per 12 month period without prior approval from Microsoft

I would encourage everyone to watch the webcast.

brad-

With that rationale, we can't even repeat the exam objectives.

astorrs

Everyone else agreed and Trika posted the following:

I know the "line" is fuzzy - I wish it were less so. I will try to summarzie the can/can not in a single post pretty soon here, but in the meantime our guidance is ----IF it is on the prep guide, talk about it. If it is not, don't. If you aren't sure you're allowed to share it, don't

---

Here is what Peggy tells me "We want people to use their best judgment and not disclose information relating to the actual exam content. Legit practice test providers and authors are not disclosing actual exam content." More on this in a quick minute.

http://blogs.msdn.com/trika/archive/2008/07/01/braindumps-and-whatnot.aspx#8679062

jbaello

Have you guys seen the "Data Forensic" timestamp 31:24 it pretty much states that they will be collecting information of people that has taken exams when they are not supposed to. As well people that takes so many exams in a period of time, also they will be checking consistent pattern in the answers as well scores, they kinda find that fishy and suspicios.

And when they find out that this particular person is infact cheating based on this "Data Forensic" they are pretty much banned for lifetime.

BEWARE!!!

eMeS

I wish Prometric would "Secret Shop" the testing center that is closest to me....

I don't suspect any cheating or other security problems. My problem is that the place is dirty, the equipment is poorly maintained, and it doesn't seem to be very well setup to be a testing center. The place looks like a train wreck.

I like most of what I hear in the presentation, particularly the part about using independent investigators to research practice test vendors.

MS

Silentsoul

eMeS

When your test are finished do you ever write in those things you just said on your post, Its been a few months since i took a test but don't you get the ability to write in questions and comments, Even if they are only expecting comments on the test you can write it in they may still see it.

eMeS

Silentsoul wrote:

eMeS

When your test are finished do you ever write in those things you just said on your post, Its been a few months since i took a test but don't you get the ability to write in questions and comments, Even if they are only expecting comments on the test you can write it in they may still see it.

Yes, I have every time I've taken a test there. I remember being offered two surveys, one specific to the exam and one specific to the testing process and delivery.

My expectations are low, however, based on the countless posts that I see here about the testing software commonly failing. Thankfully that's never happened to me (yet).

MS

Silentsoul

When I was working on my MCSE track I had 2 test crap out, I had one test that had 10 sims on it, the guys giving the test were very surprised, After that people started have major problems, test locking, failing at login before answering any questions. Prometric was no help at all. A lot of angry geeks is a bad thing

snadam

astorrs wrote:

Can I Answer
- How many questions about "X" were on your exam? No

so I cant even say there was a "bunch" of questions on 'X'? Without giving a specific number? Okay, time to be more vague!

tiersten

astorrs wrote:

- How many questions about "X" were on your exam? No
- How many questions are on the exam? Yes
- How long (in time) is the exam? Yes
- What specific can you give me about the exam? No
- Can I post or share questions if it is not word for word? No
- What is it like to take an exam? Yes
- How are they graded? No
- What is the passing score? Yes and No (your score and the passing score only)

Interesting...

Basically all you can say is the following sentence:

I had <blah> minutes to do <blah> questions and the exam was great/bad because I got <blah> and I passed/failed.

tiersten

Silentsoul wrote:

When I was working on my MCSE track I had 2 test crap out, I had one test that had 10 sims on it, the guys giving the test were very surprised, After that people started have major problems, test locking, failing at login before answering any questions. Prometric was no help at all. A lot of angry geeks is a bad thing

A coworker went to a Prometric center that allowed him to book Saturdays. He said it was in a really bad state, the equipment was old and it wasn't in a good area. It was just him there for the afternoon session so they asked if he would mind starting early so they could all go home. The minute he started, they all left the room and started tidying up. About 15 minutes before the end, his PC automatically started to shutdown. He ran out and shouted at them and they powered on the PC again and some how continued his exam session. Thankfully in the end he passed...

jibbajabba

My local Prometric center was ok ish until I realised that the status of the exam didn't change from pending or passed for like two weeks.

I went now a bit further up the road (well 15 miles ish) and it is ok .. however the first exam I took there was easy and I finished in about half an hour and was looking for that chick ... I sat like 20 minutes in reception until some instructor came along and asked me if I am waiting for a class or exam .. i said that I finished my exam 20 minutes ago but I couldn't find anyone "finishing" me up there .. he was looking for her too and finally found her in the kitchen .. I said "So basically you didn't see me taking my Phone-a-Friend joker" - and she started laughing. All I said was that it wasn't funny NOT to watch the exam taker let alone letting me wait for damn 20 minutes ...

Slowhand

I've seen some good Prometric centers, and some bad ones. The most stark contrast is between one that is a little further up north, (at a little vocational school that will remain nameless,) and the one in Alameda. The center run out of the school has been known to be dirty, the monitors are failing, and the staff is less than knowledgable. On more than one occasion have I shown up for an exam and no one's bothered to show up for work, forcing me to reschedule.

The stand-alone Prometric center in Alameda, however, is very nice. The staff is friendly, neither my roommate or I have ever had any issues taking tests there, and they even give you earplugs to wear if there are lots of people in testing on a hot day and they crank up the (rather loud) AC to keep us from boiling and to block out noise from people coming and going from exams. Then again, it's to be expected that this particular testing center is kept up to date and running efficiently, just by its designation: CA001

dynamik

There's a brand new one by me. It's a great facility with friendly staff, but it still suffers from the Prometric curse. When Cacharo and I went in to take our 293, they said that problems were preventing them from offering any tests that day and we should have gotten a call. I got the call 4-6 hours later. When we were finally able to take it, his exam crashed and they had to move him to a different machine. I've seen other people experience similar problems as well. I'm really at a loss for why it happens. All the machines are relatively new, certainly more than capable of offering an exam. They must have some serious kinks in their software and/or infrastructure.

undomiel

I expect it is problems more on the infrastructure side of things. I only had the exam kick me out once in all the tests I did and according to the on site guy it was due to a time out in contacting Prometric's servers. Of course that is just my one experience.

JDMurray

undomiel wrote:

I expect it is problems more on the infrastructure side of things. I only had the exam kick me out once in all the tests I did and according to the on site guy it was due to a time out in contacting Prometric's servers. Of course that is just my one experience.

I've often wondered if the test computer had to be connected to Prometric/Vue servers to download the exams on-demand, or there was an exam server on premises making Internet connectivity not necessary. If a network connection to Prometric is necessary, software updates can be easily pushed out, so maybe most of these crashes are either faulty hardware or just burps in the Internet.

Slowhand

JDMurray wrote:

undomiel wrote:

I expect it is problems more on the infrastructure side of things. I only had the exam kick me out once in all the tests I did and according to the on site guy it was due to a time out in contacting Prometric's servers. Of course that is just my one experience.

I've often wondered if the test computer had to be connected to Prometric/Vue servers to download the exams on-demand, or there was an exam server on premises making Internet connectivity not necessary. If a network connection to Prometric is necessary, software updates can be easily pushed out, so maybe most of these crashes are either faulty hardware or just burps in the Internet.

From my "encounters" with the not-so-good center up north, it looks like the centers download the exams directly from Prometric. And that would make sense, just imagine how easy it would be for the cheaters if the exams were stored on-site. There have been lots of cases of Prometric (and Vue) employees being the ones leaking questions and answers to the braindump providers.

JDMurray

Slowhand wrote:

From my "encounters" with the not-so-good center up north, it looks like the centers download the exams directly from Prometric. And that would make sense, just imagine how easy it would be for the cheaters if the exams were stored on-site. There have been lots of cases of Prometric (and Vue) employees being the ones leaking questions and answers to the braindump providers.

Protecting the actual exams data on an exam server is simple enough using encryption. What a test vendor must be more concerned about are people rigging the system to repeatedly take the same exams to get all of the questions and answers. If your test machines must "phone home" to start a test then tampering activity can be more easily detected.

jibbajabba

Just reading this

http://www.networkworld.com/newsletters/edu/2008/070708ed1.html?page=1

One part is a bit donky :

The difference now is Microsoft will launch enforcement actions based solely on statistical findings.

So meaning : If I studied like crazy, finish the exam in 30 minutes and pass with 1000/1000 I probably cheated and get banned for life because statisticly I cheated ?

darkerosxx

I think there are a lot more people out there finishing in 30mins with 1000/1000 than people think. Microsoft's tests are really easy.

I'm betting on them targetting the people finishing faster than humanly possible to read, think, click. Otherwise known as the statistical cheaters, because they will be the outer edge of the bell curve. Or really, they're probably just hitting up a two-valued Z test and picking the people that stand out.

Also, you have to remember everyone is doing the cheater "shuffle" right now. In my personal opinion, I think it's just a bunch of noise to make them look good and to put it out there that cheaters don't get certs given to them to deter devaluing of their cert tracks.

JDMurray

darkerosxx wrote:

Also, you have to remember everyone is doing the cheater "shuffle" right now. In my personal opinion, I think it's just a bunch of noise to make them look good and to put it out there that cheaters don't get certs given to them to deter devaluing of their cert tracks.

Don't discount the likelihood that cert vendors are actually trying to find better ways to detect and deter cheaters and not just resorting to smoke-and-mirrors to maintain the integrity of their certs. I think using testing heuristics along with an appeals process can be quite effective. If a cert is revoked based on the possibility of the candidate cheating, a cheater isn't likely to fight for an appeal. The cert vendor just needs to make sure that the criteria they use to detect cheating is moderately accurate.

darkerosxx

I think you're giving them too much credit, so we can probably agree to disagree.

In my experience, certs are implemented to increase profit, not as a public service.

jibbajabba

I mean thats the thing .. I passed my 70-642 today with 984 points - which means I probably have 1 or 2 questions wrong and was in and out within 25-30 minutes. For some questions I didn't need to think or read ...

Its like you see as answers

a) 2
b) 3
c) 5
d) 8

Without even reading the whole question I am sure they ask for prime numbers and know that d) isn't one ..

You are in and out in 10 seconds ... you may speed read whether its really about prime numbers and not the number of people sitting in a 8am coach from A to B .. but thats it

I am SO happy they finally start to implement performance based exams .. Another way would be obviously not to provide the exam takers with a specific block of questions, but have maybe 1000 questions and choose a question randomly rather than having them in a testing center where they upload the results once a day to Microsoft.

JDMurray

darkerosxx wrote:

In my experience, certs are implemented to increase profit, not as a public service.

Of course. Certifications are a profit center and a marketing device. If they didn't bring revenue into a business (either directly or indirectly) they wouldn't exist. But this isn't the point. Our disagreement is if cert vendors are really doing anything useful to ensure the integrity of their certifications.

jibbajabba

Shocking how they (M$$) complain that the creation of an exam cost $150.000

There are 40.000 MCTS for 2008 alone .. multiply that by the cost of an exam and cry

snadam

JDMurray wrote:

darkerosxx wrote:

In my experience, certs are implemented to increase profit, not as a public service.

Of course. Certifications are a profit center and a marketing device. If they didn't bring revenue into a business (either directly or indirectly) they wouldn't exist. But this isn't the point. Our disagreement is if cert vendors are really doing anything useful to ensure the integrity of their certifications.

+1

it is in every cert vendor's best interest that the integrity of their Certs hold up; or else people would stop pursing them and thus generate low revenue. Plus the fact that your line of certs are the joke of the industry, that usually doesn't help their image either.

JDMurray

Gomjaba wrote:

.. multiply that by the cost of an exam and cry

Microsoft's stockholders aren't crying. That's the point of being in business--increasing shareholder value.

jibbajabba

Did they actually say when they finally start with checking the results ?

astorrs

Gomjaba wrote:

Did they actually say when they finally start with checking the results ?

They have been "checking" for a little under a year, they are just going to start enforcing now.

earweed

The original link to the recorded meeting doesn't work anymore.moderators

mikej412

earweed wrote: »

The original link to the recorded meeting doesn't work anymore.moderators

That happens in 2 year old threads.

Let sleeping threads lie -- and maybe Google to see if there is an updated version somewhere.