More Changes to Microsoft Exams

LCA

An article on the MS exam blog called Investigating the Psychometric Performance of Our Item Types talks about the data from their exams MS have been studying and how the types of questions etc going to change.

Here is a quote of a small part of the article:

What did I learn?

• In general, non-MC items tend to be more difficult than MC items but not dramatically so, and they are equally differentiating (see my note above about the effectiveness of multiple choice items).
• Build list and drag and drop items tend to be more difficult for developer audiences than for ITPro audiences
• The most difficult items are: Labs, simulations, and build lists (but not too difficult)
• Best differentiators: Simulations, choose all that apply (really! I double and triple checked this because this was a psychometric surprise)
• Exams with these item types are perceived as more real world: Active screens, choose all that apply, simulations
• Exams with these item types are perceived as more rigorous: Active screens, choose all that apply, drag and drop, hot area, simulations

So, what does this mean to you?
Based on this analysis, you will see a larger variety of item types on our exams--more drag and drop, more active screens, more build lists, etc.

Any comments?

za3bour

I think simulations/labs are the most difficult but they're as well the best to protect the integrity of the exams and they've an excellent way to differentiate a good student from a cheater. I never really liked the "chose all that apply" kinda of questions.

Only problem I faced with labs is that they were not explained in a good way and if English is not your native language thats a problem too.

undomiel

I've always loved finding labs and simulations in the exams. To me it has always seemed like easy points. If I don't remember all the steps to complete something I can generally puzzle it out through clicking on the menus, while if it is a flat out multiple choice question then it is much more difficult to puzzle out what I'm forgetting.

MickQ

I prefer the simulation, too. Just as long as it's realistic enough to react to what I click/type.
Much like other threads about interview questions, and how your mind can sometimes go for a walk and leave you without the answer, the simulations will tend to refresh your memory of your sample labs or real life work.

it_consultant

I always have trouble with the all or none of the above. Keeps you honest.

mikedisd2

undomiel wrote: »

I've always loved finding labs and simulations in the exams. To me it has always seemed like easy points. If I don't remember all the steps to complete something I can generally puzzle it out through clicking on the menus, while if it is a flat out multiple choice question then it is much more difficult to puzzle out what I'm forgetting.

Totally agree with this; MS sims are just free points. If I don't know, just click around to find out. Non-relevant items are removed from the sim so that helps with deductions. Then when figured out, reset and do the task in one go. Easy.

Cisco sims are something else though...

it_consultant

The last Cisco sim I did was for the CCNA security and it was about as easy as a microsoft sim. When I took the CCNA originally, I had limited exposure and that was my way of getting networking knowledge. By the time I did CCNA security I had set up hundreds of firewalls from sonicwall, cisco, watchguard, to juniper. One of the sims was to set up a vpn to another firewall, and another asked details about an already existing vpn link. To someone who was new to firewalls this would have been difficult.

Most people that take MS exams have been working on MS systems for some time, so the sims seem much easier. They could make it really hard by making a sim something like "change the ACL of that folder with powershell", but really, how practical is that?

Priston

Sims can be really easy, but they can also be time consuming, so just as long as they factor that in I'm all for it.

alokin123

whatever they do if they are going to have sims and a variety of question type i hope they have fixed the bugs that cause these to crash the exam. Whether this is a prometric issue or MS it needs to be fixed. When i did the 83-640 my lab crashed and couldnt be restored so i had to come in another day. It is a bit disheartening when you have primed yourself to take an exam.

Darril

While I see some value in choose-all-that-apply multiple choice questions, I am a little surprised to see how they are being implemented.

This article (In some cases you can have too many options! - IT Career JumpStart) talks about how some choose-all-that-apply multiple choice questions have as many as 16 options. Presumably, if you miss one, you miss the question. And screen shots requiring you to pick from as many as 16 tabs. These types of questions check your knowledge of the tedious details of a topic, rather than what is needed to perform the job.

I'm also wondering about newer simulation questions. Microsoft has done a great job at making it easier to use the operating system and you can usually access tools using multiple methods. For example, in Windows 7, you can type in the partial name of a tool in the "Search programs and text files" text box after clicking start to launch the program. Similarly, from Control Panel, you can type in the name of a tool in the "Search Control Panel" text box. However, it's very difficult to program a simulation to allow the test taker to access tools using all of these methods. Instead, the simulation only allows the test taker to do it one way, and if they don't do it that way, it's incorrect.

Darril Gibson

HTH