Options
Any way to get the EWS throttling policy from Exchange Online?
msteinhilber
Member Posts: 1,480 ■■■■■■■■□□
in Off-Topic
We're finally getting away from M-Daemon as our e-mail server, decent product but hasn't been working out all that well for ~1800 users. It took us at least a year of convincing but we finally got the go ahead to switch to hosted Exchange through Microsoft. This will surely be a blessing for our staff who will get plenty of new features and better reliability and for us back in IT since we're a two man shop and will be spending a lot less time administering e-mail.
The salesperson for the 3rd party handling the migration initially pitched doing an IMAP migration as well as contacts (they knew they were in a proprietary format at that time) for 18k. Turns out the salesperson overpromised and they'll only do email for 18k, contacts would cost almost double. My employer didn't care to invest more into the migration so it fell into my lap. Using a mix of AutoIt, VBA and Excel I was able to automate pulling the contacts, calendar and task data from M-Daemon and converting it to a CSV. PowerShell seemed to be the only real way to handle automating the import, relying on EWS and impersonation. So I used a mix of scripts available online and some of my own code and was able to cobble together something that would scan a text file of e-mail addresses and import contacts, calendar then tasks from each of that users respective CSV's. I was able to make it work, but not without potential issues when I use actual data instead of the smaller test CSV's I made.
The problem:
While the script was able to import contacts, calendar and tasks for the 25 users - I received micro delay applied warnings. I somewhat expected this being a possibility as I came across posts and writeups about throttling with EWS. From what I understand about throttling, if I bump into delays the requests will be queued and remain there until either we're under budget or until they remain in the queue for 60 seconds at which point they're tossed and an error is sent. I sought out what the throttling policy was hoping I would be able to better understand where I should consider using Start-Sleep in my scripts and for how long rather than just blindly guessing.
Unfortunately I can't find anything firm about how throttling is setup for Exchange Online/Office 365 and the Get-ThrottlingPolicy cmdlet is unavailable to me as a customer. I reached out to Microsoft who couldn't tell me and their response was just let my script run and it will all be fine, might just run slower when it's queued. Clearly I can't just do that since if I'm being queued with just 25 accounts using a smaller set of test data I'll almost certainly end up with requests being queued for more than 60 seconds with my production data and ~1800 accounts. Is there anything out there I can use to help better gauge where to make my scripts play better or am I just going to have to go about this blindly? I wouldn't be as concerned about it if the requests didn't get tossed after 60 seconds and I'm probably too much of a novice with PowerShell (had practically zero PS experience prior to this) to rely much on any form of error detection & recovery so Start-Sleep along with maybe splitting it into smaller batches of users at a time are about the only ideas I can come up with.
Any ideas? I've enjoyed this project so far, I keep thinking about how in what has probably been a day and a half of work so far I'm almost all the way there with a solution that gets us calendar and task data in addition to contacts. Now I need to convince my employer to throw a little more my way for getting better results than the 15k or so would have cost them. Either that or I need to reach out to the 3rd party doing the migration and see if they're hiring
The salesperson for the 3rd party handling the migration initially pitched doing an IMAP migration as well as contacts (they knew they were in a proprietary format at that time) for 18k. Turns out the salesperson overpromised and they'll only do email for 18k, contacts would cost almost double. My employer didn't care to invest more into the migration so it fell into my lap. Using a mix of AutoIt, VBA and Excel I was able to automate pulling the contacts, calendar and task data from M-Daemon and converting it to a CSV. PowerShell seemed to be the only real way to handle automating the import, relying on EWS and impersonation. So I used a mix of scripts available online and some of my own code and was able to cobble together something that would scan a text file of e-mail addresses and import contacts, calendar then tasks from each of that users respective CSV's. I was able to make it work, but not without potential issues when I use actual data instead of the smaller test CSV's I made.
The problem:
While the script was able to import contacts, calendar and tasks for the 25 users - I received micro delay applied warnings. I somewhat expected this being a possibility as I came across posts and writeups about throttling with EWS. From what I understand about throttling, if I bump into delays the requests will be queued and remain there until either we're under budget or until they remain in the queue for 60 seconds at which point they're tossed and an error is sent. I sought out what the throttling policy was hoping I would be able to better understand where I should consider using Start-Sleep in my scripts and for how long rather than just blindly guessing.
Unfortunately I can't find anything firm about how throttling is setup for Exchange Online/Office 365 and the Get-ThrottlingPolicy cmdlet is unavailable to me as a customer. I reached out to Microsoft who couldn't tell me and their response was just let my script run and it will all be fine, might just run slower when it's queued. Clearly I can't just do that since if I'm being queued with just 25 accounts using a smaller set of test data I'll almost certainly end up with requests being queued for more than 60 seconds with my production data and ~1800 accounts. Is there anything out there I can use to help better gauge where to make my scripts play better or am I just going to have to go about this blindly? I wouldn't be as concerned about it if the requests didn't get tossed after 60 seconds and I'm probably too much of a novice with PowerShell (had practically zero PS experience prior to this) to rely much on any form of error detection & recovery so Start-Sleep along with maybe splitting it into smaller batches of users at a time are about the only ideas I can come up with.
Any ideas? I've enjoyed this project so far, I keep thinking about how in what has probably been a day and a half of work so far I'm almost all the way there with a solution that gets us calendar and task data in addition to contacts. Now I need to convince my employer to throw a little more my way for getting better results than the 15k or so would have cost them. Either that or I need to reach out to the 3rd party doing the migration and see if they're hiring