Any DevOps professionals on TE?

RobertKaucher

Anyone on in TE involved in DevOps? Just to be clear I mean the some-what loosely defined set of methodologies that generally include the following attributes:

• Use of some sort of Agile methodology like Scrums
• Continuous delivery/deployment (I.E. increasing the rate of production releases from application and business unit stakeholders)
• Wide availability of (private?) cloud infrastructure
• Producing or customizing data center automation and configuration management tools

Not a generic definition of an IT pro who can do PowerShell or Python/Bash scripts to automate common tasks.

I am mostly curious about how the shifts in mindset required for DevOps has changed your company's cultures, what you found most challenging on a personal level, and what your team found most challenging. Any replies would be heartily welcome.

Mishra

I've been learning a lot more about DevOps... We don't apply it where I work but I have 2 people I know, one of which is in a new position, that explain it to me.

DevOps seems to definitely be the way the bigger internet companies are working now-a-days. Also, it takes a hoard of young people to apply it properly. I definitely see a huge culture difference between companies that apply it. A friend is finding it "challenging" but only because he has a lot of things to learn about how the company operates... As a system engineer, so many of the operational tasks are removed off of your plate so it frees up a lot of your time to work on more architectural stuff. For example, engineers are now focused on keeping the networks and infrastructure services running. Although monitoring and other services have also been swallowed into devops land.

For the most part, he was hired to introduce other services that the company wants to apply but hasn't been able to in the short time frame they have... This company deals with 3.5 billion hits a day. All of their infrastructure is bare metal and no virtualization is happening. Security is a concern and better security needs to be applied. He seems happy.

paul78

I was thinking about attending a DevOps Days event to find out more about pitfalls and successes - Upcoming devopsdays events

I have been exploring DevOps based on these priorities:

1. Minimize the bottlenecks between development and IT, thus decreasing the time to market.
2. Increasing operational quality and reducing service downtime.
3. Decreasing expense through more efficient use of private clouds.

I am very interested in the concept of DevOps but I perceive that I will have a large cultural and organizational barrier to overcome. One of the biggest barriers would be the IT teams which have a narrow concept of DevOps. A lot of IT teams seem to believe that DevOps is simply the combination of networks/systems engineering with IT operations followed by rigid monitoring and automation.

As you pointed out – DevOps is a more holistic approach. The benefit that I am suspecting that I can derive from a DevOps approach is from combining software engineering, application release/deployment, IT engineering, and application support into a single cross-functional team. The challenge that I fear is in the ability to scale such an approach. In an organization with multiple businesses and hundreds of products and services, it may be more difficult to accomplish in a consistent manner.

My interest in the DevOps approach is primarily driven by the shift in software development from a waterfall method to an iterative method. Shorter time-boxed scrums and a need to get to market faster are being constrained by engineering and operational processes.

My fear with introducing DevOps is that we could end-up with too much of a free-for-all.

I, too, would be interested in learning how DevOps can work in a large enterprise environment in a cost-effective way.

petedude

paul78 wrote: »

I
My fear with introducing DevOps is that we could end-up with too much of a free-for-all.

I'm not as much concerned about a free-for-all as I am about a couple other issues:
1. IT becoming increasingly driven by a marketing-centric approach as opposed to a balanced governance approach that considers overall sustainability including compliance, security and stability.
2. Operational priorities for other IT functions being driven by development staff who are not expert in the outside areas.
3. Quality being sacrificed for release speed and customers having to track a frequent number of releases. I seem to remember a similar trend in the early 90s and early 2000s when the "majors" seemingly delivered new software releases at a whirlwind pace.

ccnxjr

my $0.02

DevOps as I understand it is mostly SysAdmin focused around programmers/developers.

Effectively, your "users" basket of requirements include more than just e-mail, storage, print and office applications.
Clearly more popular in smaller organizations, or organizations who do use some form of Agile methodology to software development.

I'd say one of the more challenging aspects is simply standing your ground as a SysAdmin without being an (for lack of a better way of putting it) outright jerk.
As a member of DevOps your world is of resource allocation and monitoring has expanded to include developer tools.
Which will include access to production systems, and in many cases , release management, (think CVS systems such as "Git") .
Or continuous integration systems , such as "Jenkins".
But these are programmer's tools?
Well, as DevOps, you hold the keys to them.
Not to mention, these are highly configurable tools. so it's your job to enforce, or lay down some sort of standardization guidelines.
And that's what they are, guidelines.
You may have cause to revise them from time to time, learning when a revision is needed and when to stick to them is one of those things you learn from experience.

In addition to learning how to manage these new resources, you are also vitally important to system security and making sure that new software has the right access and won't run amoc and interfere with other systems.

Expect

I am starting my DevOps Engineer role in 2 weeks working for F5 Networks, I will have alot more to contribute to the topic then

paul78

Good points @petedude. You have raised some very legitimate concerns.

petedude wrote: »

IT becoming increasingly driven by a marketing-centric approach as opposed to a balanced governance approach that considers overall sustainability including compliance, security and stability.

It's interesting that you said that. To me - that's exactly what I want - to be driven by a market-centric approach. Both IT and development priorities should be driven by the market. To me, it's business that drives IT (which I include development) - not the other way around. Any IT governance approach must be inline with business priorities. And the risk appetite of the business is what sets that governance. My comment about free-for-all is really about how to assure that if multiple DevOps groups are created because of the size of an organization or business, how do I control consistency across the DevOps teams.

petedude wrote: »

Operational priorities for other IT functions being driven by development staff who are not expert in the outside areas.

Very true and the opposite is also true. Too often, I've seen IT functions which do not understand development and business requirements. That's really where I perceive that DevOps can pay dividends - to break down that type of compartmentalization.

petedude wrote:

Quality being sacrificed for release speed and customers having to track a frequent number of releases.

I think that you hit one of the problems. I do think that there is likely a middle-ground. With my particular problem-set, I am planning to resolve it by through the use of a mirrored production environment - essentially a user acceptance test environment where customers can review and accept the changes as deployments occur. And releases into production environments would be scheduled at a more reasonable pace - such as quarterly or annually.

ccnxjr wrote:

I'd say one of the more challenging aspects is simply standing your ground as a SysAdmin...

I respectfully disagree. The reason why I like the DevOps approach is that it brings the various disciplines together to bring a technology solution to market in a way that complements software development Agile methods. A sysadmin or netadmin that simply says "no" is not someone that is part of the team. In my fairyland world, I would expect a sysadmin or netadmin to find a "cost-effective" compromise that aligns with the business need.

@Expect - Congratulations on your new job. Look forward to hearing more about your experience.

RobertKaucher

petedude wrote: »

I'm not as much concerned about a free-for-all as I am about a couple other issues:
1. IT becoming increasingly driven by a marketing-centric approach as opposed to a balanced governance approach that considers overall sustainability including compliance, security and stability.

paul78 wrote: »

It's interesting that you said that. To me - that's exactly what I want - to be driven by a market-centric approach. Both IT and development priorities should be driven by the market. To me, it's business that drives IT (which I include development) - not the other way around. Any IT governance approach must be inline with business priorities. And the risk appetite of the business is what sets that governance. My comment about free-for-all is really about how to assure that if multiple DevOps groups are created because of the size of an organization or business, how do I control consistency across the DevOps teams.

I think this really should be dictated by the business. There are plenty of cases where "market" has no significance at all but things like compliance, security, etc might all be far more important for the business. But I suspect you two are really saying the same things but from very different angles. Ultimately it must be the business that sets the priorities. For some businesses, like Netflix, 10 deployments per day might be perfectly in line witht he business's goals, but for a government site being developed for the V.A. that would be irrational.

petedude wrote: »

2. Operational priorities for other IT functions being driven by development staff who are not expert in the outside areas.

This is actually the very point of DevOps, though. Developers are not experts in the operational domains, they do not generally have the skills, and even those devs who come from an operational background (like my self) they don't have the time. It should not be my job to administer the SQL Servers, to be adding users and setting permissions on the SharePoint site or to be deploying the hypervisors and patching the DNS server. All of those things that I can do and do on occasion but if I am doing that full time I am not adding features to the applications our users need. But if the ops and developers don't work together then the people engineering the solutions begin to think in overly simplistic and abstract terms about the network and they will architect for those over-simplifications. But worse than that, once the developers begin to embrace methodologies like Agile they will become more productive and the bottle neck will be shifted to the IT pros and network teams. A system like DevOps that values Lean/Agile methodologies and demands quality/testing procedures and transparency in development and deployment is the only hope to keep up.

petedude wrote: »

3. Quality being sacrificed for release speed and customers having to track a frequent number of releases. I seem to remember a similar trend in the early 90s and early 2000s when the "majors" seemingly delivered new software releases at a whirlwind pace.

The number and speed of releases has nothing to do with DevOps. As I said these things are dictated by the business. Your customers, internal or external, should be the ones who decide what changes and how quickly. From an enterprise line-of-business development perspective my customers (the people who are doing the real work in the company) don't care if we are going from 1.2 to 1.3 or whatever. They only care about the features. And by using Agile and rapid deployment methodologies that make up the core of DevOps we can give them those features faster. And if at 11:00 A.M. we roll out a new release and a bug is reported at 1:00 P.M. we can literally get the bug patched by 2:00 P.M. and NOT by editing the JavaScript or HTML on the server's file system. But by following our processes and going from my development laptop, to being committed in SVN, pushed to test where the unit tests pass, and then pushed to production with no service interruption.

At the very least it's worth the trouble to my users that they might actually be seeing more bugs and issues in production environments provided those bugs can be fixed quickly and they get the features that they want. What most users don't want is for developers to f-up something that they need to do their jobs and then have to wait for weeks for the fix to be deployed only to find out that since the developers were working on the bug the new features didn't get built and will be delayed. That kind of stuff can only be done when there is a high degree of automation and developers and operations work closely.

When you have the opportunity for business users to work closely with developers and IT to build the solutions that the users really want users will be more tolerant of minor issues because they trust that IT (dev and ops) will get it fixed and provide them with better and smarter ways for them to do their jobs. And then developers and IT are graded on the same scale: how many new features are added and how satisfied are the users. New features inherently introduce instability into any system. If developers are graded on the number of new features added but IT is graded on up-time then there is a bright line between the the priorities of the two groups within the organization and having two teams in the same company that rely on each other but have opposing priorities is bad.

paul78

Very well said Robert. The bottleneck in IT are precisely the reason why I am interested in DevOps. The other opportunity that I perceive is a potential reduction in architectural issue because either infrastructure was an after-thought or lack of understanding of operational nuances degrade quality. One aspect that I am unsure is whether a DevOps organization can be shaped to provide actual operational support as a means to reduce time to repair. I would be interested in anyone's thoughts on that dimension.

RobertKaucher

I guess it really just comes down tot he fact that continuous deployment/delivery are not models applicable to every industry or situation.

paul78 wrote: »

One aspect that I am unsure is whether a DevOps organization can be shaped to provide actual operational support as a means to reduce time to repair. I would be interested in anyone's thoughts on that dimension.

paul78, I find that topic interesting as well.

I think that might only occur as a side effect. If your team is truly running lean, then it might have more time to respond quickly to repair situations. And if DevOps culture reduces burnout, then they might be better able to fix it quickly from a mental/emotional stand point. But I guess if it relies on a piece of hardware being shipped, then moving atoms becomes the bottleneck. You can script/automate anything that has to do with bits, but moving matter is not so easy.

ccnxjr

Wait a minute,

Haven't gone through everything, but i work for a small company, heavy on the tech side (13/35 employees are engineers, about 8/35 customer support) and usually business needs get filtered through a PM .
Code is extensively tested, before getting pushed out to production, I can't remember the last time we've had to push a hotfix, let along within an hour of pushing to production.
A rapid dev cycle should be no substitute for testing.
If there is a suspect production bug our CS team leader is usually good at aggregating data and he contacts our CTO directly, who then double checks to be sure it's a bug in production code rather than a alternative use case.
A borderline bug/new use case is something that in that's more of a managerial decision on how they wish to allocate resources and adjust priorities.
Not sure how that would translate in a larger org.

phoeneous

RobertKaucher wrote: »

I guess it really just comes down tot he fact that continuous deployment/delivery are not models applicable to every industry or situation.

Yeah, my environment definitely doesn't follow a continuous deployment/delivery model. But that could very well be because our erp is on 3rd party lockdown mode and aside from ms office, it's really the only application that we use. Plus our company is less than 200 users.

ccnxjr

paul78 wrote: »

Good points @petedude. I respectfully disagree. The reason why I like the DevOps approach is that it brings the various disciplines together to bring a technology solution to market in a way that complements software development Agile methods. A sysadmin or netadmin that simply says "no" is not someone that is part of the team. In my fairyland world, I would expect a sysadmin or netadmin to find a "cost-effective" compromise that aligns with the business need. @Expect - Congratulations on your new job. Look forward to hearing more about your experience.

Not saying that DevOps should automatically say no, but they shouldn't automatically say "yes" either. Rather there should be some thought put into what is brought to bear and not a blind acceptance. I emphasize the "no" as there is strong culture of jr developers steam rolling jr dev ops, without a senior person (developer or ops) present to referee them. Occasionally a healthy debate can bring new information to bear, however, not all personalities can engage in healthy debate without taking it personally. For example, if a developer isn't able to reach a certain resource, some new ones may assume it's a network issue or may be quick to blame the network if there were prior network issues instead of performing due diligence. Very general example. Other nuances might be (although rare) obscure bugs in an IDE where things test fine in the development environment and fail well when pushed to production, or simply a dirty development environment. If dev ops is consistently fixing system faults , then maybe it's time for the current dev ops team to step back and re-evaluate the system design.

Think for example a debate such as "My code doesn't work well because your environment is buggy" or "My environment is compromised because your code is buggy" .

paul78

@ccnxjr - sounds like we are in agreement and perhaps just looking at it from different perspectives. My job tends to be a lot of mediation between development, engineering, and operations teams. And I am not a member of any of those functions.

I actually view the DevOps team as a cross-section comprised of senior-level developers and architects and engineering architects and operations leads. I have this fantasy that quality may improve because the DevOps team has to eat their own dog food. And because they provide operational support – does that reduce time to repair.

You mentioned you worked for a smaller company. Do you have one product or many products?

That’s the part about DevOps that I’m also wondering about. I work in a pretty big organization with many solution sets. So I’m not quite sure how DevOps can work in this type of environment.

ccnxjr

We have several products, which were recently migrated to a single platform (our home brewed cloud) .

I'd say our development environment is where Dev-OPs spend 70% of our time, when code is ready for production a good roll out involves minimal dev-ops time.

Repair time would definitely be contingent on the problem, could be ops , could be code, could be mercury in retrograde?

Our platform itself is a product and occasionally we do have tickets that are platform specific.
This is where management decides whether it makes business sense to commit resources to :
-building/testing that specific use case,
-classifying it as a bug, reverting to a stable state and working on it or
-shelve it as a new feature to be worked on as time permits

Of course , if it were a simple fix that was overlooked in the testing phase, it gets added to our QC process.

In a round about way, Dev-Ops doesn't own a crisis nor do developers.
If there's a bug that needs fixing quick, the entire team looks at it.
Which is a far cry from many other organizations that point fingers first and expect a miracle right off the bat.
Dev-ops might be more efficient at identifying where the problem *may* lie (ie , code, harderware, OS, integration) as they tend to be mulch-disciplinarians (and generally have root access), and interact with other parts of the business.
However a key component is that it's a team effort, ie, the developer is in the same room (or general area , close enough where you don't have to shout, and your normal conversational tone can be heard by other team members).

Like this:
Dev-ops receives a call from support:
Support: we're seeing a large number of calls this morning about time outs, is something going on?
Dev-ops: We pushed a patch out last night, is the problem reproducable?
Developer lifts his head up from the laptop and looks at Dev-ops intently, paying attention
Dev-ops: send me the details and I'll test it.
Support: just sent an e-mail
Dev-ops starts testing
Developer: so what are the symptoms?

Teamwork....


Suffice it to say, no one at our company is running around like chickens with their heads cut off.

paul78

ccnxjr wrote: »

However a key component is that it's a team effort, ie, the developer is in the same room (or general area , close enough where you don't have to shout, and your normal conversational tone can be heard by other team members).

That's an interesting comment. I would be curious if you or anyone else felt that physical proximity is a requirement for successful DevOps culture. I'm constraint by the fact that people are all over the world in my environment.

ccnxjr

I will take the easy way out and say "thats how things work at our shop" :P

Seriously though, we do have some people that work offsite, the ability for people to have open lines of communication, and actively communicate helps a LOT.
I usually sign in to a Google hangouts environment (we previously had an irc channel ) first thing in the morning, then we have a teleworker who logs in via video phone, and works like that , he's got his own desk and everything

It might work differently in different environments.
This is my first gig with a DevOps team, so I'm not sure how it works in other shops.

paul78

Thanks for sharing your experience. It's very helpful.

ptilsen

I've been very interested in it for a while and read different blogs about it every once in a while. Just from a day-to-day standpoint, it seems like the type of work I'd like to be doing more than more traditional systems engineering. However, I have a lot of trouble seeing how I'd get there or even how the concept applies to my existing job. This being the main issue:

RobertKaucher wrote:

I guess it really just comes down to the fact that continuous deployment/delivery are not models applicable to every industry or situation.

This aspect is so far removed from what I do day to day and have done for my career that when I read about Devops I actually wonder what the hell they're talking about half the time.

That, and that it seems to be mostly a Nix world. I did find this and it's a lot more relatable, but I've found maybe five or six jobs in the country (mind you, I just look periodically when bored) that were hiring Windows devops engineers, and even most of those were titled differently, just clearly were devops-oriented roles or used the term in the job posting. I'm not afraid of Linux, but I have only basic working knowledge of it compared to deep Windows knowledge.

ccnxjr and Expect, what kind of experience and skills did you have that got you into devops roles? ccnxjr, what does your daily life look like? Most importantly, do you enjoy it?

paul78

ptilsen wrote: »

...that it seems to be mostly a Nix world.

That's another interesting perspective that I had not considered. Is there a reason why you believe that DevOps is more well-suited towards non-Windows environments?

I was looking at DevOps more as a complement to organizations that utilize Agile development practices - regardless of choice of operating system environment. Are there inherent limitations with DevOps organizations because of Windows?

ptilsen

Every job posting I see wants heavy *nix and languages mostly used in the open source world. Most don't even mention Windows. Other than the very few Windows-targeted ones I mentioned finding, there's nothing on MS products or Windows components. Most of the blogs I've seen are all about Linux environments. I don't know if there's some limitation or trait common in MS shops. I just know that almost everything I see devops related is 100% or nearly 100% about open source products.

russcollier

ptilsen wrote: »

I don't know if there's some limitation or trait common in MS shops.

(full disclaimer, I'm one of the authors of the DevOps on Windows blog mentioned earlier, so I'm a bit biased )

I don't think there's any limitation of the MS toolset that's holding back DevOps on the Microsoft platform. I think there's just a lack of understanding of the available toolset. Unfortunately as most people know, historically the barrier to entry for hacking/tinkering on the Microsoft platform has been high because (if you want to go the legal route, which we all do ) you have to pay a lot more upfront to gain access to some of the more advanced tools, environments, etc.

But I do agree, for now, DevOps seems to be much more concentrated in the Unix world. That's something my friend and I are hoping to change with our site We feel that DevOps is less about a particular toolset or platform and more about an attitude and philosophy (software should be simple to operate, don't create a "DevOps" team - enable ways for ops and dev to work together with consistent processes, etc. etc.). The platform shouldn't matter in the end as long as you apply the right philosophies.

That being said (again, I'm biased), I'm trying to help spread the word about some of the options folks have (in both ops and dev) for doing automation, sysadmin type tasks, and general ops oriented development on the Microsoft platform.

(and while I can't say for certain what the job market is like for DevOps exactly, there are several of us Windows "DevOpelers" tucked away in the Chicago financial IT sector )

Expect

ptilsen wrote: »

I've been very interested in it for a while and read different blogs about it every once in a while. Just from a day-to-day standpoint, it seems like the type of work I'd like to be doing more than more traditional systems engineering. However, I have a lot of trouble seeing how I'd get there or even how the concept applies to my existing job. This being the main issue: This aspect is so far removed from what I do day to day and have done for my career that when I read about Devops I actually wonder what the hell they're talking about half the time.

That, and that it seems to be mostly a Nix world. I did find this and it's a lot more relatable, but I've found maybe five or six jobs in the country (mind you, I just look periodically when bored) that were hiring Windows devops engineers, and even most of those were titled differently, just clearly were devops-oriented roles or used the term in the job posting. I'm not afraid of Linux, but I have only basic working knowledge of it compared to deep Windows knowledge.

ccnxjr and Expect, what kind of experience and skills did you have that got you into devops roles? ccnxjr, what does your daily life look like? Most importantly, do you enjoy it?

I personally started with Windows system administration, I do not have years of experience (about 4 in total) but I am a good nixer, I know coding in multiple languages(py,pl,bash) required for the devops field, I guess I did well enough at what they(F5) expected from me which brought me to this point.

ccnxjr

ptilsen wrote: »

ccnxjr and Expect, what kind of experience and skills did you have that got you into devops roles? ccnxjr, what does your daily life look like? Most importantly, do you enjoy it?

I was initially looking for a job with more network/datacenter work, even with a Bachelor's, CCNA and 3+ years on the helpdesk it was still very challenging.
However as unemployment funds were used up I became less picky and applied for a position as a LAN admin for a small office.
Once things got stable I volunteered myself in new roles and slowly worked myself into the culture.
When I first started I just got my CCNA and toyed with some VB and powershell at my last job, only to make my work easier.
I did take a class on Linux and general scripting in college, I didn't know much about system automation or open source projects quite yet.
However I was given a few projects, and graciously, the opportunity and time to learn .
There were quite a few late evenings and some weekends I just had to buckle down and hack away at stuff without a solid base, but projects eventually got completed satisfactorily.

You could say I lucked out finding a position with people who value the ability to learn new things and get along well with others over credentials and trophies.

ptilsen

russcollier wrote: »

(full disclaimer, I'm one of the authors of the DevOps on Windows blog mentioned earlier, so I'm a bit biased )

Wow, Russ, thanks for joining and posting, and props on investigating from where you're being linked. I've enjoyed quite a few of your articles (and your co-authors') over the last few months.

russcollier wrote: »

I don't think there's any limitation of the MS toolset that's holding back DevOps on the Microsoft platform. I think there's just a lack of understanding of the available toolset. Unfortunately as most people know, historically the barrier to entry for hacking/tinkering on the Microsoft platform has been high because (if you want to go the legal route, which we all do ) you have to pay a lot more upfront to gain access to some of the more advanced tools, environments, etc.

Absolutely. There are also the tremendous advancements made with WMI and PowerShell over the last several years, which I think open the doors for a paradigm shift in Microsoft shops. Let's be honest, though; a lot of the people who have been able to succeed as Windows admins are able to because MS made GUI tools so effective. That puts many of our colleague's mindsets as the real barrier.

russcollier wrote: »

But I do agree, for now, DevOps seems to be much more concentrated in the Unix world. That's something my friend and I are hoping to change with our site We feel that DevOps is less about a particular toolset or platform and more about an attitude and philosophy (software should be simple to operate, don't create a "DevOps" team - enable ways for ops and dev to work together with consistent processes, etc. etc.). The platform shouldn't matter in the end as long as you apply the right philosophies.

Absolutely, which, granted, is why I'd even consider a change to open-source just to be in the right team with the right mindset. At this point it's more about the mindset and processes in use than the technology itself. I see the same mistakes every day, and conceptually they apply to any technology.

russcollier wrote: »

That being said (again, I'm biased), I'm trying to help spread the word about some of the options folks have (in both ops and dev) for doing automation, sysadmin type tasks, and general ops oriented development on the Microsoft platform.

It is well-appreciated. Keep fighting the good fight.

ptilsen

ccnxjr wrote: »

I did take a class on Linux and general scripting in college, I didn't know much about system automation or open source projects quite yet.
However I was given a few projects, and graciously, the opportunity and time to learn .
There were quite a few late evenings and some weekends I just had to buckle down and hack away at stuff without a solid base, but projects eventually got completed satisfactorily.

You could say I lucked out finding a position with people who value the ability to learn new things and get along well with others over credentials and trophies.

It sounds like you found a great opportunity. The one big flaw I'll say I've found is that most devops shops seem to be looking for expertise in all the stuff (products, software) they're already using, which in my opinion isn't very devopsy, if you will, but it is what it is. Of course, Windows shops are even worse. I still laugh whenever I see postings or talk to recruiters that require knowledge of a specific hypervisor or SAN product or even server OS version.

paul78

Great discussion so far. I couldn't help noticing that most comments take the perspective that DevOps is an IT operations function where IT operations staff include individuals with the software development skills to perform automation and deployment. I was wondering if anyone has had experience with the line of thought that Robert originally posed - that DevOps is more of a Development led function where development staff had the operations skills to assure that software is developed with operations in mind.

petedude

Had to post a tidbit from Bob Lewis that made me think of this thread. . . slightly off-topic, but also more than slightly related:

"Anyway, Galen’s piece is one of many that are appearing these days that predict we’ve entered the end-times for the CIO, and probably for corporate IT as well. Read any of these pieces. Squint at them sideways and you can predict the outcome of following this advice: A proliferation of “islands of automation,” because when companies push IT into business departments, nobody will be willing to pay for integration, let alone have the skills to handle it, let alone the authority."

At Home - Welcome to IS Survivor Publishing this week.

That last sentence speaks volumes-- exactly what concerns me about the new decentralization trends: a lack of integration and a lack of authority. DevOps ties into this conversation, because even though it blends development into operational IT, it could pull for many IT activities away from traditional infrastructure teams and (potentially) subjects them to the whims of less-informed staff inside or outside IT.

And. . . we've been through all this before. I/T is SO cyclical. . . PCs decentralized data processing away from the mainframe teams and now new trends (e.g. DevOps, BYOD) are pulling access and control from central I/T again. Ha, first it was monster mainframes and now monster VM servers. . .and so it goes.

russcollier

paul78 wrote: »

Great discussion so far. I couldn't help noticing that most comments take the perspective that DevOps is an IT operations function where IT operations staff include individuals with the software development skills to perform automation and deployment. I was wondering if anyone has had experience with the line of thought that Robert originally posed - that DevOps is more of a Development led function where development staff had the operations skills to assure that software is developed with operations in mind.

Personally I've always been on the operations/service-delivery side of the fence the majority of my career so I'm a bit biased, but I think it requires awareness on both sides of the fence. I'm not sure if it necessarily means that development staff need operations skills per-se, but rather, developers need to appreciate and understand operational concerns and minimize operational complexity of their software.

Basically developers need to treat operational concerns and requests as importantly as a business/end-user level request. Because at the end of the day, the operations staff deploying and/or supporting the developers' applications are just another end-user of the software if you think about it. Developers need operations skills in the sense that they need to understand how their application is deployed and run in any environment (dev, QA, production, etc.). Deployment and support concerns (and the build system, and the configuration management, etc.) need to be considered at all stages of the development process/application life-cycle, not just at the end when it's time to chuck the app over the wall to operations.

In my experience some of the best developers I've worked with understood that minimizing operational complexity and listening to the concerns of the operations team are just as important as the features they're creating. Likewise, the best "operations engineers" I've worked with are those that understand enough about the development process to work with development teams - not just email them an error message and say "please look into it". Both teams work together to create a consistent process for things like application deployment, configuration management, etc. and everyone wins.

DevOps is not an ops led process or a dev lead process. It's a common philosophy/attitude shared by both teams and working towards the goal of creating software that is simple to operate - after all, both ops and dev are really just two sides of one coin working towards the same business goals.

Sorry, I think I'm starting to ramble now, but that's my 2c YMMV

paul78

Thanks for your viewpoint Russ. It is certainly interesting to read. With DevOps, my hope is that if developers are part of operational support model, there is less tendency to "chuck the app over the wall". If the developers have to eat their own dog food, perhaps more attention would be paid to operational concerns. Similarly, if infrastructure and operations engineers have a stake, my hope is that they would be less likely to impose artificial constraints via processes which may not be aligned with business goals.