In this episode of DevOps Unbound, hosts Alan Shimel and Mitch Ashley are joined by Adam Kalsey of Tricentis, Brian Dawson of the Linux Foundation and Caroline Wong of Cobalt to discuss why testing is considered the most frustrating bottleneck for developers and engineers. Panelists will explain the benefits of testing early on in the SDLC and how to accelerate the feedback cycle with automated and continuous testing, as well the pros and cons of testing, the value of shift-left testing and how to effectively balance quality and speed. The video is below, followed by a transcript.
DevOps Unbound is sponsored by Tricentis, the world wide leader in automated testing. DevOps Unbound airs biweekly on TechStrong TV, with all episodes available on Digital Anarchist. There is also a live monthly roundtable open to audience participation.
Alan Shimel: Hey, everyone. I’m Alan Shimel, and you’re watching DevOps Unbound. DevOps Unbound is sponsored by our good friends at Tricentis, and it consists of a biweekly, 40, 45 minute video panel on relevant topics within the DevOps sphere, and featuring a panel of experts that rotates every other show, and really great people, and something I personally really enjoy doing.
And then, about once a month or once every six weeks, we actually have a live studio audience—or there is no studio audiences in COVID, actually, but we have a live virtual audience that participates in our show. Today’s show is just our panel, though, without a live audience, and I’m really looking forward to our discussion.
Our discussion today is around testing. You know, with the pressure on developers to deliver code faster, sometimes testing is considered a road bump in the way. Testing is considered slowing down the whole process. There are a lot of developers who’d like to push testing off that plate. Of course, no one wants to go back to testing after we’ve deployed software, primarily, but there is that tension there.
Let me introduce you to our panel today; we’re gonna discuss this in depth. First of all, making his first appearance as a member of The Linux Foundation on our Tech Strong TV network is our old friend, Brian Dawson. Brian, welcome to DevOps Unbound.
Brian Dawson: Thank you, Alan. Thank you, Alan. And it’s fitting that the first panel, first webinar I do since joining The Linux Foundation is with you.
Shimel: It’s our pleasure to have you on. Brian, what’s your position at Linux Foundation now?
Dawson: So, at The Linux Foundation, I am functioning as the VP of Developer Relations and Ecosystem Development. So, just to help, we’re gonna help The Linux Foundation to the development community while driving more of a DevOps opinion into a growing ecosystem of solutions that support open source development.
Shimel: Fantastic. Next up is my good friend, Caroline Wong. Caroline is—oh, she’s an anchor on many of our different shows here at MediaOps and Tech Strong. Caroline, it’s great to see you again. A little background, maybe, for people who aren’t familiar with you?
Caroline Wong: Great to be here. Thank you so much. My name’s Caroline Wong, I’m the Chief Strategy Officer at Cobalt. We’re a pen test as a service company.
I actually started my DevSecOps career 16 years ago leading information security teams at eBay and Zynga. So, in both cases, we were running online operations 24/7 with millions of simultaneous users daily. And now, at Cobalt, we build security software. And, like many DevOps companies, we’ve got data driven, product based teams. We value automation and failing fast, and super excited or today’s conversation.
Shimel: Fantastic. Welcome. Great to have you, Caroline. Our third panel member is another returning DevOps Unbound veteran, and that’s Adam Kalsey of Tricentis. Adam, welcome.
Adam Kalsey: Thanks. Thanks for having me. I lead Developer Relations for Tricentis. I’ve been building web applications since 1995, and have always had a soft spot in my heart for bad applications and broken things. And so, I love to help developers build higher quality software and do it in a more efficient, faster way.
Shimel: Absolutely, and thanks for being on with us today, Adam. Last but not least, in his Florida outfit here, visiting me in Florida, is my Co-host of DevOps Unbound, Mitch Ashley. Mitchell, welcome.
Mitch Ashley: Coming to you from the MediaOps Studio B.
Shimel: Yes.
Ashley: Great to be here. I’m a former product, both security and software SaaS, CTO as well as a CIO, and now I run Accelerated Strategies Group, an analyst firm that focuses on those things.
Shimel: Absolutely. Hey, Mitch, thank you.
So, let’s get started. Adam, you’re with Tricentis, the sponsor of DevOps Unbound, and they’re a testing company, probably the leaders, one of the leaders worldwide in automated testing, continuous testing. Have we put too many straws on the developers’ camel’s back by making them, you know, be more responsible for testing to say we need to test more of our code earlier.
Is it a bridge too far for many developers? Is it making them resent testing? You know, what’s the current state?
Kalsey: I think where we’re failing developers is by handing them lots of more things to do without giving them any—without giving them what tools, and I don’t mean just necessarily software tools, but without giving them the things that they need to do it effectively. If we just pile things on developers and say, “Oh, this is your job now” but we don’t give them the support, we don’t give them the training, we don’t give them the expertise, we don’t give them anything to make that better—sure, it’s just gonna be harder on ‘em.
Shimel: To play devil’s advocate—oh, those poor little testers making a quarter of a million dollars a year each. Do we want ‘em to do testing, too? Right? Is there some of that here, Brian? Or—go ahead, Adam?
Kalsey: No, no, Brian—please, please, jump in.
Shimel: Well, he’s from The Linux Foundation, he now represents these folks. [Laughter]
Dawson: Well, first, not all of them make a quarter of a million dollars a year, right? But I do think what you raise is important when you bring up the investment, right? We are hiring talent with a particular expertise in a particular domain, a particular ability to solve problems and innovate.
And there is an argument to be said, Alan, that, “Do I want to spend this money, this quarter of a million dollars plus 200,000 in RSUs on having somebody write test harnesses?” Now, I think—you know, that can be balanced by what Adam said. Well, first of all, yes, because we want everybody thinking about quality. But you know what, if we want a return on that investment, we wanna free the cognitive load, then you know what? We need to invest in automation, we need to invest in better collaboration and reducing friction from the process.
Shimel: Agreed, agreed. Caroline, you know, it’s been a big—this whole DevSecOps movement and shift left has been about shifting security testing left and getting developers more involved in security testing, right? Yet another camel on the—yet another straw on the camel’s back. Should we be doing that?
Wong: So, I—oh, my Internet is awful. Can you hear me?
Shimel: Yes, we hear you fine.
Wong: Okay, fantastic, awesome. I’ll just—I know that there’s post editing, so it’s okay for me to say that.
Shimel: Mm-hmm.
Wong: So, Alan, I bring a particularly unique perspective to this group in the sense that I actually oversee people at our company, I oversee Human Resources. And I’ve been with the company since we were 10 people, and today we’re 200. And one of the super interesting things that happens is, naturally, as a company grows, what people do, they specialize and then we have to figure out how to communicate more effectively.
And I think that if you’ve got an engineering team and you say, “Okay, now you’re responsible for Development and you’re responsible for Ops, and your responsible for security, then there’s got to be some level, depending on the maturity and the size of the organization and what the organization’s trying to accomplish of specialized roles and protocols for engagement.”
And I think that when, either number one, those roles are not clear to, I think Adam was making a point earlier about security folks need to be providing guidance to engineers where—because engineers are not expected to be security experts, frankly. Security experts are not expected to be engineers. And then, at the point where information is being handed off where the responsibility for taking an action, those, all of those sort of procedures, they have to be clear, otherwise they’re simply not going to happen.
Shimel: Yeah, agreed. I think they have to be clear. But even if we make them as clear as we possibly can, is it just—are we putting too much on developers? I mean, I think fundamentally, that’s the question here, right?
Wong: There’s—
Ashley: Alan, it’s not only—sorry, I’m just gonna jump in. It’s not only quantity of work—is it the right work?
Shimel: Right.
Ashley: Because I know from writing software and being part of software teams for so many years, there’s this thing called unintended consequences or unintended use. Well, you’re not supposed to use it that way, but the person who designed it intended it to be used a certain way, but one of the important functions of tests is not just to uncover bugs and elicit those things or security issues, it’s also to push the boundaries about how you might use the software and essentially stretch it to try to find where it’s gonna break where they can. And oftentimes, that’s a different mentality than or perspective, certainly, than the person that wrote the code.
Dawson: And if I may jump in and say maybe part of the discussion can get to generalist versus specialist, right? There is this idea, Alan, that when you talk about putting a lot in the developer’s hands, this idea that there is this prolific person called the full stack developer that can separate themselves from what they develop to look for security vulnerabilities, to test it, then ultimately deploy and run it, right? And I think we need to give thought to, are we trying to build a bunch of generalists, which doesn’t potentially enable them to do any job the best, or are we enabling specialists to better work together, share information, and crowdsource quality and crowdsource security?
And I’m just curious if any of the other panelists have an opinion on this concept of full stack developers versus, say, needing traditional, lifelong quality experts.
Kalsey: Yeah, I like to say that we decided to create cross-functional teams and we did it by removing all of the functions, and we centralized them all in a single person—and that’s not a cross-functional team. That’s—
Shimel: That’s a cross-functional person.
Kalsey: Yes. Well, and you end up with the jack of all trades, master of none problem. You have to have the specialists on even a cross-functional team. You have to have, there’s a lack of testers, there’s a lack of UX people, there’s a lack of security people. They have to be part of that team. You have to have people that, dedicated folks to do the hard work. A developer may be able, and a generalist may be able to do a lot of the easy things and a lot of the, “Oh, yeah, we’ve done this before or we’ve seen this, and this is a known know, let’s go fix that.”
It’s the hard work of figuring out—what don’t we know, and then having the expertise to be able to know what that hard work is, even, to go after it. And without that, of course the developers are going to be overloaded, they’re being asked to do everything and they don’t know what everything is.
Shimel: Agreed, agreed. I mean, I think, again, security is the poster child for this, right? We’re asking developers to be security people or to make security a priority. Now, you ask most security people four or five years ago, they’d tell you you’re smoking your socks. Developers don’t care about security. They never did. Only we care about security.
It’s changed. It’s changed a little bit. We do think other—no one wants to develop insecure code. But we can’t ask them to be security people—or can we, Caroline?
Wong: I sort of think we have to. I think that, you know, the big, bad thing that no one wants to happen is a security breach, and a security breach cripples an engineering team’s ability to execute on a roadmap. You know, if something big and bad happens, then entire teams have to rush and commit to fixing this problem sometimes for months, you know, time which, of course, could be invested more profitably elsewhere.
I have a quote that I’d love to share from a colleague. Denali Lumma is the VP of Engineering over at Project Ronin, and she also makes a really good point about tech debt. She says, at a certain point, it’s too late, there’s too much tech debt, too much complexity, the code base is too big, and it becomes impossible to do anything about it. What you see there is a business will slowly die because shipping features becomes harder and harder, critical vulnerabilities and bugs are introduced more frequently, and eventually it kills the business.
You know, I often think of software as a living organism, right? Whether you think of it as a tree or as an animal, there is this balance between making sure you’re doing sort of your fun things, your running, but you’re also resting when you need to heal. You’re also—you know, if we’re talking about a human being, you’re also sort of going to the doctor once a year.
Shimel: Agreed.
Wong: Yeah.
Shimel: Agreed. Thoughts?
Kalsey: Quality has to be a mindset, security has to be a mindset, user experience has to be a mindset on the developer time, among everybody there. The user experience person needs to understand and have a mindset of developing and of creating a secure experience and not, “Well, we’re gonna get rid of all the security because that’s hard for people to use.” Everybody has to have that mindset, but you still need the experts and you still need the expertise to be able to do things well. Just having a mindset doesn’t make you fantastic at it. You can say, “Oh, I love to get out and I’ve got an exercise mindset, so I’m gonna go run a triathlon tomorrow.” It’s probably not going to work.
Shimel: Probably not.
Ashley: You know, it’s interesting—we could’ve had this conversation 20 years ago, 10 years ago.
Shimel: Absolutely.
Ashley: You know, why are we still frustrated with tests? What is it with tests that is, you know, it used to be sort of the orphan of the development process, you know, the only thing that happens later than test is documentation. Well, now, [Laughter] it’s still an issue. Why is that? Why are we so frustrated with tests today? I’m curious what folks think.
Dawson: I’m gonna start with one not very complex thought—it’s actually hard.
Shimel: It damn right is.
Ashley: A little more respect—we need a little more respect for tests. [Laughter]
Shimel: Well, but this was like, I remember when we first launched DevOps.com, a popular theme was that, you know, DevOps would kill the testing star, right? That there would be less QA jobs because everything was gonna be automated continuously and shift left, we wouldn’t need testers—poppycock, right? We have as many testers as we ever had, if not more. It didn’t kill off testing jobs. I think it was Adam that said we’ve made testing part of the DevOps team. So, we have people on the DevOps team who are responsible for the various, you know, styles and types of testing that we do, including security.
I think what’s happened, though, is, these DevOps teams—when I first started DevOps.com, if you spoke to developers, they’d say, “Oh, this DevOps is all about the Ops.” And then you’d talk to the Ops people and they’d say, “This is DevOps, it’s all about Dev, it’s Dev-centric,” right? And that’s the sign of a good compromise when they’re both pointing the finger at that.
But it has come—at least as far as testing goes, from what I see, it has sort of tilted where, you know, as we’ve shifted testing left, we’ve shifted responsibility left as well, which kinda winds up closer to the Dev, or certainly to the DevOps teams.
However, though, let’s look at the alternative, and I’m gonna ask all three of you—all four of you have been around the block once or twice. My personal feeling is, is that we do a much better job of test coverage today than we did 7 or 10 years ago, and I think DevOps and continuous automated testing is a big reason for that.
Thoughts? Anybody disagree? Show me wrong.
Wong: I will challenge you, Alan.
Shimel: Okay. I was hoping someone did.
Wong: I’ll challenge you, and the reason is because—so, we’re all familiar with the OWASP Top Ten.
Shimel: Very much.
Wong: There have been a handful of versions of the OWASP Top Ten since the first was produced in 2003. I believe we’re gonna see another version later this year. The current version is from 2017, and if you take the 2017 version of the OWASP Top Ten and you put it right next to the 2003 version, it is sort of mind blowing how similar they are.
Shimel: Yeah.
Wong: And what that says to me is, we actually know exactly what the problems are. We know how to find them, we know how to fix them, and we know how to prevent them, but they’re still around, and so I think that’s—
Shimel: They are, but are there as many of them, right?
Wong: Absolutely. [Laughter]
Shimel: You think there’s as many cross-site scripting or buffer overflows than there was 10 or 15 years ago?
Wong: So, there are some categories that have been eliminated. You know, there are some categories that, for which we can define a secure pattern, and we can encourage folks to use that secure pattern, and for which we can actually eliminate some of these things.
But there’s others or which we cannot. You know, I think about—I recently visited my sister’s home, and they have two young children, a 2-year-old and a 4-year-old. And at night, what they do is, they run a Roomba, which is brilliant, you know? If you could spend your weekend playing at the park with your kids instead of vacuuming, of course you’d wanna do that. And the Roomba works because they sort of take the time to get all the toys off the floor and make sure everything’s clear and once they use the Roomba, they clean it. There is this different work involved in effectively using a Roomba that’s different from sort of manually using a vacuum.
And I do think there are these two distinct categories of testing, one category which you could say is known, repeatable, and automatable, and another for which manual testing, I believe, will always need to exist. There are types of security flaws like business logic flaws, race conditions, chained exploits that I don’t believe that in my lifetime, we will be able to figure out how to teach a computer to find these particular types of issues.
Kalsey: And it’s beyond just security testing. I mean, exploratory testing, from a tester, your automated tests are not going to find, “This error message makes no sense in this context. I did this, I expected—I don’t know what this error message is trying to tell me to do.”
Your automated tools aren’t going to find that. All they’re going to be able to find is, “It didn’t work, we sent an error message. We had the things we wanted to put in the error message there.” You can test that your code is doing the right thing, but if you didn’t design it to do it right, you’re not gonna find that with an automated test.
Shimel: Fair.
Dawson: I would tend to say, for the surveys that we’ve issued and I’ve read, your original statement, “Has test coverage improved significantly?”—yes, I think all indications are, we are getting better at automated testing, we are getting better at that pursuit of that thing we call test coverage.
But I think what’s more important is, has the quality of test improved? Are we testing the right things, right, from functional testing to security testing, various shades of security testing? And I’d say (1) yes, we have improved there by sort of the qualitative and quantitative measures that I’ve seen. But I also think, as we truly start to focus more on quality, we start to achieve more coverage, we start to implement better tests, it becomes more clear, as I think Adam and Caroline are speaking to how much more work there is to do, how much other things there are to improve testing on. Pair that with a constantly changing technological landscape, right? We’ve gotta learn new stacks, we’ve gotta figure out how to test new stacks. We’ve got half of our put in in a SaaS platform, but then we deploy on prem, et cetera, et cetera, et cetera. And then you add to that that, at the same time, we’re trying to transform organizations.
So, Adam, you talked about exploratory testing; Caroline, the human element. I think one of the things that pursuit of the full stack Dev, that pursuit of the fully automated processes failed to do in a lot of orgs I speak to is, we talk quality first, but oftentimes, the DevOps transformation doesn’t involve quality engineering first, right? And I think one of the things we need to get better at is engaging with and partnering with Quality and establishing our automated and modern processes.
Kalsey: Yeah. We talk good in the testing industry a lot about shift left and that testing is moving earlier in in the process, and too many organizations have taken that as, “Well, the developers are the ones that are early in the process, so that means the developers do it.” No! Shift your damn testers left. Get them involved in the design phase. Get them involved in talking about the architecture. Get them involved in the entire thing, not shift the task to the people who are earlier in the process. Shift the whole process, please.
Dawson: Or to a script, yeah.
Ashley: I’m gonna take a whole ‘nother tack, and I thought you were gonna go here, Brian, is, I think we need to shift tests down. We need full stack testing. It’s not application testing anymore. Most of the security errors that we have are due to configuration errors. And what Caroline said was absolutely true. We still have SQL injection, cross-site scripting, and overflows may be less so. But why? It’s because it’s not just in the app layer that those things happen, it’s in that software stack. And I know not every tester can be an expert at the stack itself, but maybe that’s, instead of kinda dividing the DAST and SAST and the different kinds of testing that we do and just deciding, can we shift those in one direction is, let’s think about it from a depth perspective, because that’s where a lot of those errors are coming from. It’s not just the app.
Kalsey: Oh, yeah, and the tooling on this is also, can be super fragmented. You need one thing to test your infrastructure code, you need another thing to test your UI, you need another thing to do the security—and they’re not, consolidating those reports and combining them and figuring out what worked and everything is hard work and there’s nothing that does that. This is a place that Tricentis could definitely improve and improve our own tooling here.
And then, once you’ve done that testing, the results are often in a silo. Okay, I found out I had a problem—how do I fix it? How do I go see the data behind what was going on in the system to understand this and solve this? And now I’ve gotta dive into a monitoring tool or my logs or an observability tool instead of just having everything all in one place.
Shimel: So, I think there’s a—I’m sorry, Caroline, you go.
Wong: No worries, and thank you, Alan. So, we’ve talked about shift left, shift right. Mitch is kinda talking about shift up and down. I think there’s another axis, if you will, which is sort of what I’ll call shift in and shift out. So, you can test things in and out of band, and that’s gonna make sort of a wildly different difference in terms of how those processes run.
You know, I think 10 years ago, security people appreciated the power that came with being a gate in the process, being an end band process. You know, today, it actually makes sense, I think, for a lot of security testing to be out of band, but that removes the sort of power and authority of security people to say something like, “You must fix this, otherwise you’re not gonna go live if it’s happening out of band.”
Shimel: Fair. So, I’m getting a little dizzy, we’re going left, right, up, down, in, out. But—
Ashley: It’s a circle. [Laughter]
Shimel: Yeah, you know, shake it all about. But here, fundamentally, though, are we just kinda dancing—no pun intended—are we just dancing around the real issue, which is, we wanna do more pre-deploy, we wanna do more, we want better quality software, right? I think we could all wrap our heads around that idea and vote for that. We all want better quality software. Security people are at a premium. We can’t shift security people left or en masse. Testing people are also at a premium, though we’ve integrated them into the DevOps teams. Developers are at a huge premium, they’re probably our most expensive resource in terms of humans.
What do we need to do to make everyone’s job easier here, to make this function better? Is it a question we’ve gotta automate more? I mean, maybe that’s what it is, if we automate more, or is it a cultural thing, right, where—
Ashley: There’s also test-driven design, right, which is, as you’re designing it, you talk about, you put together your test strategy of, “Okay, we do it this way. Let’s talk about how we are going to test it, start to lay out the framework for how we think this can be validated.” I mean, it’s not the only thing to do, but I think—so, if you start with that in mind, now you’ve got a path to pursue as opposed to, “Here’s a chunk of software, you folks figure out how to test it, right? We built it, you test it.”
Dawson: Yeah, and I’d add, I think there’s no—again, not only is the practice of testing harder than we often like to admit, but the, you know, the path to kind of getting to where we fully wanna be, where we don’t dread testing, complex, hard path, and I’d say it’s the DevOps trinity, you know, Alan, and we all can probably hit a few points on each.
But you have to address people and culture, as Adam brought in earlier. There’s no getting away from, security needs to be a thought for everybody, right? I can install all the locks I want on the front of my door. Actually, over 24 years—today is my anniversary—my wife has programmed me to lock the door when I come in the house without thinking, right? She’s kind of impressed a security culture on me.
You do have to address people and culture, process, practice, tools and technology. Automation alone will not solve it. A mindset alone will not solve it. All of that needs to come to fruition.
Shimel: Mitch, make a note—invite Brian’s wife to our next security panel.
Dawson: [Laughter]
Ashley: I’ve already looked up her e-mail address—thank you, Brian, appreciate it.
Shimel: Happy anniversary, by the way, too, Brian.
Dawson: Thank you, thank you.
Kalsey: This is one of the places where I think what Caroline said is so important, that this in bound, out of bound—if you’re trying to do everything in line, of course, it’s gonna be harder. In shifting that mindset to doing things continually in the background, we already do so much stuff and process automation in the background where we don’t have people check, “Here’s a step in the process.” No, this is going along in the background, dependency watching and that sort of thing and updates.
We can be doing testing in the background as well for a lot of it, and identifying what needs to be in the foreground and what needs to be in the background. Look, I’ve seen CI/CD pipelines that take hours and hours and hours to run because the tests take six hours to execute and to build. That can’t work in any sort of a reasonable DevOps system.
Wong: I think there’s two parts to a solution, and I’m really, frankly, just parroting what Brian said. I think there’s a collaboration piece between the people. You know, to an earlier part of our conversation, you’ve gotta have the folks with those specialized skill sets. Ideally, you’re able to get access to those folks on demand, which historically has not been something that’s been possible, but that’s beginning to happen.
The second thing is—so, you’ve got your people collaborating, you’ve also gotta have your technology integrated. So, for whatever security tool or processors, how do those outputs actually get to the engineers? What we need is bidirectional integrations for things like Dev ticketing systems. We need things like APIs for test results. I think collaboration at the people place and integration at the technology place—both of which we could call processes—then it’s just, it’s really a repeat of what [Audio skips].
Shimel: Mm-hmm. I mean, but wasn’t this one of the promises of DevOps? I mean, I got into DevOps because I thought it was a better way to do security, truth be told. But one of the promises of DevOps is, we’re gonna automate more, so we could go faster with more quality—two out of three ain’t bad, right? And, you know, did we lose that somewhere? Is that no longer a goal, or have we taken that as far as it goes?
Kalsey: We keep trying to solve it with technology instead of by looking at the culture and changing how people think. And you can’t think the same way and apply new technology to it and expect the technology just to magically solve all your problems.
Shimel: Brian, you agree, or Caroline?
Dawson: I absolutely 100 percent agree, and I’ll agree and then I’ll just add a point. And I know I’m being a bit nebulous about it, but there’s a human element to this. At the end of the day, it’s humans developing software for humans, right? Whether it’s the code on your electronic control module in your car or Waze on your phone or the OS, at the end of the day, the final endpoint is a human. And you know what? At least now, and I know we’re improving, there’s no way that we can ensure—and I don’t mean to accept security here, but to test the quality of the human experience, right?
And think about this—at the end of the day, knowing that the language on a screen is self-descriptive and informs me what to do. Testing the fact that, or ensuring that when I submit my credit card number it’s not vulnerable, right? Whether we’re talking API testing, whether we’re talking security testing, all of this at the end of the day boils up to customer or user experience testing.
And I guess what I’m gonna posit is that we can never entirely automate that. And where we’ve hurt ourselves and possibly made a false promise is, we’re gonna create a bunch of full stack engineers, we’re gonna get rid of, away from operations functions, testing functions, and we’re gonna use TDD and automate everything from the start, Gherkin, yadda yadda yadda. And I believe—look, that has enabled us to go faster, but our impedance mismatch, the limit we’ve hit right now is where we’ve failed to better connect people and enable them to collaborate and share knowledge. So, not just security, not just back end, not just data, but the entire customer experience is of quality. Dev can’t do that alone.
Shimel: No, sir.
Dawson: I’m sorry, Alan, did I get, did I—
Alan Shimel: No, no, that’s all—I’m saying no, sir, you’re 100 percent correct, but I know Caroline wanted to say something.
Wong: So, I do have a comment about automation. I think that DevOps has figured out how to do automation when it comes to defect discovery. I think we’re sort of good at that. I think what we’re still trying to figure out is automation when it comes to workflow and process so that, actually, again, to Brian’s point, the collaboration can happen automatically, rather than due to a human who’s involved and making it happen. How do we get the right information to the right people in the easiest, most efficient way at the right time, and how do we make it super easy for them to do their jobs?
I do think that there remains—I do think that the next evolution of automation and DevOps is gonna go from defect discovery automation, which I think we’ve gotten quite good at, to workflow and process automation.
Shimel: Fair enough. Adam, thoughts?
Kalsey: I like it. I mean, identifying what you’re automating and what’s working in that automation is important, but also identifying what’s not being automated or what in your automation is broken. And not broken from the standpoint of, it doesn’t execute or it doesn’t run to completion, but broken in, it’s not solving the problem you wanted it to solve.
Shimel: Fair enough. Guys, we have time for maybe one more question. I’m gonna pose it and throw it out there to you. Is it possible we’ve shifted too far left or that you can shift too far left and we need a little rubber band, here, going the other way, a little pendulum swing? Or you think we just scratched the surface? Caroline, you look like you wanna say something.
Wong: I think that’s a great question. I do think it’s possible to shift too far left, and the reason is because, while we, I think, are all familiar with the statistics and the studies that say if you fix something earlier in the cycle, it’s cheaper—there’s stuff that exists to the right that doesn’t exist yet on the left. Software development is, you know, to sort of a little analogy that we talked about earlier, I think it’s a dance. I think it’s a dance between people. I think software comes about as a result of the behaviors and the actions and the decisions that various folks take together. And security and quality are emergent properties. They’re not Band-Aids, they’re not vitamins, they’re not painkillers. They have been, as a result of this dance. So, there are some things for which, if you try to test it too early, it doesn’t actually exist in its final form yet. If I take my 6-year-old to the doctor and then I don’t when she’s 21, you know, might we have caught something super important as a 6-year-old that we were able to step in and address right away that would’ve been great? Yes, but as a 21-year-old—guess what? She’s gonna have different problems that were not apparent at age six.
Shimel: Spoken like a true mom and parent, right? Parents, we worry about that. Anyone else? Thoughts on that? Can we go too far left, or have we gone too far left?
Dawson: I, Alan, think it just depends. I think we’ve interpreted left different ways. I think it’s a good, catchy term. It’s an important and powerful catchy term, but as I’ve heard others discuss, we sort of overemphasized this idea of left. One, to Caroline’s point, right, we just do everything here and we’re done, good to go, right? But then this idea that we can put everything in Dev? Well, no, I think—I think, for what it’s meant to communicate, shifting left is an apt term.
I think in terms of ensuring that we’re doing everything we can as far left or as early in the process to mitigate issues later—great. Right? But we don’t just shift security left. Right? Like Caroline said, inbound, out of bound. Actually, we need to shift security everywhere. We don’t just shift quality left, we need to shift quality everywhere.
So, I think there are—we need to be ensuring that it’s almost zero trust software delivery, right? At every stage of the process, from concept where you should be sitting down with QA and the security team to capture the requirements to test, to deploy, to run, we need to be thinking about it, also.
Alan, I’d say shift left is good—shift everywhere is really what we meant to be doing, right?
Shimel: It’s better. Great.
Kalsey: Any time we engage in sloganeering, there’s gonna be people that pick the wrong things from the slogan. I mean, “Hey, we have DevOps. That means we don’t need Ops anymore!” And that’s not what it meant, and there’s people that have taken the wrong cues from shift left, and instead of shifting the processes left, they shifted the tasks onto the people who were left already, and that’s not gonna work.
Shimel: Absolutely. Hey, Mitchell, you wanna tie a bow on this?
Ashley: I’ll put a bow on this. You know, there are admittedly many, many things that I don’t know, but one of the things that I do know is that, if a developer says it’s tested, it’s ready and a QA person says it’s tested and it’s ready, five out of six dentists will choose the QA person. I will, too.
So, there’s something there. There’s a reason why we don’t have DevQA or DevTest. We don’t talk about it that way, we talk about DevOps or DevSecOps. QA is extremely valid and necessary, and we can’t live without it, and I think we need to invest in helping, elevating how we test and how we think about our software applications to test in. So, those dentists will be happy.
Shimel: And Crest, the Crest test for QA. Thanks very much, Mitchell. Caroline, Brian, Adam—thanks for joining us on this episode of DevOps Unbound. Great conversation. For our audience out there, do check it out. We have another episode coming up in—well, not next week, the week after, and then I think we have a roundtable the two weeks after that.
But for now, though, we’re gonna wrap. Many thanks again to Tricentis for sponsoring DevOps Unbound. Thank you for watching, thanks to our panel. This is Alan Shimel, we’re out.
[End of Audio]
