Software and applications have taken on an outsized role in today’s government – helping agencies and military organizations more effectively and efficiently accomplish their missions and service constituents. But to keep pace with innovation, stay ahead of adversaries and keep solutions secure in the face of an increasingly-sophisticated cyber threat landscape, agencies need to develop and deploy software and application updates at a much faster pace.
Organizations are expediting the software development process by turning to new development strategies and technologies that embrace microservices for smaller yet faster, easier, and targeted updates, advanced development tools like containers, and platforms that make deployments smoother and more seamless.
However, the necessary switch to new approaches like Agile Development and DevSecOps for accelerated ATO – has proven difficult for many of today’s government agencies and military organizations.
The Need for Security in Speed and Functionality
The challenges facing organizations looking to embrace Agile Development and DevSecOps were recently laid bare during a Webinar sponsored by Checkmarx and hosted by the cybersecurity thinktank, Institute for Critical Infrastructure Technology (ICIT), entitled, “DevSecOps: Analyzing Legacy Apps for Agile Development.”
The Webinar, moderated by Nick Sinai, former Obama Administration U.S. Deputy CTO, explored the common deep-seated, underlying organizational issues that keep government and military agencies from employing Agile Development and DevSecOps best practices in the modernization of legacy IT systems.
<Click HERE to watch an on-demand replay of the Webinar>
Joining Nick were Kendra Charbonneau, a lead engineer and enterprise agile transformation coach at U.S. Air Force Business Enterprise Systems, and Rajive Mathur, the former CIO at the Social Security Administration (SSA). And both Mathur and Charbonneau made it very clear what’s at stake should agencies fail to embrace Agile Development and DevSecOps and what was driving their respective organizations to make that shift.
“Agile is important to responding to the need of the warfighter more quickly,” Charbonneau said. “Back when we were developing software with a waterfall methodology, it could take two to three years to get that functionality out to the end-user.” Development speed and the time to innovate are essential for the Air Force. All DoD organizations and branches of the Armed Forces are under pressure to stay ahead of the adversaries’ development and deployment.
However, accelerating software updates isn’t always about functionality. Often, it’s a matter of security. Vulnerabilities in enterprise applications and software remain among the most commonly-exploited entrance points to organization networks. Identifying and fixing those vulnerabilities is increasingly essential for today’s government agencies, as Charbonneau explained:
“…cyberattacks are at an all-time high. They’re happening rapidly. We have to be positioned to change quickly and address [those vulnerabilities]. If we’re to continue doing business the way we’ve been doing business in regards to application development, then we’re going to have a hard time adjusting and addressing these cyberattacks.”
For the SSA, the impetus to innovate quickly wasn’t directly tied to cybersecurity requirements or keeping pace with adversaries. Instead, it was about helping a 60,000 employee-strong agency that provides services to American citizens at every stage of their lives to deliver better service to their constituents. “For us, it was all about service,” Mathur said. “How do you deliver more service, better service, faster – and not necessarily just through phones or field offices, but through any way possible?”
While there was the demand to embrace Agile Development and DevSecOps, shifting to these software development approaches wasn’t easy, and sometimes met with developers’ resistance.
“…there were so many resources [within the Air Force] that simply didn’t understand what it meant to go faster with Agile,” said Charbonneau. “It seemed silly to a lot of them, honestly, because they had only known waterfall development. That’s what they had done for years…”
Encountering Turbulence on the Path to Agile Development
Early on in the panel discussion and presentation, Charbonneau did an excellent job of laying out the application development environment that she inherited at U.S. Air Force Business Enterprise Systems. A decade before her arrival, her organization was mandated to embrace Agile Development and DevSecOps best practices.
“Agile is important to responding to the need of the warfighter more quickly. Back when we were developing software with a waterfall methodology, it could take two to three years to get that functionality out to the end-user.”
Kendra Charbonneau
With the shift to Agile supposedly underway, Charbonneau’s first step was to audit the organization, identify where they were either excelling or stalling to implement new approaches within the SDLC. What she found was less than encouraging. According to Charbonneau:
“My findings were significant….What I found out was – of the 90 programs that had been assessed – we had 23 percent in the Infancy [category], which means they haven’t even begun their Agile journey for one reason or another. There was 46 percent in the fall [category]. That means that they had just started [embracing] Agile and were starting to implement the Agile Development terminology and principles…The Walk [category] had 21 percent. That means they had established a disciplined approach to Agile, were looking at different metrics, and were starting to think about the automation of the application development process. And then we had 11 percent in the Run [category] and zero in the Fly [category].”
That means that significantly more than half of application development teams (69 percent) were stuck – either having failed to make any progress towards embracing Agile Development and DevSecOps or having just started their journey.
Why? Charbonneau’s audit sought to answer that as well and found that six distinct factors played a role in a team’s inability to move towards an Agile Development culture. These included:
- Technical debt: stuck with outdated legacy systems, many of which remained mission-critical and shared resources across multiple applications.
- Product owner involvement: having a product owner that was simply a “bill payer” and not in open communication and collaboration with the end-user or familiar with end-user requirements.
- Contracts: having legacy contracts built around a waterfall approach to application development.
- Training: not having the right training, appropriate training, or failing to apply training quickly enough.
- Tooling: having existing tools suited to a waterfall approach to development and not for Agile Development.
- Resources: considerable constraint in funding, environment, skillsets, and other resources.
Not surprisingly, greenfield projects (new applications and projects starting from scratch) were much more likely to use Agile Development and DevSecOps best practices since many of these six factors did not apply. Unfortunately, many of these six factors probably look very familiar to our government readers, who may find themselves facing some – if not all – of them within their own organizations.
Luckily, both Mathur and Charbonneau offered advice for agencies making the transition to Agile Development and DevSecOps.
Best Practices: Agile Program Management for AppSec
Common themes or threads apparent in the best practices shared by both Mathur – who spearheaded a successful shift towards Agile Development at SSA starting in 2014 – and Charbonneau were the need for portfolio analysis, organizational buy-in, and transparency. These three things were essential to the SSA, particularly as they moved their application development processes towards Agile and DevSecOps.
With the Air Force, understanding what existed and why there were challenges to Agile Development and DevSecOps adoption was vital in identifying the steps, tooling, and training necessary to address the problem.
“[Developers] wanted to do what…senior leaders in our government were asking. However, they just hadn’t been armed with the proper resources, money, tooling, etc., to do it,” said Charbonneau. “But…we know what we’ve got now, and we know what [senior leadership] has to do at this point – start having those hard discussions and finding budget dollars to make it happen.”
Building a coalition of stakeholders invested in making the change to Agile development is as important as knowing the current status of an organization’s journey and an actionable plan to complete the transition.
“Regarding consensus across the agency and across business lines – there’s no doubt that everyone’s heart is in the right place. There’s no doubt that everyone wants to do the right thing. Recognizing that and finding common ground has to be point number one,” Mathur explained. “I had great partners at the SSA and great partners in IT who helped in that regard. You can’t do it alone. You can’t do it unless you have that sort of partnership.”
If you want to learn more about the U.S. Air Force and Social Security Administrations’ respective journeys to Agile Development and DevSecOps, click HERE to listen to a recording of the Webinar.
