This article was contributed by Carolyn Duby, the Solutions Architect and Cyber Security Lead at Cloudera.
Gartner anticipates an estimated 300 percent growth in connected devices by 2020 sparked by faster 5G networks bringing an explosion of inexpensive, connected devices.
While this explosion in connected devices enables new and more innovative applications and capabilities to improve constituent service and experience, all these new applications and connections also broaden the network attack surface. This increases the risk of data breaches and cyberattacks, especially for government agencies, which face an increasingly large and sophisticated threat landscape.
Agency security operations and teams are already short-staffed and struggling with alert fatigue. Too many security tools, too many alerts – and not enough context or staff resources for investigations.
Security professionals need new ways to find the right data and prioritize alerts for appropriate investigation. And perhaps the greatest challenge of all – identifying and preventing Advanced Persistent Threats (APTs) – one of the most insidious cyber threats in technology today.
Once intruders get into a system, they may lurk for months or even years before they strike. The results can have devastating consequences, including large-scale theft of constituent data, government secrets, intellectual property (IP), personally identifiable information (PII), or other sensitive data.
So it should come as no surprise to anyone that cybersecurity has become an urgent topic of conversation for organizations across the entire government, and a priority investment among most IT departments. And for good reason: theft, fraud, lost intellectual property are a persistent threat. Reputational damage alone can even kill a business outright.
As a result, organizations are looking for new ways to detect and investigate cyber threats. And today’s advanced artificial intelligence (AI) and machine learning (ML) technologies could play a role in making threat investigation easier, more intuitive, and more effective.

Harnessing the power of AI for security
A new generation of advanced cybersecurity platforms leverages the power of AI to increase security operations efficiency by weaving security point solutions together into a common repository and identifying the most important alerts – all the way from the Edge to AI.
These platforms collect and move, prepare, prioritize, and organize security data in real-time, at scale, enabling users to store, visualize, and train models on years of security data. This allows users to spot the hidden threats or improve on traditional rules in changing environments. But ML is about more than just new behavior models for detection.
Embed ML in processes can augment people with timely, actionable insights so they can focus on the right risks while machines do the busy-work. Their fast transition from the lab to production enables users to meet fast-evolving threats and discoveries. They even allow users to address their challenges related to limited or poorly categorized data.
But what should a government agency be looking for in a next-generation cybersecurity platform? Here are three characteristics that ensure an advanced cybersecurity platform is capable of meeting the rigorous security risks and challenges facing government agencies:
Offers real-time data ingestion. If data isn’t being moved and processed in real-time, it becomes increasingly difficult to respond rapidly and minimize the impact of cyber incidents.
Has scalability baked-in. Storing longer context of data and new types of high-volume security data can be increasingly expensive or potentially exceed legacy tool ingestion limits. For example, storing DNS, netflow, packet, and IoT captures can quickly tax an agency’s infrastructure. This is why it’s essential that the cybersecurity platform that an agency chooses be scalable and cost-effective.
Is open-source, extensible, and configurable. Government agencies are inherently resistant to vendor lock-in and want to keep their security data unlocked. The cybersecurity platform that an agency chooses should offer flexibility to control how security data is organized, retained, and accessed and give users the ability to use their favorite visualization and analytics tools to explore and build custom dashboards.
Utilizing machine learning and artificial intelligence for advancing the detection of cyber threats is a hot topic across state, local, and federal governments and agencies. And for very good reasons. The benefits are real, and the threats are relentless and sophisticated. If agencies are going to tackle the challenges of an increasingly connected world, their cybersecurity platforms are going to have to get smarter.
For additional information on how to leverage advanced ML and AI technologies in threat detection and investigation, click HERE to watch complimentary Webinar, “Detect, Respond, Comply: Cybersecurity at Scale.” Or, click HERE to download the complimentary whitepaper, “Enhanced Cyber Readiness.”