AI is best applied where there are high volumes of consistently structured problems with patterns within domains, and where individuals have high levels of technical competency. There is no surprise that an area with high adoption of AI and proven benefits in enterprises is within IT organizations, and especially application development. Recent studies consistently point toward 20% to 60% improved efficiency of application development, cited in studies by Goldman Sachs, Deloitte, and McKinsey.
Even with the efficiencies, the real benefits appear to be higher quality, and increased maintainability in the applications. Though productivity and cost saving by developers and testers have been significant, they are small potatoes compared with the potential when applied to complex applications and technology management, where value has been sporadic and elusive.
Based on Bain 2024 Survey supplemented by analysis from Deloitte, McKinsey, and IEEE
An in-depth study by the IEEE on the impact of AI on software development found significant efficiencies across three areas:
- Code Generation – 40% improved efficiencies with automation of boilerplate code, increased speed in development, and reduction of manual/bespoke coding.
- Automation Testing – 50% efficiency improvement by automating repetitive testing, more comprehensive testing, and reducing manual interventions.
- Bug Detection –improved efficiencies by identifying bugs more quickly, accurately, and across a wider range of new and legacy applications.
The use of hybrid cloud-based infrastructure, standard architectures and platforms, and API-based integration brings standardization and predictability to application development and enables AI agents to accelerate development with greater efficiency and higher quality. Among the most common tools for code generation are:
- GitHub Copilot: a common AI-based tool that can suggest code structure based on comments describing functionality. During development, it helps write implementation code, generate unit tests, and even identify edge cases that developers might miss. When developers write test cases.
- Copilot: can generate appropriate test data and suggest error scenarios to validate.
- Google’s Gemini Code Assist: another common tool that demonstrates contextual understanding across the system development life cycle. It can analyze changes for potential issues, suggest performance improvements, and ensure compliance with coding standards. It helps document code by generating clear explanations and usage examples.
- Claude@Code: Anthropic code generation tool.
- Cursor: This AI startup makes an AI tool that learns a developer’s coding style to autocomplete, edit, and review lines of code. It has recently raised over $2B in funding and is valued at $29B.
Testing is the software development area with the greatest AI return and effectiveness. AI can detect code issues and inconsistencies very quickly. DeepCode is a tool focused on bug detection and can be utilized as part of a bug prevention strategy, and will suggest improvements in code. AI also enables nearly 100% test case execution through data simulation and aggregation. The resulting code is of high quality, simpler to understand, and easier to maintain.
The above examples of AI adoption in IT are in common use today but should be considered point solutions. They are deployed mostly as programmer productivity tools as part of the development process and best applied to new development on standard platforms within the purview of a single development team.
Even with this positive return, most organizations have not invested enough in processes, training, standards, data aggregation, and technology to realize the potential for higher-quality applications delivered in 50% less time and with more than 50% cost savings.
Future State of AI Adoption within IT
There is a shift in AI utilization from individual developer/team productivity to a more holistic approach that brings another, higher level of quality and productivity to IT operations. There are three broad areas of opportunities that are just starting to be addressed by AI:
Complex core applications – Core business processes and workflows are being automated. Human decision-making is being augmented by AI Agents, and all this is fed by data collected from multiple sources (IoT, internal applications, embedded devices, customers, and external sources). “Business services” are now the focus of technology enablement. Technology is so embedded into business processes that it’s all merging, and core business operations are being enabled by AI. These applications are complex, cut across multiple functional areas, and require integration across many applications. To achieve success here corporate and external information must be organized, managed, analyzed and simplified. Business processes connecting functional teams must be understood. Then, multi-agent management can coordinate and augment human decision-making across the organization. This is where the highest business value will be realized but requires both technical investment and disciplined business transformation efforts
Legacy application remediation – Legacy applications are exceedingly difficult to change due to code language, obscure logic, and opaque business rules. This is where there is a great deal of technical debt and unmaintainable code. This technical debt not only limits future capabilities but also increases the time and cost of adding or changing functionality. AI tools are starting to discover the business rules and application logic embedded in the existing legacy applications and even suggest how to begin to structure new modules to replace the old modules, given the existing architecture and standards. The ability to re-platform is limited, but the ability to understand what functionality currently exists allows the current productivity tool sets to be utilized.
Multi-platform and operations management – While in production, the agentic systems need agent ownership and oversight, approvals as needed, human accountability for their actions and responses, training and retraining requirements across multiple model levels, and upgrades and upkeep of agentic memories and the expert knowledge base. In today’s complex, hybrid hosting and data environments, observability is a key challenge. A lack of real time discovery limit compounds the infrastructure management challenges. The utility and warranty of the services delivered by agents also needs to be monitored and managed by their human supervisors. Therefore, the entire IT operations and management space is going to be completely reimagined by agentic AI systems. There are emergent needs for AgentOps, ModelOps and DataOps tools, FinOps, RiskOps, GRC and CloudOps platforms, to work in an integrated and intelligently orchestrated manner, potentially also through autonomous AI.
To achieve the above three opportunities, organizations need to significantly change how technology is used in developing applications, in bring about operational change and in how to manage the technology-based operation. AI must be deployed holistically across multiple applications and functions. The analysis of all existing applications and infrastructure data must be fed into the AI engine so it can recognize patterns, architecture, processes, inconsistencies, and challenges. This requires investment in the tools and scale of AI-supported infrastructure. Most importantly, like all changes, this requires significant investment of capital, executive focus, and commitment of managerial time.
Steps to Increase Impact of AI within IT
- Continue to encourage and support point AI usage – this builds experience while at the same time bringing efficiencies for personal productivity of IT professionals.
- Inventory and monitor development-centric AI efforts to date (developer productivity) and more complex efforts (legacy modernization, complex core applications, business operations) to gauge maturity and identify best practices.
- Invest in AI-based analysis of full technology-based infrastructure – to go beyond personal productivity, data across multiple functions, environments and applications relative to IT infrastructure, operations, use cases and business rules must be aggregated, organized and managed to take advantage of AI. A significant opportunity is to understand legacy applications, and this requires large investment into data analysis and AI tools to review and analyze applications across the enterprise.
- Establish an IT modernization board to revise operations. The use of AI requires a different approach to IT operations, management and development for complex or cross functional efforts. A team must be assembled to lead the changes and identify how business and IT must work collaboratively in a different manner. This must include how the approach enterprise is using to adapt to continuous operational changes.
- Pilot revised project(s) on high-priority complex technology project. Once a revised approach to IT development and operations is developed a pilot project should be selected to execute, iterate and refine this approach.
- Establish enterprise governance to link strategic intent with endeavors and programs. There is no shortage of opportunities to leverage AI as part of business operational improvement, and managers are continually bringing these opportunities to IT and executive leadership. A steering committee for AI deployment and operational modernization should be formed. This team must focus on strategic intent, evaluate AI and modernization opportunities relative to this intent and ensure that priorities are given to those with the highest alignment, value generation, and/or rapid ROI.
All too often, AI’s potential impact is limited by a lack of vision, an inadequate skills base, and a reluctance to invest. As one of my mentors has said…. “if you are going to the future, bring money.”
The Prometheus Endeavor Digital and IT Management Team develops useful frameworks and insights to help organizations master transformational change.