At an oversubscribed NTT DATA round-table event held recently in partnership with TechCentral, the topic, “The impact of gen AI on the IT landscape and architecture”, was well received by delegates.

With a diverse range of executives across multiple industries in attendance, the discussion covered a broad range of topics. The session explored the challenges and opportunities presented by this rapidly evolving technology, focusing on its implications for IT infrastructure, cloud strategy, data security and business operations.

Participants were engaging on the subject, with many opinions and views shared from almost all delegates in attendance.

The discussion commenced with an acknowledgment that we are at a pivotal point in technology evolution, where generative artificial intelligence is reshaping IT infrastructure, software development, cybersecurity and business operations at an unprecedented pace. Organisations worldwide are grappling with decisions on cloud strategy, data security and AI deployment models, all of which have significant technical and strategic implications for organisations.

A key focus was the increasing demands gen AI places on IT infrastructure and operations, particularly in compute power, data storage and bandwidth. A banking executive emphasised how customer-facing digital services drive infrastructure demands, leading to greater data storage and bandwidth needs. This trend reflects broader challenges, especially for organisations with legacy infrastructure struggling to maintain agility. Notably, the ease of procuring cloud resources in the gen AI era introduces sustainability concerns, necessitating robust governance to prevent infrastructure sprawl.

Gen AI’s impact

The discussion shifted to gen AI’s impact on cloud and hybrid IT strategies. Cloud environments offer a cost-effective platform for AI experimentation and avoids upfront capital expenditure. At the same time, concerns about cloud cost management and governance came up as critical considerations. While cloud services allow scalability and access to advanced AI capabilities, participants acknowledged that cost reduction should not always be the key driver, necessitating a thorough evaluation of the total cost of ownership. Many organisations anticipate adopting hybrid models, leveraging both cloud and on-premises infrastructure. The increasing demand for colocation services, alongside the shrinking demand for some internal data centres, further supports this transition.

Data security and sovereignty are key concerns shaping cloud adoption decisions. There is a growing trend of organisations prioritising private data centres and colocation services to comply with data sovereignty regulations. Operational technology systems, often requiring on-premises presence due to sensitivity, further complicate cloud strategies. Organisations must balance the benefits of cloud adoption with security, regulatory and infrastructure considerations.

Real-world AI and gen AI applications across industries were shared, showcasing their potential impact. Examples included a university leveraging a large language model to classify research outputs, a counselling chatbot for students, AI-driven logistics solutions for shipment analysis and AI optimisation in energy sector gasifier operations. However, participants noted the need to define practical AI benefits beyond pilot phases, as many organisations struggle with inadequate infrastructure to support their ambitions.

The need for skills development and AI literacy was extensively discussed. Organisations face the challenge of fostering gen AI experimentation while maintaining data security and governance. Participants emphasised the importance of tailored training programmes at various organisational levels to ensure responsible AI adoption. The shortage of AI-related skills in the market exacerbates these challenges, necessitating internal upskilling initiatives and partnerships with external vendors and partners.

Finally, governance of AI usage and data emerged as a crucial theme. The tension between enabling innovation through accessible cloud resources and maintaining control over data and costs was evident. Participants discussed implementing internal guardrails and automated governance processes to strike a balance. A multifaceted governance approach – combining technical controls, employee education and strong organisational values – was deemed essential. As AI and gen AI become increasingly democratised, allowing broader organisational access to AI development tools, ensuring robust governance frameworks will be key to sustainable adoption.

In conclusion, the NTT DATA round table provided great insights into the evolving gen AI landscape, highlighting both opportunities and challenges. While organisations recognize gen AI’s transformative potential, they must navigate infrastructure demands, security concerns, cost management and governance frameworks strategically to harness its full benefits.