If you listen to Big Tech companies, generative AI is going to lead to a sustained boom in productivity, propelling the global economy to new heights.
There’s no doubt that the rise of gen AI has spawned huge excitement in the technology sector – and across industries. Yet many of the technology’s purported benefits remain unproven and largely untested.
The buzz around gen AI presents a quandary for company leaders. They want to remain at the forefront of the technology to take advantage of the opportunities it presents. But they also want to avoid making hype-driven investments that could prove to be a waste of money in the long run.
One area that generative AI is promising to revolutionise is software development. Already, tools such as DeepCode AI, Devin and OpenAI Codex – the ChatGPT-inspired code generator which powers Microsoft’s GitHub Copilot and Visual Studio Code – are promising to make the job of coding much easier, with copilots to assist developers every step of the way.
But are these AI developer tools a boon for rapid software development or a fad that will only make existing codebases more complex to work with?
“We see a lot of AI tools that promise to accelerate our development and help us create products faster,” said Lucky Nkosi, head of research and development at software house BBD. Nkosi said BBD tests AI development tools by first giving them to its senior programmers for feedback. Junior developers are then given a chance, too.
“Our senior developers found it took away a lot of the labour-intensive and repetitive tasks they would usually dread doing. Our juniors also found them to be useful, but they feared they would lose the opportunity to learn the fundamentals that the seniors have mastered over the years.”
Significant risk
Nkosi said AI coding tools have proven to be great when developers “know what they are looking for”. But he said they present significant risk in the hands of a novice. Experienced software engineers, he added, have a better understanding of the context the code they (or their AI tools) write. They take the rest of the code base into consideration, including the required integrations with other applications in the enterprise.
“When you are building a small app, you can a make mistake here and there. For enterprises, it is not about just getting things to work; it must work all the time under all circumstances, and it must be maintainable,” said Nkosi.
He likened the risks associated with the adoption of AI tools to low- and no-code platforms that rose to popularity in recent years for their speedy development cycles. The speed advantage these platforms provide often vanishes, however, when applications need to scale.
Read: Corporate South Africa dips its toes into the AI waters
Some organisations have found they have to build such applications from scratch when scaling them or trying to integrate them into other ecosystems. “My concern is the scalability of these [AI-developed] applications both in terms of performance and customisability as you go from, say, 10 000 to 10 million users/requests,” he said.
Another risk presented by AI development tools relates to the sanctity of data, since exposing these tools to sensitive intellectual property could lead to data leaks that erode a company’s competitive advantage.
Also posing a risk to companies, said Nkosi, is the AI “arms race” and the pressure that puts on software vendors to ship products that “are not ready for human consumption” because of fears of being outpaced by competitors. There is a good chance that there are a lot of “half-baked solutions” in the market, he said.
Who AI-generated code belongs to is also open to question. Companies making use of gen AI software often have little insight into where the data the generative model was trained on comes from, which makes assessing the associated IP-related risks difficult.
“We don’t yet have the legal precedents that dictate how such matters are going to be interpreted.”
According to Nkosi, the usefulness of AI coding tools is not in doubt. But how they’re adopted is critical to organisational success, and supplementary training to hammer home the fundamentals of software engineering is useful in keeping employees skilled enough to know when the AI is misbehaving.
But some businesses are not finding AI-infused software development tools to be as useful to their coding practices as BBD. According to Patrick O’Reilly, CIO of MIP Holdings, a specialist software development house for the financial services sector, gen-AI software tools are proving to be less effective for the kind of complexity the company has to deal with on a daily basis with its clients.
“We found that we can develop snippets of code using AI, but for more complex applications you have to be explicit in telling it what you want it to do – and then it becomes questionable whether you are saving time or not. We think that an experienced developer who knows what they are doing can go faster,” said O’Reilly.
Known use cases
He said that when MIP tested gen-AI coding tools, it found them to be effective at solving for known use cases where multiple examples of the given problem are available as training data online. However, in scenarios where the problems are open ended or the context surrounding the solution requires a lot of industry-specific expertise across multiple disciplines, the gen-AI tools are not up to the task.
“Some of our tasks require deep industry expertise across multiple disciplines, including legislation that is not well fleshed out yet. When you are trying to introduce a new concept for a never-seen-before product, there are lots of assumptions or ambiguities that an industry expert would implicitly understand, whereas you have to tell the AI everything [for it to work effectively],” said O’Reilly.
He conceded, however, that the same thing could be said about people who are new to a particular industry. Gen-AI tools could become more useful over time, especially if the AI models are trained on industry-specific data sets.
O’Reilly said the lack of visibility on the data used to train commercial AI tools may inject biases into code, leading to potential problems if that code isn’t corrected before being deployed in production systems. “If you don’t know how it’s being trained, then you don’t know if you can trust that training,” he added. – © 2024 NewsCentral Media
