ChatGPT & Sustainability - Green Software Foundation “Expert Perspectives”
Artificial intelligence systems use vast amounts of energy. ChatGPT is arguably the most well-known AI application in today's IT landscape. How do experts see the role of ChatGPT and other Large Language Models (LLM) in light of the climate crisis? What are the benefits of AI, and how can they outweigh the negative aspects of increased energy consumption and, more specifically, carbon emissions attributed to developing, testing, and operating such applications?
LLM a major source of competitive advantage for businesses
We spoke to the experts at Boston Consulting Group, one of the world’s leading technology consultancies. Here is what they had to say:
“We have built a custom LLM for our knowledgebase Q&A interface, and we have numerous experiments running currently using OpenAI GPT-3.5 and GPT-4 API for various internal applications. We are actively pitching projects to clients to use LLMs for their use cases. We see this as a major source of business over the next few years,” says Mathew Kropp, Managing Director and Partner at BCG X.
“LLM are a step-change in language understanding and generation performance vs. prior NLP/NLU models. This unlocks a host of use cases that can transform entire business functions. We believe that for many businesses, adopting GenAI solutions within their internal operations will be table-stakes to remain cost-competitive, and incorporating them into their proprietary business processes can create a huge competitive advantage,” adds Niels Freier.
The current focus for many corporations is on developing the functionality and adoption of LLM. Training them is enormously compute-intensive, and they are continuously retrained to incorporate more real-world knowledge.
“The carbon footprint is likely to be massive. But there is promising work reducing the number of parameters used with smarter training data set curation. We expect that the underlying base language models may soon get to a sufficient quality level and that the focus will turn to the much smaller and less carbon-intensive models. But this will take an effort on behalf of the industry to avoid a GPT arms race for ever-larger numbers of parameters,” said Kropp.
Societal Impacts and Sustainability Concerns Top of Mind
Avanade, the leading innovator of solutions across the Microsoft ecosystem, has been at the forefront of understanding the potential of ChatGPT and other large language models available on Azure Open AI. A recent blog published by Avanade CTO, Florin Rotar, shares a perspective on these technologies, representing remarkable AI advances. We spoke with Miranda Hill and Chris McClean.
“In my role as Global Sustainability Lead at Avanade, I am a champion for technology yet must also walk a fine line where I need to understand the dark side of tech. Let’s face it; technology is carbon intensive. Gartner predicts that by 2025, AI will consume more energy than the human workforce. That sounds like a bold and dystopian statement,” says Miranda Hill.
Chris McClean, Global Lead Digital Ethics at Avanade, sees an additional layer to the conversation in the ethical and societal impacts. Generative AI engines, as they are trained to use content from the Internet, are picking up the Internet’s biases, stereotypes, and sometimes precarious points of view into their knowledge base and treating them as facts.
“From a societal perspective, there are legitimate concerns about a massive increase in misinformation as well as jobs being taken over by AI. So the question is, how can we use these systems to improve people’s lives rather than reduce people’s worth? We can all admire the tech, but taking a more systematic view of the risks, trade-offs, and impacts on people and society will be a necessary step in the process. And that’s definitely the more courageous and responsible thing to do,” says McClean.
When asked about possible remedies to the carbon footprint of LLMs, Miranda Hill answered:
“Turning the dials on how you approach infrastructure and your tech estate investments can be a good start. For instance, migrating to cloud solutions can reduce your total cost of ownership by 30-40%, and if you go with a sustainable cloud provider, you can lower your IT footprint even further. Once you migrate to the cloud, you want to do everything you can to optimize the carbon footprint.”
Avanade has developed an excellent tool for optimization actions, which offers a carbon footprint view for every project. But that is only one angle:
“That’s just the tech estate…it doesn’t even touch the code in app development. This is where green software principles come into play. The principles established by Green Software Foundation seek to lower the carbon intensity of the code itself, which means AI algorithms are put through standardized frameworks that lower software carbon intensity, thereby empowering engineers to reduce the carbon footprint in their AI workloads," says Hill.
If you are a software or data engineer, one of the best things you can do is to get the Green Software Practitioner Training and understand how to be more sustainable with your code.
The pivotal role of cloud vendors in disclosing emission data
We spoke to Yusuke Kobayashi, Manager Green Innovation Office at NTT DATA, a renowned multinational IT services company.
“We are worried that the amount of electricity consumption and GHG gas emissions with LLM apps will increase. We are also concerned that as the models become more sophisticated, LLM will be provided more often through third-party API services rather than as a self-developed service, so we may no longer keep track of the electricity usage and GHG emissions or have room for our reduction efforts.”
Kobayashi believes it is important to understand and quantify the environmental impact of using LLMs. Both will enable comparing the models and stimulate efforts to improve them, such as considering the environmental impact reduction by adjusting queries.
“To achieve this, we believe it will become necessary to require service vendors to disclose their environmental impact, i.e., their CO2 emissions. That enables us to make choices, such as selecting and using services that disclose their environmental impact, reduce associated emissions, and use green power. In addition, we have to consider non-environmental perspectives such as human rights.”
“Also, technologies to run large models with low power consumption are evolving. With the use of next-generation computing technology with low power consumption and the emergence of technology to run large models with smaller resources, we believe that these technologies should be used in parallel efforts to reduce the baseline emissions.”
Overcoming the Carbon Emissions Conundrum
Software development shouldn’t happen without considering sustainability, and LLM apps are no exception. While introducing extensive computing needs, they have the potential to bring about carbon footprint reductions across industries as the productivity gains from using these models start to be realized.
The industry is also working on improving the carbon footprint of LLM with system level optimizations across hardware and software. We spoke to Patrick Chanezon, General Manager Cloud Developer Advocacy at Microsoft.
“With our Azure OpenAI Services, along with ChatGPT being trained and running on Azure, Microsoft is considered one of the large LLM providers. As part of our commitment to creating a more sustainable future, Microsoft is investing in research to measure the energy use and carbon impact of AI while working on ways to make large systems more efficient in both training and application. We are also continuing to invest in purchasing renewable energy and other efforts to meet our sustainability goals of being carbon negative, water positive, and zero waste by 2030,” says Chanezon.
Microsoft has been at the vanguard of system level optimization for years. In their article How Microsoft’s bet on Azure unlocked an AI Revolution, author John Roach puts it like this:
"The system level optimization includes software that enables effective utilization of the GPUs and networking equipment. Over the past several years, Microsoft has developed software techniques that have grown the ability to train models with tens of trillions of parameters while simultaneously driving down the resource requirements and time to train and serve them in production.”
Microsoft has also helped develop a framework for measuring software carbon intensity and collaborated with leading universities, the Allen Institute, and Hugging Face to create a tool that measures the electricity usage of machine-learning programs that run on Azure.
“We’ve made advances with DeepSpeed, for training efficiency and ONNX Runtime, which gives high-performance inference support for large Transformer-based models, helping to optimize consumption and latency,” adds Chanezon.
This article is licenced under Creative Commons (CC BY 4.0)