Potential environmental impact of AI
Given the huge problem-solving potential of artificial intelligence.
It wouldn’t be far-fetched to think that AI could also help us in tackling the climate crisis.
When we consider the energy needs of AI models, it becomes clear that the technology is as much a part of the climate problem as a solution.
The emissions come from the infrastructure associated with AI, such as building and running the data centres that handle the large amounts of information required to sustain these systems.
But different technological approaches to how we build AI systems could help reduce its carbon footprint.
Two technologies in particular hold promise for doing this,
Spiking neural networks
Lifelong learning.
The lifetime of an AI system can be split into two phases:
Training
Inference.
During training, a relevant dataset is used to build and tune, improve, the system.
In inference, the trained system generates predictions on previously unseen data.
Training an AI that’s to be used in self-driving cars would require a dataset of many different driving scenarios and decisions taken by human drivers.
After the training phase, the AI system will predict effective manoeuvres for a self-driving car.
Artificial neural networks, are an underlying technology used in most current AI systems.
They have many different elements to them, called parameters, whose values are adjusted during the training phase of the AI system.
These parameters can run to more than 100 billion in total.
While large numbers of parameters improve the capabilities of ANNs, they also make training and inference resource-intensive processes.
To put things in perspective, training GPT-3 (the precursor AI system to the current ChatGPT) generated 502 metric tonnes of carbon, which is equivalent to driving 112 petrol powered cars for a year.
GPT-3 further emits 8.4 tonnes of CO₂ annually due to inference.
Since the AI boom started in the early 2010s, the energy requirements of AI systems known as large language models — the type of technology that’s behind ChatGPT — have gone up by a factor of 300,000.
With the increasing ubiquity and complexity of AI models, this trend is going to continue, potentially making AI a significant contributor of CO₂ emissions.
In fact, our current estimates could be lower than AI’s actual carbon footprint due to a lack of standard and accurate techniques for measuring AI-related emissions.
Ways to make AI more sustainable
L2 is another strategy for reducing the overall energy requirements of ANNs over the course of their lifetime that we are also working on.
Training ANNs sequentially (where the systems learn from sequences of data) on new problems causes them to forget their previous knowledge while learning new tasks.
ANNs require retraining from scratch when their operating environment changes, further increasing AI-related emissions.
L2 is a collection of algorithms that enable AI models to be trained sequentially on multiple tasks with little or no forgetting.
L2 enables models to learn throughout their lifetime by building on their existing knowledge without having to retrain them from scratch.
The field of AI is growing fast and other potential advancements are emerging that can mitigate the energy demands of this technology.
For instance, building smaller AI models that exhibit the same predictive capabilities as that of a larger model.
Advances in quantum computing — a different approach to building computers that harnesses phenomena from the world of quantum physics — would also enable faster training and inference using ANNs and SNNs.
The superior computing capabilities offered by quantum computing could allow us to find energy-efficient solutions for AI at a much larger scale.
The climate change challenge requires that we try to find solutions for rapidly advancing areas such as AI before their carbon footprint becomes too large.
Spiking Neural Networks (SNNs) - Artificial neural networks (ANN)
Artificial Neural Networks (ANNs)
Inspired by the structure and function of the brain, but do not necessarily mimic them closely.
Information is processed through a series of interconnected nodes called artificial neurons.
These neurons use mathematical functions to process and transmit information.
Widely used in various applications, including image recognition, natural language processing, and self-driving cars.
Spiking Neural Networks (SNNs)
More closely mimic the biological neural networks found in the brain.
Neurons in SNNs communicate with each other using short electrical pulses called spikes.
The timing and frequency of these spikes encode information.
SNNs are theoretically more powerful than ANNs and have the potential to be more energy-efficient.
However, they are still under development and face challenges in training and hardware implementation.
COMMENTS