Recently leaked data allows us for the first time to estimate the carbon emissions from training OpenAI’s GPT-4 — With recent news alerting us that global average temperatures keep rising [1] it’s important to remind ourselves that most human activities have a carbon footprint that contributes towards global warming and other climate change. This is also true for digital technology in general and AI in particular. This article serves…

Two ways to easily estimate the carbon footprint of machine learning models and 17 ideas for how you might reduce it — The environmental impact of machine learning models is increasingly receiving attention, however mostly from academia. Here, the conversation tends to focus on the carbon footprint of language models which are not necessarily representative of the general machine learning field and not enough attention is paid to the environmental impact of…

This post will show you how to transform a time series Transformer architecture diagram into PyTorch code step by step — Transformer models have shown state of the art performance in a number of time series forecasting problems [1][2][3]. In this post, you will learn how to code a transformer architecture for time series forecasting in PyTorch. Specifically, we’ll code the architecture used in the paper “Deep Transformer Models for Time…

This is a discussion of the results of the paper “The carbon emissions of writing and illustrating are lower for AI than for humans” — An interesting study recently made the rounds on the internet. Its title is “The Carbon Emissions of Writing and Illustrating Are Lower for AI than for Humans” [1] and it argues that for some tasks, generative AI is more energy efficient than humans. In essence, the authors estimate the carbon…

Generative AI might have a noticeable environmental footprint, and this story discusses what we may stand to gain in return. — Generative AI is receiving a lot of attention these days. ChatGPT reportedly has hundreds of millions of users and similar functionality is allegedly being built into a multitude of digital products ranging from Microsoft Word and Teams to search engines. The environmental footprint of generative AI could be noticeable if…

What could happen to our environment if billions of people began to use generative AI technology on a daily basis? — This article ponders the question: What would be the environmental impact of large-scale adoption of generative AI like ChatGPT? That is, what might the environmental impact be if billions of people began to use generative AI extensively on a daily basis? The reason this question is interesting to contemplate is…

This guide will show you how to resize your Ubuntu virtual machine to e.g. add more storage — Ever been in a situation where your virtual machine runs out of disk space? This quick guide will show you how to easily assign more disk space to an existing virtual machine (VM) and how to let the VM partition utilize the extra space. Let’s go. Step 1: Expand disk space assigned to VM from your host machine

An estimate of ChatGPT’s costs supports estimate that ChatGPT uses millions of kilowatt hours per month. — I recently published an article in which I estimate ChatGPT’s electricity consumption in January 2023 to be between 1.1M and 23M KWh. 23M KWh is roughly the same amount of energy that 175,000 Danes use in a month on average. ChatGPT’s Electricity Consumption ChatGPT may have consumed as much electricity as 175,000 people in January 2023.towardsdatascience.com

Facebook used 2.6 million KWh hours of electricity and emitted 1,000 tons of CO2 when developing their new LLaMA models. — Facebook recently released four new language models called LLaMA, which outperform GPT-3 — ChatGPT’s underlying language model — on a number of tasks. Following a recent trend on resource consumption transparency in LLM training, Facebook estimate the electricity consumption and carbon footprint of their models in the paper published on…