Recent searches
Search options
Has anyone got a good source of energy costs of training and running an object detection model (eg YOLO) vs LLM/image generator AI? Getting some pushback at work over using AI to count gulls in drone images because "AI uses hideous amounts of energy"
@sarahdalgulls This is one of my main concerns about AI. LLMs are awful and they risk taking down useful applications of AI with them as sentiment turns against them.
@Kimberley me too. I was explaining how object detection works to a colleague, who asked "why cant you just use an algorithm to do this? Why does it have to be AI?"
@sarahdalgulls @Kimberley are your colleagues maybe thinking along the lines of something like OpenCV which predates all the cloud-based stuff?
@scottish @Kimberley I think so, I'd completely forgotten about OpenCV
@sarahdalgulls only anecdata based on experiences at a past employer of guesstimation of training costs based on elapsed hours and specs of Azure virtual machines, encouraging teams to publish model cards which include a co2eq measure. and not taking into account inference! Size of the dataset, model architecture, it's really "how long is a piece of string" :/
https://huggingface.co/blog/carbon-emissions-on-the-hub - Huggingface have tried, but it's opt-in to reporting and recording, so very self-selecting
@sarahdalgulls @concretedog You'lll probably have to start from scratch trying to model the energy consumption of the data centres running the AI models - which would take far longer than getting a Raspberry Pi and rolling your own AI image detection using the new AI hat. https://www.raspberrypi.com/products/ai-hat/
In a previous life I was involved with cloud data centres/hosting and calculating energy consumption is complicated, unless you can find an existing model, but AI and Bitcoin use a LOT of energy.
@roger_w_ @concretedog @d40cht honestly, was not looking for anything more complicated than being able to say that the popular LLMs and generative image creation use a lot more energy than us just training a YOLO image detection model on a dataset of a few thousand images.
But is that right?
@roger_w_ @concretedog @d40cht for the counting I have been using a QGIS plugin with an ONNX model I created in YOLO https://plugins.qgis.org/plugins/deepness/
@sarahdalgulls @roger_w_ @concretedog The YOLO models have 10s of millions of parameters. The largest LLMs have 100s of billions of parameters. The inference cost is (somewhat simplifying) proportional to the number of parameters. It also partly depends on the efficiency of the hardware that the models are run on - but at a very conservative estimate I think you could say your YOLO models are at least 100-1000x more energy efficient.
@d40cht @roger_w_ @concretedog thanks for this - this is really helpful
@sarahdalgulls Object detection models like YOLO can use tiny amounts of energy (e.g. run efficiently on modern battery-powered phones etc). Are you using LLMs for the count or some more traditional ML models? Even if you are using LLMs - you could use them to get some training data (e.g. a few thousand images) and then use this to train/fine-tune something tiny like YOLO.
@sarahdalgulls - sounds like a great project, and I agree that it's frustrating that all of AI is being tarred with a single brush. In terms of data there's this on the YOLOv5 GitHub page (https://github.com/ultralytics/yolov5):
"Training times for YOLOv5n/s/m/l/x are approximately 1/2/4/6/8 days on a single NVIDIA V100 GPU."
A back of the envelope calculation might therefore be 8 days x 24 hours/day x 1kW (*) x 200gCO2e/kWh (**) ~= 40kgCO2e, in which:
* Assume 1kW to power sever + GPU + associated infrastructure. I'd actually expect it to be well under this.
** Approx UK electricity carbon intensity, from government data.
I'm guessing you're fine-tuning a pre-trained model though, and maybe not using the largest size, so it might be much lower. My own experience is that you're talking about milliseconds per inference, even on a laptop with no GPU, so that element will probably only be significant of you're processing lots of images.
@philcowans thanks! This is really helpful, especially as I've been using Ultralytics to train the model.
@vroom 10/10 incredible