Yes they do, but the power requirement of LLMs are much higher.

twoblondesmaybemore:

sztupy:

AI data centres can warm surrounding areas by up to 9.1°C

They discovered that land surface temperatures increased by an average of 2°C (3.6°F) in the months after an AI data centre started operations. In the most extreme cases, the increase in temperature was 9.1°C (16.4°F). The effect wasn’t limited to the immediate surroundings of the data centres: the team found increased temperatures up to 10 kilometres away. Seven kilometres away, there was only a 30 per cent reduction in the intensity.

This is fine

Do non-AI datacenters not also do this? AI as it’s used is generally marketing bullshit and bad for a number of reasons, but the problem here is data centers in an environment that’s already too hot, really; the servers for your favourite website also generate a lot of heat, proportionate to their usage. They could install them somewhere the runoff heat can benefit homes or e.g. greenhouses.

(But also, we do need to stop burning our limited resources on this Silicon Valley slop, ofc)

Yes they do, but the power requirement of LLMs are much higher.

As a rough estimate you can check how much energy it takes to run things locally. A small raspberry pi 5 can use 15W of power, and you can easily run a small Mastodon instance on it for a couple users. It can definitely handle at least 2-3 concurrently, and this includes all of the chatter that Mastodon requires with other servers.

As a contrast running a local LLM will completely use up all the resources in your GPU for a single user. A mid-range GPU that you can run an okayish LLM model will easily eat up 120-130W of power for the duration of the inference session which can easily take a minute for a more complex query.

So in minute your 15W raspberry pi can handle the full federated social media requirement of at least 2-3 users (scrolling, liking, watching media content, reblogging, whatever).

On the other hand a single LLM query will use around 100W of power for the same minute, and will result in a single response for a single user. (which will likely be crap and you’ll need to continue the session with a new request).

So on a per-user basis a “classic” website takes up like 5W of power for a single user, while the LLM inference will take up 100W. That’s a 20x difference.

A data-center will be similar. Yes, classic data center usages also use power, and some of it will actually be power intensive even if they’re not actually for AI use (for example things like video transcoding), but generally for plenty of usual websites and mobile apps, they will be able to handle a lot-lot more users on the same amount of power that a single ChatGPT request take.

And do note while the rPI is not really designed for this, as more specialized equipment will be much more power efficient on a per-user basis for “classical use”. In contrast GPUs are basically very similar regardless if they’re in your home or in a data centre, they usually have more RAM which is more important for the larger models than the GPU speed, and which uses less power, but they will not be much more efficient. Also we have compared a small website to a small, local LLM, and the huge models that you usually run in a datacentre need waaaaay more power than that, so the difference can actually be much more than the ballpark 20x I calculated above.