• 0 Posts
  • 5 Comments
Joined 8 months ago
cake
Cake day: April 8th, 2026

help-circle
  • Monument@piefed.worldtoSelfhosted@lemmy.world•Is anyone self-hosting their own AI?
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    1
    ·
    edit-2
    1 month ago

    I wouldn’t say I ‘run’ it, but I mess around with it

    I have a base level Mac Mini I got before the price increase a few years ago, also from when models were smaller and my ambitions were less. I’ve contemplated upgrading it, but I have other things to spend my money on.

    M4 processor, 16 gb ram, 256 gb storage, and a 4tb external HDD drive (an old NAS drive).

    It’s fine. It’s mostly a docker machine, but I do throw AI tasks at it a few times a week. Mostly stuff that I just let it chew on for 5-10 minutes and come back to.

    I run openwebui, and have several models I bounce between. I mostly run Gemma4 (Google) and Granite4.2:8B (IBM).

    Editing to add: the self hosted version of all these models, via llama, just in case that was unclear.





  • So I recently(ish) went through this - migrating from consumer hardware to rolling my own.
    Here’s what I did:

    I bought a mini-PC router and loaded OPNsense onto it.
    I needed wireless AP’s in some odd places, so I bought a pair of POE-powered Netgear WAX620 AP’s because they were a decent price, and a 2.5G POE+ Switch.
    I probably would not go with Netgear again. They try to lock you into their cloud (subscription) platform. I don’t dig it. I would probably also not go with a POE switch unless I had to, because it adds a lot to the cost.
    If I had planned better, I’d have waited until a decent older switch became available from a local surplus source. (The local university has a public surplus site that sometimes has interesting and cheap networking gear.)
    If you plan to set up VLANs, make sure your switches are up to the task.