Interests: programming, video games, anime, music composition

I used to be on kbin as e0qdk@kbin.social before it broke down.

  • 0 Posts
  • 27 Comments
Joined 3 years ago
cake
Cake day: November 27th, 2023

help-circle
  • If you just pulled the default version of qwen3.5 from ollama’s repo you downloaded a mediocre one that only uses ~6GB.

    Check ollama show qwen3.5 and see if you get something like this in the result:

      Model
        architecture        qwen35    
        parameters          9.7B      
        context length      262144    
        embedding length    4096      
        quantization        Q4_K_M 
    

    This is the default version I got when I first tried using ollama without any experience. It worked, but it’s a heavily quantized, lower parameter version of the model – i.e. it’s pretty dumb – compared to what you can actually run on your hardware.


  • e0qdk@reddthat.comtoSelfhosted@lemmy.world•Do you host your own AI?
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    1
    ·
    5 months ago

    I started running LLMs a couple months ago on my own hardware. I have a Framework Desktop that I ordered last year and also recently picked up a refurbished 24GB AMD RX 7900 XTX which I’m doing some performance testing against. The dGPU is much better for dense models, and slightly faster for MoE if I’m willing to run them at a lower quant – but uses more power and has annoying coil whine. The Framework Desktop uses ~100W under load, is quieter, and for the MoE models already runs them fast enough for most of my needs – so most of my LLM use happens on that system still.

    For software: I’m using ollama on the Framework currently, but I want to replace it with just using llama.cpp directly eventually. I’ve been using llama-cli for testing the dGPU. I wrote my own chat client to interact with ollama as well as a few other programs for specific tasks.

    I’ve been using the LLMs for a mix of research (both personal and professional), entertainment, practical coding tasks (mostly debugging and brainstorming, plus a bit of UI prototyping, automatic generation of sequence diagrams for documentation, and light scripting), as well as automation of tedious tasks.

    As an example of the latter, people often send me requests to prepare data sets by email but don’t specify the sources they want precisely so I have to go match the name against the real name in our archives; LLMs are great for mapping the imperfect name – with typos, missing prefixes, incorrect addition of spaces, addition/removal of hyphens, etc. – to the exact name I actually need to pull the data off disk when given a lookup table to compare against.

    As far as models go, I’m mostly using various Qwen 3.6 and Gemma4 variants. I have multiple versions of each for different purposes. llmfan46’s uncensored Qwen 3.6 35B-A3B @ Q6_K (from Hugging Face) is my default model currently.


  • Should be trivial to set up something like that if you’ve got parts you want to work with. Any desktop with an automatic background switcher should be able to cycle through images in a directory you specify on a timer. Set up your favorite remote access software (SSH, Samba, NFS …) and you’re done. If you want more control over the behavior, you could script up something custom with a little more effort – but it’s still not particularly hard to implement something like that.

    Watch out for burn in on the screen if you’re leaving it on all the time.



  • Assume an unlimited budget for now, I just want to know what’s out there.

    I mean, if you’re willing to pay the price of a car per SSD they go up to at least 122TB density per drive… (e.g. Solidigm SBFPF2BV0P12001 D5-P5336 – $16K~$20K depending on supplier from a quick search)

    I don’t actually recommend that for personal use, but since you were curious about what’s out there, there’s some absolutely crazy shit in enterprise server gear if you have deep enough pockets.



  • There’s something else going on there besides base64 encoding of the URL – possibly they have some binary tracking data or other crap that only makes sense to the creator of the link.

    It’s not hard to write a small Python script that gets what you want out of a URL like that though. Here’s one that works with your sample link:

    #!/usr/bin/env python3
    
    import base64
    import binascii
    import itertools
    import string
    import sys
    
    input_url = sys.argv[1]
    parts = input_url.split("/")
      
    for chunk in itertools.accumulate(reversed(parts), lambda b,a: "/".join([a,b])):
      try:
        text = base64.b64decode(chunk).decode("ascii", errors="ignore")
        clean = "".join(itertools.takewhile(lambda x: x in string.printable, text))
        print(clean)
      except binascii.Error:
        continue
    

    Save that to a file like decode.py and then you can you run it on the command line like python3 ./decode.py 'YOUR-LINK-HERE'

    e.g.

    $ python3 ./decode.py 'https://link.sfchronicle.com/external/41488169.38548/aHR0cHM6Ly93d3cuaG90ZG9nYmlsbHMuY29tL2hhbWJ1cmdlci1tb2xkcy9idXJnZXItZG9nLW1vbGQ_c2lkPTY4MTNkMTljYzM0ZWJjZTE4NDA1ZGVjYSZzcz1QJnN0X3JpZD1udWxsJnV0bV9zb3VyY2U9bmV3c2xldHRlciZ1dG1fbWVkaXVtPWVtYWlsJnV0bV90ZXJtPWJyaWVmaW5nJnV0bV9jYW1wYWlnbj1zZmNfYml0ZWN1cmlvdXM/6813d19cc34ebce18405decaB7ef84e41'
    https://www.hotdogbills.com/hamburger-molds/burger-dog-mold
    

    This script works by spitting the URL at ‘/’ characters and then recombining the parts (right-to-left) and checking if that chunk of text can be base64 decoded successfully. If it does, it then takes any printable ASCII characters at the start of the string and outputs it (to clean up the garbage characters at the end). If there’s more than one possible valid interpretation as base64 it will print them all as it finds them.


  • Nginx is running in Docker

    Are you launching the container with the correct ports exposed? You generally cannot make connections into a container from the outside unless you explicitly tell Docker that you want it to allow that to happen… i.e. assuming you want a simple one-to-one mapping for HTTP and HTTPS standard ports are you passing something like -p 80:80 -p 443:443 to docker run on the command line, adding the appropriate ports in your compose file, or doing something similar with another tool for bringing the container up?







  • I’ve worked for a university before and it was very common for staff to remote into their systems from home – usually with SSH for CS types or Remote Desktop/Team Viewer/etc. for less computer-focused folks. (The former usually didn’t have much issue – the folks using the latter mechanisms got compromised a number of times… -.-) There was also a campus provided VPN that was required to access certain systems with instructions to students and staff on how to use it, but other systems just got public IP addresses.

    If what you’re doing is related to your work and campus IT doesn’t object, you’re probably fine to do it. I’ve run various kinds of websites and web apps for colleagues to collaborate on research projects. Being able to do things like that is kind of the point of the internet.

    Having seen a number of students, uh, push the limits and find the boundaries of acceptability the hard way though… I’d strongly advise you not to install cryptominers, run TOR exit nodes, or torrent TV shows/movies/etc. That kind of thing tends to get your systems in hot water with IT or other parts of the bureaucracy…





  • Two quick ideas on possible approaches:

    1. Static page route. You can just write some Javascript to load the image from a file input in HTML, draw it resized to a canvas (based on an input slider or other input element), then save the canvas to an image. (There might even be simpler approaches if I wasn’t stupidly tired right now…) This can be done in a single file (HTML with embedded JS – and CSS if you want to style it a little) that you toss on any web server anywhere (e.g. Apache, nginx, whatever). Should work for JPEG, PNG, and probably WebP – maybe other regular image types too. Benefit: data never needs to leave your device.

    2. Process on server route. Use Python with a simple web server library (I usually opt for tornado for stuff like this, but flask or cherrypy or similar would probably work). Set up a handler for e.g. an HTTP POST and either pass the image into a library like Pillow to resize it or shell out to ImageMagick as others have suggested. (If you want to do something clever with animated GIFs you could shell out to ffmpeg, but that’d be a fair bit trickier…) The image can be sent back as the response. Be careful about security if you take this route. Probably want some kind of login in front of it, and run it in a VM or some other secure environment – especially if you’re using AI to kludge it together…

    Best of luck and let me know if you need any help. Will probably have some time this weekend if you can’t get it on your own. Happy hacking!


  • I would be happy with a FOSS desktop app I can install in linux too

    On the command line, you can do this with ImageMagick (e.g. use the command convert once it’s installed).

    With a (desktop) GUI, there’s a bunch of programs. GIMP is probably the most well known and has a ton of capabilities but is a bit complex. I use Kolourpaint as a quick-and-dirty “MS Paint”-like program for very simple tasks where I want a GUI.

    If you want a simple web UI I’m sure there is one already, but I don’t know one specifically. It wouldn’t be too complicated to hack something up if all you need is a quick-and-dirty file input and percentage rescale or something like that. If you don’t get a better suggestion and don’t know how to make something like that yourself, let me know and I can write an example.


  • People have already covered most of the tools I typically use, but one I haven’t seen listed yet that is sometimes convenient is python3 -m http.server which runs a small web server that shares whatever is in the directory you launched it from. I’ve used that to download files onto my phone before when I didn’t have the right USB cables/adapters handy as well as for getting data out of VMs when I didn’t want to bother setting up something more complex.