• CoolerOpposide [none/use name]@hexbear.net
    link
    fedilink
    English
    arrow-up
    2
    ·
    5 months ago

    All of this to run a program that is essentially typing a question into Google and adding “Reddit” at the end of it.

    They spent so much time disconnected from reality and trying to create artificial intelligence that they forgot regular intelligence exists

  • Kalkaline @leminal.space
    link
    fedilink
    arrow-up
    1
    ·
    5 months ago

    Don’t forget AMD, good potential if they bring out similar technology to compete with NVIDIA. Less so Intel, but they’re in the GPU market too.

    • TheRealKuni@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      5 months ago

      I doubt it. Regardless of the current stage of machine learning, everyone is now tuned in and pushing the tech. Even if LLMs turn out to be mostly a dead end, everyone investing in ML means that the ability to do LOTS of floating point math very quickly without the heaviness of CPU operations isn’t going away any time soon. Which means nVidia is sitting pretty.

      • umbrella@lemmy.ml
        link
        fedilink
        arrow-up
        1
        ·
        5 months ago

        the WWW wasn’t a dead end but the bubble burst anyway. the same will happen to AI because exponential growth is impossible.

    • Naz@sh.itjust.works
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      5 months ago

      I’m an AI Developer.

      TLDR: CUDA.

      Getting ROCM to work properly is like herding cats.

      You need a custom implementation for the specific operating system, the driver version must be locked and compatible, especially with a Workstation / WRX card, the Pro drivers are especially prone to breaking, you need the specific dependencies to be compiled for your variant of HIPBlas, or zLUDA, if that doesn’t work, you need ONNX transition graphs, but then find out PyTorch doesn’t support ONNX unless it’s 1.2.0 which breaks another dependency of X-Transformers, which then breaks because the version of HIPBlas is incompatible with that older version of Python and …

      Inhales

      And THEN MAYBE it’ll work at 85% of the speed of CUDA. If it doesn’t crash first due to an arbitrary error such as CUDA_UNIMPEMENTED_FUNCTION_HALF

      You get the picture. On Nvidia, it’s click, open, CUDA working? Yes?, done. You don’t spend 120 hours fucking around and recompiling for your specific usecase.

      • barsoap@lemm.ee
        link
        fedilink
        arrow-up
        1
        ·
        edit-2
        5 months ago

        Also, you need a supported card. I have a potato going by the name RX 5500, not on the supported list. I have the choice between three rocm versions:

        1. An age-old prebuilt, generally works, occasionally crashes the graphics driver, unrecoverably so… Linux tries to re-initialise everything but that fails, it needs a proper reset. I do need to tell it to pretend I have a different card.
        2. A custom-built one, which I fished out of a docker image I found on the net because I can’t be arsed to build that behemoth. It’s dog-slow, due to using all generic code and no specialised kernels.
        3. A newer prebuilt, any. Works fine for some, or should I say, very few workloads (mostly just BLAS stuff), otherwise it simply hangs. Presumably because they updated the kernels and now they’re using instructions that my card doesn’t have.

        #1 is what I’m actually using. I can deal with a random crash every other day to every other week or so.

        It really would not take much work for them to have a fourth version: One that’s not “supported-supported” but “we’re making sure this things runs”: Current rocm code, use kernels you write for other cards if they happen to work, generic code otherwise.

        Seriously, rocm is making me consider Intel cards. Price/performance is decent, plenty of VRAM (at least for its class), and apparently their API support is actually great. I don’t need cuda or rocm after all what I need is pytorch.