@ticoombs@reddthat.com

I tried to upload an image, but got the error IO error: Os { code: 28, kind: StorageFull, message: "No space left on device" }

I’m assuming it’s a different storage area than the main database, otherwise this will fail too…

  • TiffMAEnglish
    2·
    4 months ago

    What! We use a s3-like service for that :( Investigating

    • TiffMAEnglish
      4·
      4 months ago

      Turns out there is a bug with pictrs which doesn’t clean up temp files. So we had ~88GB of temp data!

      And since we moved to a dedicated pictrs server (so the main Lemmy procesees & DB could benefit) I didnt add that server to our monitoring 🫢

      It’s fixed now and I’ll be adding disk monitoring once I get a moment away from $job

      • TiffMAEnglish
        2·
        4 months ago

        We are on 0.4.7 and 0.5.0 fixes this bug (apparently).
        I was waiting for the “dust to settle” as it were on the 0.5 release but will schedule the upgrade this weekend sometime.

      • LocustOfControlOPEnglish
        2·
        4 months ago

        That’s how it goes, if this was a universe where you’d monitored that, the bug wouldn’t exist. Thanks for fixing 👍