Hacker Newsnew | past | comments | ask | show | jobs | submit | rattt's commentslogin

Shouldn't jdupes like tools already work now that ZFS has reflink copy support?


No, because none of these tools use copy_file_range. Because copy_file_range doesn't guarantee deduplication or anything. It is meant to copy data. So you could just end up copying data, when you aren't even trying to copy anything at all.

All modern tools use FIDEDUPRANGE, which is an ioctl meant for explicitly this use case - telling the FS that two files have bytes that should be shared.

Under the covers, the FS does block cloning or whatever to make it happen.

Nothing is copied.

ZFS does support FICLONERANGE, which is the same as FIDEDUPRANGE but it does not verify the contents are the same prior to cloning.

Both are atomic WRT to concurrent writes, but for FIDEDUPRANGE that means the compare is part of the atomicness. So you don't have to do any locking.

If you used FICLONERANGE, you'd need to lock the two file ranges, verify, clone, unlock

FIDEDUPRANGE does this for you.

So it is possible, with no changes to ZFS, to modify dedup tools to work on ZFS by changing them to use FICLONERANGE + locking if FIDEDUPRANGE does not exist.


Microsoft's now shut down Mixer (formerly Beam.pro) primarly used WebRTC and had streams with up to 100k viewers at some point all via WebRTC, but can't find much info about it anymore now sadly.


*Up to 3x of average monthly data stored, then $0.01/GB for additional egress.

Or free if you go through Cloudflare since they have the bandwidth aliance.


Presumably referring to iPhone 14+ satelite emergency texting https://support.apple.com/en-us/HT213426


I assume so, just wondering what would've happened if they had the phone, how far were they from the rescue service, what was the circumstances etc.


Yeah it supports generic tcp forwarding, I only tried it once when it released but worked without issues. Needs cloudflared on the client as well but so does the method in the blogpost so should be about the same:

https://developers.cloudflare.com/cloudflare-one/application...


> You could see that by how little RAM ended up being used in llama.cpp when they moved to mmaping the model.

From what I've read that was just an error in reading memory consumption after switching to the mmap version and not more memory efficient at all in the end.


Not exactly. It's that the model is loading less stuff out of the mmap'ed weights that you would expect.

The author of the mmap patch chimes in here:

https://news.ycombinator.com/item?id=35393615


Ohh wow that actually worked, that's awesome: https://i.imgur.com/4tYEphX.png

Tested on Intel MacOS 12.5 PC with AMD 8GB RX 580 GPU, about 28 secs for 20 steps, surprisingly fast too. I did have to go to chrome://flags and enable "Unsafe WebGPU" even on Chrome Canary (113.0.5656.0) before it would work, otherwise I just got "no adapter" errors.


Yep it surprisingly works on my AMDGPU too, even if it’s designed only for M1/M2


You don't need a HF account to download the checkpoint, can be downloaded straight from the website/browser, direct url: https://huggingface.co/openjourney/openjourney/resolve/main/...


Is it possible to download with curl or git lfs (or other "free" command line tool) with no login? I couldn't find a way to do that with the original sd checkpoints.


Yes works with anything now, they removed the manual accepting of the terms and auth requirement some months after release.


I will try it, thanks!


Something like HyperLogLog might be fine assuming you don't need perfect accuracy.

https://en.wikipedia.org/wiki/HyperLogLog


They could still be doing that through an Apple hosted proxy though? I believe services like discord and gmail webclient do that for links to not leak IPs.


Is it really better for SMS content to be sent to a server to achieve that? I like my SMS content to only be known to my telecom provider and my device.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: