- 0 Posts
- 36 Comments
300i https://www.bilibili.com/video/BV15NKJzVEuU/
M4 https://github.com/itsmostafa/inference-speed-tests
It’s comparable to an M4, maybe a single order of magnitude faster than a ~1000 euro 9960X, at most, not multiple. And if we’re considering the option of buying used, since this is a brand new product and less available in western markets, the CPU-only option with an EPYC and more RAM will probably be a better local LLM computer for the cost of 2 of these and a basic computer.
That’s still faster than your expensive RGB XMP gamer RAM DDR5 CPU-only system, and you can depending on what you’re running saturate the buses independently, doubling the speed and matching a 5060 or there about. I disagree that you can categorise the speed as negating the capacity, as they’re different axis. You can run bigger models on this. Smaller models will run faster on a cheaper Nvidia. You aren’t getting 5080 performance and 6x the RAM for the same price, but I don’t think that’s a realistic ask either.
You can run llama.cpp on CPU. LLM inference doesn’t need any features only GPUs typically have, that’s why it’s possible to make even simpler NPUs that can still run the same models. GPUs just tend to be faster. If the GPU in question is not faster than an equally priced CPU, you should use the CPU (better OS support).
Edit: I looked at a bunch real-world prices and benchmarks, and read the manual from Huawei and my new conclusion is that this is the best product on the market if you want to run a model at modest speed that doesn’t fit in 32GB but does in 96GB. Running multiple in parallel seems to range from unsupported to working poorly, so you should only expect to use one.
Original rest of the comment, made with the assumption that this was slower than it is, but had better drivers:
The only benefit to this product over CPU is that you can slot multiple of them and they parallelise without needing to coordinate anything with the OS. It’s also a very linear cost increase as long as you have the PCIe lanes for it. For a home user with enough money for one or two of these, they would be much better served spending the money on a fast CPU and 256GB system RAM.
If not AI, then what use case do you think this serves better?
CUDA is not equivalent to AI training. Nvida offers useful developer tools for using their hardware, but you don’t have to use them. You can train on any GPU or even CPU. The projects you’ve looked at (?) just chose to use CUDA because it was the best fit for what hardware they had on hand, and were able to tolerate the vendor lock-in.
GPUs being able to spy on you is a problem. Linux and OSs should work on ways to minimise this possibility. But specifically worrying about them ratting you out to the CPC is in my opinion much less serious than spying on you for commercial interests. Ultimately it’s possible to avoid visiting China if you aren’t Chinese. Including Chinese websites.
You could watch two YouTube films at once. (No but seriously 2Mb/s is too low even for just YouTube. YouTube recommends 20Mb/s. And that’s probably assuming 30hz. So you probably actually want at double or more. https://support.google.com/youtube/answer/78358)
It’s not zero, each fiefdom has very little power to keep users. As it is right now, a user unhappy with their instance culture or laws can move to another instance. Comparing it to moving in real life, in real life you have a lifetime worth of things that tie you to your fiefdom. Comparing it to well established and centralised social media, then those fiefdoms still have a lot of power over you.
Your social network can’t come with you, they’re SSO providers, they’re tracking and human-verification providers, they have high quality exclusive content, they’re sometimes the only channel for interacting with some third parties you have to interact with (Government, utility company, etc).
War crime speedrunners doing something shouldn’t make you think it isn’t illegal. Booby-traps are illegal. https://www.icrc.org/sites/default/files/external/doc/en/assets/files/other/icrc_002_0811.pdf
ITT people claim that a Google VPN is a bad product for all use cases because Google is not a privacy-respecting company. This ignores all non-privacy use cases for using a VPN.
And even for privacy, this would’ve been a product where the vendors interest in protecting your privacy and your interest in protecting your privacy aligned in the case where you were not hiding from Google. For example if you used a Chromebook laptop, used the Google Chrome browser, or used Google services like Google Search and Google YouTube, then Google would already know everything about you. You can’t hide your activity from them, but they can help you hide it from others.
Similar situations exist for other privacy disrespecting companies like Microsoft and Apple, where a user might reasonably want to hide from everyone other than their vendor of choice, whose product they consider good enough to allow them to see their computer activity as part of their payment. If you already subscribe to one privacy disrespecting vendor, it makes the most sense to go all-in.
If a device makes an encrypted connection to a server the device makers own, there’s nothing further you can gleam from studying the DNS lookups. They can route traffic through the first server, and they can resolve any IPs through the first server. And since you insist the person you’re replying to doesn’t know what DNS is because they said it’s encrypted, I feel you might also not know that DNS can be encrypted. In that case, the network owner can see that a device makes a connection to the nameserver, but they can’t see which addresses the nameserver was asked to resolve. And similarly, the device can refuse a connection to the wrong nameserver.
Medal of Honour VR is 180GB, or about 24 minutes if your download speed never falls below the advertised speed. And that’s even a ‘nice’ example, some of the super large games like Call of Duty and MS Flight Sim update between every time you find time to play and make you download hundreds of gigabytes.
And to say nothing of node_modules.
I’m pretty sure the Tiananmen square massacre was a bigger crime.
Even the propaganda version pales compared to the shit they forgive.
https://en.wikipedia.org/wiki/Tulsa_race_massacre
https://www.vox.com/the-highlight/2019/8/8/20747198/philadelphia-bombing-1985-move
genocide of the Uighur people
Even the propaganda version pales compared to the shit they forgive.
https://www.mintpressnews.com/15-years-on-the-staggering-death-toll-in-iraq-keeps-climbing/239055/
https://www.aljazeera.com/news/longform/2023/10/9/israel-hamas-war-in-maps-and-charts-live-tracker
their oppression of Hong Kong
Even the propaganda version pales compared to the shit they forgive.
https://www.theguardian.com/us-news/2023/jan/06/us-police-killings-record-number-2022
https://en.wikipedia.org/wiki/Incarceration_in_the_United_States
their attempts to steal Taiwan’s sovereignty
Yeah alright go home mate.
Look up China Ghost towns. Some insightful shit.
No it isn’t. This is the insightful western coverage.
https://edition.cnn.com/travel/article/chongqing-china-metro-station-nowhere/index.html
But as it turns out, predictably, if you check now the city has reached it and it’s a completely unremarkable piece of infrastructure.
Next they’ll write articles about the next stop in the line. Maybe western media can’t wrap their heads around purpose oriented modes of production.
Prolewiki is the Lemmy aligned wiki, run and maintained by Lemmy/grad users and staff. Greenland is not considered global north because they are a colony of Denmark. Geopolitically their interests don’t align, and for the sake of economic classification, Greenland is exploited and Denmark is doing the exploitation. The map uploader didn’t decide who is or isn’t in the global north, they’re repeating the classification used by the United Nations Conference on Trade and Development https://unctadstat.unctad.org/EN/Classifications.html.
The USA is not a democracy. Not a weak or flawed democracy, as those require some democratic input. There is no democratic consideration done by the ruling elite when dictating their rule. https://www.cambridge.org/core/journals/perspectives-on-politics/article/testing-theories-of-american-politics-elites-interest-groups-and-average-citizens/62327F513959D0A304D4893B382B992B The extent of the democratic process in the USA is voting blue or red. It’s a meaningless and infrequent vote.
Don’t you have that backwards? Without TSMC’s outstanding technology, the island’s value decreases, both for China and for the USA. Conventional wisdom is that reduced tensions also reduces the risk of war.