- 1 Post
- 81 Comments
in any way shape or form
I’d normally accept the challenge if you didn’t add that. You did though and it, namely a system (arguably intelligent) made an image, several images in fact. The fact that we dislike or like the aesthetics of it or that the way it was done (without prompt) is different than how it currently is remains irrelevant according to your own criteria, which is none. Anyway my point with AARON isn’t about this piece of work specifically, rather that there is prior work, and this one is JUST an example. Consequently the starting point is wrong.
Anyway… even if you did question this, I argued for more, showing that I did try numerous (more than 50) models, including very current ones. It even makes me curious if you, who is arguing for the capabilities and their progress, if you tried more models than I did and if so where can I read about it and what you learned about such attempts.
Language models on their own do indeed have lots of limitations, however there is a lot of potential in coupling them with other types of expert systems.
Absolutely, I even have a dedicated section “Trying to insure combinatoriality/compositionality” in my notes on the topic https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence
Still, while keeping this in mind we also must remain mindful of what each system can actually do, not conflate with what we WANT it do yet it can not do yet, and might never will.
Image gen did not exist in any way shape or form before.
Typical trope while promoting a “new” technology. A classic example is 1972’s AARON https://en.wikipedia.org/wiki/AARON which, despite not being based on LLM (so not CLIP) nor even ML is still creating novel images. So… image generation has been existing since at least the 70s, more than half a century ago. I’m not saying it’s equivalent to the implementation since DALLE (it is not) but to somehow ignore the history of a research field is not doing it justice. I have also been modding https://old.reddit.com/r/computationalcrea/ since 9 years, so that’s before OpenAI was even founded, just to give some historical context. Also 2015 means 6 years before CLIP. Again, not to say this is the equivalent, solely that generative AI has a long history and thus setting back dates to grand moments like AlphaGo or DeepBlue (and on this topic I can recommend Rematch from Arte) … are very much arbitrary and in no way help to predict what’s yet to come, both in terms of what’s achievable but even the pace.
Anyway, I don’t know what you actually tried but here is a short list of the 58 (as of today) models I tried https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence and that’s excluding the popular ones, e.g. ChatGPT, Mistal LeChat, DALLE, etc which I also tried.
I might be making “the same mistake” but, as I hope you can see, I do keep on trying what I believe is the state of the art of a pretty much weekly basis.
What an impressive waste of resources. It’s portrayed as THE most important race and yet what has been delivered so far?
Slightly better TTS or OCR, photography manipulation that is commercially unusable because sources can’t be traced, summarization that can introduce hallucinations, … sure all of that is interesting in terms of academic research, with potentially some use cases… but it’s not as if it didn’t exist before at nearly the same quality for a fraction of the resources.
It’s a competitions where “winners” actually don’t win much, quite a ridiculous situation to be in.
Meanwhile, I deleted my Windows partition (even though I paid for it, damn OEM tricks) and feel better for it https://lemmy.world/comment/12818969
So if you are genuinely worried about this, don’t.
First because, as numerous persons already clarified, researchers here are breaking deprecated cryptography.
It’s a bit like taking toothpicks and opening a lock while the locks used in your modern car is very different. Yes, it IS actually interesting but the same technique does not apply in practice, only in principle.
Second because IF in principle there IS a path to radically grow in power, there are already modern cryptography techniques which are resistant to scaling the power of quantum computers. Consequently it is NOT just about small the key is, but also HOW that key is made, what are the mathematical foundations on which a key is made, and can be broken.
Anyway for a few years now there has been research, roughly matching the interest in quantum computers, to what is called post-quantum encryption, or quantum resistant encryption. Basically the goal of the research is to find new ways to make keys that are very cheap to generate and verify, literally with something as cheap and non powerful as the chip in your credit card, BUT practically impossible to “crack” for a computer, even a quantum computer, even a powerful one. The result of that on-going research are schemes like Kyber, FALCON, SPHINCS+, etc which answer such requirements. Organizations like NIST in the US verify that the schemes are actually without flaws and the do recommendations.
So… all this to say that a powerful quantum computer is still not something that breaks encryption overall.
If you are worried TODAY, you can even “play” with implementations like https://github.com/open-quantum-safe/oqs-demos and setup a server, e.g Apache, and a client, e.g Chromium, so that they can communicate using such schemes.
Now practically speaking if you are not technically inclined or just want to bother, you can “just” use modern software, e.g Signal, which last year https://signal.org/blog/pqxdh/ announced that they are doing just that on your behalf.
You can finally expect all actors, e.g hosts like Lemmy, browsers like Firefox, that you use daily to access content to gradually both integrate post-quantum encryption but also gradually deprecate older, and thus risky, schemes. In fact if you try to connect today to old hardware via e.g ssh you might find yourself forced to accept older encryption. This very action is interesting because it does show that over the years encryption changes, old schemes get deprecated and replace.
TL;DR: cool, not worried though even with a properly powerful quantum computer because post-quantum encryption is being rolled out already.
What this show is a total lack of originality.
AI is not new. Open-source is not new. Putting two well known concepts together wasn’t new either because… AI has historically been open. A lot of the cutting edge research is done in public laboratories, with public funding, and is published in journals (sadly often behind paywall but still).
So the name and the concept are both unoriginal.
A lot of the popularity gained from OpenAI by using a chatbot is not new either. Relying on always larger dataset and benefiting from Moore’s law is not new either.
So I’m not standing on any side, neither this person nor the corporation.
I find that claiming to be “owning” common ideas is destructive for most.
To consider related to evaluation of said hardware, cf https://lemmy.ml/post/20849010/14003579
If you were implying that I said being funded by Alphabet/Google was a good thing then let me be explicit, I did NOT say that nor believe it to be the case. Now, once again, cf my actual comment about pragmatic better alternative we can rely on and support today. If you meant to suggest better and are supporting that, please do share.
I thought saying
contribute however they can up to their own capabilities
was actually very clear but seems I wasn’t clear enough so that means… literally doing ANYTHING except only criticizing. That can mean being an open-source developer, yes, but that can also means translation, giving literally 1 cent, etc. It means doing anything at all that would not ONLY be saying “this is good, but it’s not good enough” without doing actually a single thing to change, especially while actually using another free of charge browser that is funded by advertisement. Honestly if that’s not clear enough I’m not sure what would be … but please, do ask again I will genuinely try to be clearer.
I hope everybody criticizing the move either do not use products from Mozilla or, if they do, contribute however they can up to their own capabilities. If you don’t, if you ONLY criticize, yet use Firefox (or a derivative, e.g. LibreWolf) or arguably worst use something fueled by ads (e.g. Chromium based browsers) then you are unfortunately contributing precisely to the model you are rejecting.
As per usual, in order to understand what it means we need to see :
- performance benchmark (A100 level? H100? B100? GB200 setups?)
- energy consumption (A100 performance level and H100 lower watt? the other way around?)
- networking scalability (how many cards cards can be interconnected for distributed compute? NVLink equivalents?)
- software stack (e.g can it run CUDA and if not what alternatives can be used?)
- yield (how many die are usable, i.e. can it be commercially viable or is it R&D still?)
- price (which regardless of possible subsidies would come from yield)
- volume (how many cards can actually be bought, also dependent on yield)
Still interesting to read after announcements, as per usual, and especially who will actually manufacture them at scale (SMIC? TSMC?).
PS: full disclosure, I still believe self-hosting AI is interesting, cf my notes on it https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence but that doesn’t mean AGI can be reached, even less that it’d be “soon”. IMHO AI itself as a research field is interesting enough that it doesn’t need grandiose claims, especially not ones leading to learned helplessness.
I haven’t seriously read the article for now unfortunately (deadline tomorrow) but if there is one thing that I believe is reliable, it’s computational complexity. It’s one thing to be creative, ingenious, find new algorithms and build very efficient processors and datacenters to make things extremely efficient, letting us computer things increasingly complex. It’s another though to “break” free of complexity. It’s just, as far as we currently know, is impossible. What is counter intuitive is that seemingly “simple” behaviors scale terribly, in the sense that one can compute few iterations alone, or with a computer, or with a very powerful set of computers… or with every single existing computers… only to realize that the next iteration of that well understood problem would still NOT be solvable with every computer (even quantum ones) ever made or that could be made based on resources available in say our solar system.
So… yes, it is a “stretch”, maybe even counter intuitive, to go as far as saying it is not and NEVER will be possible to realize AGI, but that’s what their paper claims. It’s a least interesting precisely because it goes against the trend we hear CONSTANTLY pretty much everywhere else.
It’s a classic BigTech marketing trick. They are the only one able to build “it” and it doesn’t matter if we like “it” or not because “it” is coming.
I believed in this BS for longer than I care to admit. I though “Oh yes, that’s progress” so of course it will come, it must come. It’s also very complex so nobody else but such large entities with so much resources can do it.
Then… you start to encounter more and more vaporware. Grandiose announcement and when you try the result you can’t help but be disappointed. You compare what was promised with the result, think it’s cool, kind of, shrug, and move on with your day. It happens again, and again. Sometimes you see something really impressive, you dig and realize it’s a partnership with a startup or a university doing the actual research. The more time passes, the more you realize that all BigTech do it, across technologies. You also realize that your artist friend did something just as cool and as open-source. Their version does not look polished but it works. You find a KickStarter about a product that is genuinely novel (say Oculus DK1) and has no link (initially) with BigTech…
You finally realize, year after year, you have been brain washed to believe only BigTech can do it. It’s false. It’s self serving BS to both prevent you from building and depend on them.
You can build, we can build and we can build better.
Can we build AGI? Maybe. Can they build AGI? They sure want us to believe it but they have lied through their teeth before so until they do deliver, they can NOT.
TL;DR: BigTech is not as powerful as they claim to be and they benefit from the hype, in this AI hype cycle and otherwise. They can’t be trusted.
I… agree but isn’t then contradicting your previous point that innovation will come from large companies if they only try to secure monopolies rather than genuinely innovate? I don’t understand from that perspective who is left to innovate if it’s neither research (focusing on publishing, even though having the actual novel insight and verifying that it does work), not the large companies… and startups don’t get the funding either. Sorry if you mentioned it but I’m now confused as what is left.
They just provide the data. They can question the methodology or even provide another report with a different methodology but if the data is correct (namely no fabricated) then it’s not up to them to see how it’s being used. The user can decide how they define startup, i.e which minimum size, funding types, funding rounds, etc. Sharing their opinion on the startup landscape is unprofessional IMHO. They are of course free to do so but to me it doesn’t question the validity of the original report.
Please clarify, as I asked in https://lemmy.ml/post/20245112/13688624 I don’t see how that’s relevant. They are sharing opinions from startup CEO or numbers that are about large “old” (much earlier than the boom, e.g Ant, Shein, ByteDance). That’s certainly interesting but does not contradict figures from the article.
Research happens through university, absolutely, and selling products at scale through large companies, but that’s not innovation. Innovation is bringing new products, that is often the result of research yes, to market. Large companies tends to be innovative by buying startups. If there are no startups coming from research coming from universities to buy, I don’t see how large companies, often stuck in the “innovator dilemma”, will be able to innovate.
Thanks for linking to criticism but can you highlight which numbers are off? I can see things about ByteDance, Ant group, Shein but that’s irrelevant as it’s not about the number of past success, solely about the number of new funded startups. Same as the CEO of ITJUZI sharing his opinion, that’s not a number.
Edit: looks totally off, e.g “restaurants, in a single location, such as one city, you could immediately tell that there were large numbers of new companies.” as the article is about funding, not a loan from the bank at the corner of the street.
Thanks for the in depth clarification and sharing your perspective.
this is a good development
Keeping finance in check is indeed important so I also think it’s good.
What about the number of funded startups though and the innovative products they would normally provide customers? Do you believe the measures taken will only weed out bad financiers or will it also have, as a side effect, to bring less products and solutions out? Does it mean research will remain academic but won’t necessarily be commercialized or even scaled? If you believe it will still happen, how? Through state or regional funding and if so can you please share such examples that grew for the last 5 years?
Founded in 2019, right after the peak according to the very graph I highlighted.
Neither I nor the article is saying there are no more startups nor innovation from China. What the article is saying is that it’s radically less than 7 years ago. You can still list few amazing Chinese startup from 2023 or 2024 and it would still not make the article “nonsense”.
Care to unfold a bit more what’s hilarious? Which metrics from the article are wrong or irrelevant for example? You might disagree with the conclusion, and maybe rightly so, but are you saying the data itself, e.g number of companies funded is false? Or it does not matter and something else could help better understand the situation?
- @[email protected] to
- •
- www.ft.com
- •
- 4M
- •
I love how you just assumed that I’m Chinese
I bet most people reading “I live in Canada, my family moved here back when I was still in school. I’d like to move to China one day” would assume the same, especially “back” as I understood, but my English isn’t perfect, return FROM China. It has nothing to do with “race”, culture, politics or economy.
Anyway, this makes it even more interesting, have you already been to China at all then? Worked there? Because I did but I don’t want to make assumptions so again feel free to clarify.
PS: also want to make it clear, I didn’t say nor assumed that you were Chinese, but of Chinese heritage, a bit different.
I was going to ask how come you (OP) posts regularly such posts on the “Chip War” with recurrent arguments claiming that China is catching and will surpass the “West” soon, e.g https://lemmy.ml/post/19683899/13322374 . It seems from reading your post history https://lemmy.ml/post/19683748/13336881 that you are Canadian but would like to go back to China to live and work there.
So to be direct, are you sharing those articles in the hope of adjusting a biased perspective from the West on its control of the semiconductor industry? Are you yourself hoping to, maybe if your personal circumstances were different, work in that industry with a Chinese patriotic motivation? Feel free to expand a bit more on your motivations more broadly if you feel like it could help myself and others understand your viewpoint and goals.
Also interesting to note “The focus on mature nodes also positions Chinese companies to dominate markets where advanced nodes are not necessary, such as in automotive and industrial applications.” which is indeed very viable. Namely they focus on “old” processors used in “simpler” situations. The machinery from ASML to make such chips is actually purchasable (unlike the latest ones). China is already positioned on the lower end of the market.
Still, even though going from the production of older chips is a step to higher end one, it is not the same, especially when machinery to do so can’t be purchased.
“Increased Spending on Equipment” is not evidence of progress. In fact there are numerous examples in the past of fraud, e.g https://www.ft.com/content/1e3fe107-1b6e-43dd-8f04-e3c88502c36b (cf ““Big Fund”, which raised $51bn in its last two funding rounds.” 2 years ago, setup 10 years ago)
China is indeed pouring money on the problem and they are making significant progress. Yet it’s not competitive in terms of performance and, much harder to evaluate, it seems not to be competitive in terms of economics. To make a processor a lot of low quality ones are discarded, leading to the idea of “yield” (cf https://en.wikipedia.org/wiki/Semiconductor_device_fabrication#Device_yield ). So, even if one CPU/GPU/TPU is genuinely produced, in “full autonomy” (so without e.g ASML unique machinery) and it actually on independent benchmarks comparable in terms of performance to the state of the art produced outside, it’s still impossible to evaluate how viable the production process is. Maybe there yield is very high and thus producing those chips is efficient and thus cheap, maybe the chip used in the benchmark is the single existing one and thus is prohibitively expensive.
I recommend the 2022 Chip War https://en.wikipedia.org/wiki/Chip_War:_The_Fight_for_the_World's_Most_Critical_Technologyon the topic, it is quite interesting.
Tinkered with a https://www.banana-pi.org/en/banana-pi-sbcs/175.html recently and… it’s really cool to have that at home, like, it works! In itself that’s quite a feat. Yet… to become actually usable due to “just” raw power but also to be economically comparable to mass produced other architectures from other manufacturers is indeed quite some road ahead.
Because it’s a race I wish such articles would bring forward comparison points :
- benchmark allowing to pinpoint past equivalents
- when was the first equivalent actually put on sale (and where)
- what’s the volume produced, even if only an order of magnitude
- inflation adjusted (as it might be several year gap) price comparison
otherwise it mostly feels like tech-propaganda pieces.
Why is anyone surprised that the country […] that has historically dominated the Top500 list, has the fastest supercomputers?
Because since then bans have been issued, specifically preventing the purchase of the “best” hardware, and that said country does not produce such hardware internally (e.g NVIDIA and AMD top of the line, and upstream with ASML). That’s what why it is surprising, precisely because the situation has changed, cf e.g https://www.foreignaffairs.com/china/limits-china-chip-ban leading to possibly counter intuitive effects.
I imagine most people would like to better understand what hardware is being used, especially chips and to know where they come from, i.e
- are they still somehow top of the line the country can’t have through normal channels
- somehow an order of magnitude of older chips they can legally purchase, so wasting quite a bit of energy but still similar results
- the most unexpected using own hardware that is believed not to be available at scale
So yes it’s arguably surprising because the situation is not as it was just a couple of years ago.
I don’t get the hype around LLM, it is a terrible way to search
I’ll be playing devil’s advocate here just for a moment (despite the huge ecological, moral, political and economical costs) :
- what LLM does provide is a looser linguistic interface. That means instead of searching for exact words, one can approximately search for the “idea”. That means instead of hitting just the right keywords that an expert might know, one can describe a partial solution, a very rough guess of what the problem might be, and possibly get a realistic sounding answer. It might be wrong yet it might still be a step in the right direction.
So… yes I also don’t think the hype is justified but IMHO it’s quite clear that providing a solution that makes an interface easier to get some OK-looking result would appeal to masses. That means a LOT of people get their hopes up about potential empowerment and a few people ride that bubble making money on promises.
PS: for people interested in the topic but wanting to avoid the generative aspect I believe https://en.wikipedia.org/wiki/Semantic_search is a good starting point.
“brute force is brute force” what a strange thing to say, it precisely is NOT.
If you have a lot of processors but they are poorly linked together, i.e low bandwidth, then they are NOT more powerful. That’s why e.g NVIDIA is selling InfiniBand and other very expensive solutions to datacenter.
Sure a supercomputer might have more CPU/GPU/etc than another but it doesn’t make it automatically more powerful, in term of what can actually be computed in comparable time (and arguably energy consumption).
That being said, China might be secretly #1 on TOP500 but until evidence of it is provided, I’m not sure what’s the point of such speculation is.
Well that’s one position, another is to say AI, being developed currently, is :
- not working due to hallucinations
- wasteful in terms of resources
- creates problematic behaviors in terms of privacy
- creates more inequality
and other problems and is thus in most cases (say outside of e.g numerical optimization as already done at e.g DoE, so in the “traditional” sense of AI, not the LLM craze) better be entirely ignored.
Edit : what I mean is that the argument of inevitability itself is dangerous, often abused.
Which… is “funny” because even though it is a genuine arm race where 2 powerful nations are competing… it’s a pointless one.
Sure, we do get slightly better STT, TTS, some “generation” of “stuff”, as in human sounding text use for spam and scam, images and now videos without attribution, but the actual hard stuff? Not a lot of real change there.
Anyway, interesting to see how the chips war unfold. For now despite the grand claim though from both :
neither have produced anything genuinely positive IMHO.