• 0 Posts
  • 48 Comments
Joined 2Y ago
cake
Cake day: Jun 26, 2023

help-circle
rss

That’s a great line of thought. Take an algorithm of “simulate a human brain”. Obviously that would break the paper’s argument, so you’d have to find why it doesn’t apply here to take the paper’s claims at face value.


There’s a number of major flaws with it:

  1. Assume the paper is completely true. It’s just proved the algorithmic complexity of it, but so what? What if the general case is NP-hard, but not in the case that we care about? That’s been true for other problems, why not this one?
  2. It proves something in a model. So what? Prove that the result applies to the real world
  3. Replace “human-like” with something trivial like “tree-like”. The paper then proves that we’ll never achieve tree-like intelligence?

IMO there’s also flaws in the argument itself, but those are more relevant


This is a silly argument:

[…] But even if we give the AGI-engineer every advantage, every benefit of the doubt, there is no conceivable method of achieving what big tech companies promise.’

That’s because cognition, or the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain. ‘If you have a conversation with someone, you might recall something you said fifteen minutes before. Or a year before. Or that someone else explained to you half your life ago. Any such knowledge might be crucial to advancing the conversation you’re having. People do that seamlessly’, explains van Rooij.

‘There will never be enough computing power to create AGI using machine learning that can do the same, because we’d run out of natural resources long before we’d even get close,’ Olivia Guest adds.

That’s as shortsighted as the “I think there is a world market for maybe five computers” quote, or the worry that NYC would be buried under mountains of horse poop before cars were invented. Maybe transformers aren’t the path to AGI, but there’s no reason to think we can’t achieve it in general unless you’re religious.

EDIT: From the paper:

The remainder of this paper will be an argument in ‘two acts’. In ACT 1: Releasing the Grip, we present a formalisation of the currently dominant approach to AI-as-engineering that claims that AGI is both inevitable and around the corner. We do this by introducing a thought experiment in which a fictive AI engineer, Dr. Ingenia, tries to construct an AGI under ideal conditions. For instance, Dr. Ingenia has perfect data, sampled from the true distribution, and they also have access to any conceivable ML method—including presently popular ‘deep learning’ based on artificial neural networks (ANNs) and any possible future methods—to train an algorithm (“an AI”). We then present a formal proof that the problem that Dr. Ingenia sets out to solve is intractable (formally, NP-hard; i.e. possible in principle but provably infeasible; see Section “Ingenia Theorem”). We also unpack how and why our proof is reconcilable with the apparent success of AI-as-engineering and show that the approach is a theoretical dead-end for cognitive science. In “ACT 2: Reclaiming the AI Vertex”, we explain how the original enthusiasm for using computers to understand the mind reflected many genuine benefits of AI for cognitive science, but also a fatal mistake. We conclude with ways in which ‘AI’ can be reclaimed for theory-building in cognitive science without falling into historical and present-day traps.

That’s a silly argument. It sets up a strawman and knocks it down. Just because you create a model and prove something in it, doesn’t mean it has any relationship to the real world.


Not sure if this my app or a Lemmy setting, but I only see it once, because it’s properly cross-posted


Man what a clusterfuck. Things still don’t really add up based on public info. I’m sure this will be the end of any real attempts at safeguards, but with the board acting the way it did, I don’t know that there would’ve been even without him returning. You know the board fucked up hard when some SV tech bro looks like the good guy.


But personally I’ll continue to advocate for technology which empowers people and culture, and not the other way around.

You won’t achieve this goal by aiding the gatekeepers. Stop helping them by trying to misapply copyright.

Any experienced programmer knows that GPL code is still subject to copyright […]

GPL is a clever hack of a bad system. It would be better if copyright didn’t exist, and I say that as someone that writes AGPL code.

I think you misunderstood what I meant. We should drop copyright, and pass a new law where if you use a model, or contribute to one, or a model is used against you, that model must be made available to you. Similar in spirit to the GPL, but not a reliant on an outdated system.

This would catch so many more use cases than trying to cram copyright where it doesn’t apply. No more:

  • Handful of already-rich companies building an AI moat that keeps newcomers out
  • Credit agencies assigning you a black box score that affects your entire life
  • Minorities being denied bail because of a black box model
  • Being put on a no-fly list with no way to know that you’re on it or why
  • Facebook experimenting on you to see if they can make you sad without your knowledge

Why should they? Copyright is an artificial restriction in the first place, that exists “To promote the Progress of Science and useful Arts” (in the US, but that’s where most companies are based). Why should we allow further legal restrictions that might strangle the progress of science and the useful arts?

What many people here want is for AI to help as many people as possible instead of just making some rich fucks richer. If we try to jam copyright into this, the rich fucks will just use it to build a moat and keep out the competition. What you should be advocating for instead is something like a mandatory GPL-style license, where anybody who uses the model or contributed training data to it has the right to a copy of it that they can run themselves. That would ensure that generative AI is democratizing. It also works for many different issues, such as biased models keeping minorities in jail longer.

tl;dr: Advocate for open models, not copyright


I wouldn’t be concerned about that, the mathematical models make assumptions that don’t hold in the real world. There’s still plenty of guidance in the loop from things such as humans up/downvoting, and people generating several to many pictures before selecting the best one to post. There’s also as you say lots of places with strong human curation, such as wikipedia or official documentation for various tools. There’s also the option of running better models as the tech progresses against old datasets.



The rules I’ve seen proposed would kill off innovation, and allow other countries to leapfrog whatever countries tried to implement them.

What rules do you think should be put in place?


Good to hear that people won’t be able to weaponize the legal system into holding back progress

EDIT, tl;dr from below: Advocate for open models, not copyright. It’s the wrong tool for this job


I understand you’re upset, but my sources have been quite clear and straightforward. You should actually read them, they’re quite nicely written.


So you admit that you were wrong?




Your concept of a chair is an abstract thought representation of a chair. An LLM has vectors that combine or decompose in some way to turn into the word “chair,” but are not a concept of a chair or an abstract representation of a chair. It is simply vectors and weights, unrelated to anything that actually exists.

Just so incredibly wrong. Fortunately, I’ll have save myself time arguing with such a misunderstanding. GPT-4 is here to help:

This reads like a misunderstanding of how LLMs (like GPT) work. Saying an LLM’s understanding is “simply vectors and weights” is like saying our brain’s understanding is just “neurons and synapses”. Both systems are trying to capture patterns in data. The LLM does have a representation of a chair, but it’s in its own encoded form, much like our neurons have encoded representations of concepts. Oversimplifying and saying it’s unrelated to anything that actually exists misses the point of how pattern recognition and information encoding works in both machines and humans.


You really, truly don’t understand what you’re talking about.

The vectors do not represent concepts. The vectors are math

If this community values good discussion, it should probably just ban statements that manage to be this wrong. It’s like when creationists say things like “if we came from monkeys why are they still around???”. The person has just demonstrated such a fundamental lack of understanding that it’s better to not engage.


processed into a script for ELIZA

That wouldn’t accomplish anything. I don’t know why the OP brought it up, and that subject should just get dropped. Also yes, you can use your intelligence to string together multiple tools to accomplish a particular task. Or you can use the intelligence of GPT-4 to accomplish the same task, without any other tools

LLMs lack the capability of understanding and comprehension

Also not true

states that it is not using an accepted definition of intelligence.

Nowhere does it state that. It says “There is no generally agreed upon definition of intelligence”. I’m not sure why you’re bringing up a physical good such as leather here. Two things: a) grab a microscope and inspect GPT-4. The comparison doesn’t make sense. b) “Is” should be banned, it encourages lazy thought and pointless discussion (Yes I’m guilty of it in this comment, but it helps when you really start asking what “is” means in context). You’re wandering into p-zombie territory, and my answer is that “is” means nothing. GPT-4 displays behaviors that are useful because of their intelligence, and nothing else matters from a practical standpoint.

it is clear that LLMs may be useful components in building actual general intelligence.

You’re staring the actual general intelligence in the face already, there’s no need to speculate about perhaps being components. There’s no reason right now to think that we need anything more than better compute. The actual general intelligence is yet a baby, and has experienced the world through the tiny funnel of human text, but that will change with hardware advances. Let’s see what happens with a few orders of magnitude more computing power.


For your edit: Yes, that’s what’s known as the context window limit. ChatGPT has an 8k token “memory” (for most people), and older entries are dropped. That’s not an inherent limitation of the approach, it’s just a way of keeping OpenAI’s bills lower.

Without an example I don’t think there’s anything to discuss. Here’s one trivial example though where I altered ChatGPT’s understanding of the world:

If I continued that conversation, ChatGPT would eventually forget that due to the aforementioned context window limit. For a more substantive way of altering an LLM’s understanding of the world, look at how OpenAI did RLHF to get ChatGPT to not say naughty things. That permanently altered the way GPT-4 responds, in a similar manner to having an angry nun rap your knuckles whenever you say something naughty.


LLMs can certainly do that, why are you asserting otherwise?

ChatGPT can do it for a single session, but not across multiple sessions. That’s not some inherent limitations to LLMs, that’s just because it’s convenient for OpenAI to do it that way. If we spun up a copy of a human from the same original state every time you wanted to ask it a question and then killed it after it was done responding, it similarly wouldn’t be able to change its behavior across questions.

Like, imagine we could do something like this. You could spin up a copy of that brain image, alter its understanding of the world, then spin up a fresh copy that doesn’t have that altered understanding. That’s essentially what we’re doing with LLMs today. But if you don’t spin up a fresh copy, it would retain its altered understanding.


Give Eliza equivalent compute time and functionality to interpret the data type and it probably could get something approaching a result.

Sorry, but this is simply incorrect. Do you know what Eliza is and how it works? It is categorically different from LLMs.

That’s not something that is seriously debated academically

This is also incorrect. I think the issue that many people have is that they hear “AI” and think “superintelligence”. What we have right now is indeed AI. It’s a primitive AI and certainly no superintelligence, but it’s AI nonetheless.

There is no known reason to think that the approach we’re taking now won’t eventually lead to superintelligence with better hardware. Maybe we will hit some limit that makes the hype die down, but there’s no reason to think that limit exists right now. Keep in mind that although this is apples vs oranges, GPT-4 is a fraction of the size of a human brain. Let’s see what happens when hardware advances give us a few more orders of magnitude. There’s already a huge, noticeable difference between GPT 3.5 and GPT 4.


What exactly do you think would happen if you could make an exact duplicate of a human and run it from the same state multiple times? They would generate exactly the same output every time. How could you possibly think differently without turning to human exceptionalism and believing in magic meat?


From scratch in the sense that it starts with random weights, and then experiences the world and builds a model of it through the medium of human text. That’s because text is computationally tractable for now, and has produced really impressive results. There’s no inherent need for text to be used though, similar models have been trained on time series data, and it will soon be feasible to hook up one of these models to a webcam and a body and let it experience the world on its own. No human intelligence required.

Also, your point is kind of silly. Human children learn language from older humans, and that process has been recursively happening for billions of years, all the way through the first forms of life. Do children not have intelligence? Or are you positing some magic moment in human evolution where intelligence just descended from the heavens and blessed us with it?


This is an unfortunate misunderstanding, one that’s all too common. I’ve also seen comments like “It’s no more intelligent than a dictionary”. Try asking Eliza to summarize a PDF for you, and then ask followup questions based on that summary. Then ask it to list a few flaws in the reasoning in the PDF. LLMs are so completely different from Eliza that I think you fundamentally misunderstand how they work. You should really read up on them.


That’s an architectural choice, there’s nothing inherent to the approach that would prevent that from happening.


How would creating a world model from scratch not involve intelligence?


That doesn’t mean it’s not intelligent. Humans can get broken in all sorts of ways. Are we not intelligent?


“Simulating conversations” to a good enough degree requires intelligence. Why are you drawing a distinction here?


That’s kind of silly semantics to quibble over. Would you tell a robot hunting you down “you’re only acting intelligent, you’re not actually intelligent!”?

People need to get over themselves as a species. Meat isn’t anything special, it turns out silicon can think too. Not in quite the same way, but it still thinks in ways that are useful to us.


Define intelligence. Your last line is kind of absurd. Why can’t intelligence be described by an algorithm?


The biggest problem is, who can tell if they’re telling the truth, and ensure that it continues to be true? Not the general public, that’s for sure. This is why we need regulation with auditors.


Also, it’s claimed that it’s uncloneable. We’ll see how well that actually stands up to a counterfeit market with lots of money to throw at it


Never going to happen. Even if something was signed, everyone would just develop the tech secretly. It’s not like nukes where you need large, special equipment like refineries that investigators can track, and special ores. If you’ve got some nerds with nice graphics cards, you’re off to a good start. Getting more advanced is easy if you produce your own chips, and looks the same as regular consumer demand to anyone suspicious.


Have you tried including the docs in the prompt as well? I wouldn’t have guessed that S__ is a variable in your example, but I would’ve known that after reading the docs. I’d expect ChatGPT to at least do OK if it had access to the docs.



I like Python and use it a lot, but Dhall has several guarantees that Python doesn’t. If you hand someone clever a Python interpreter, they can and will find a way to run rm -rf / or what-have-you, no matter what safeguards you put in place. Dhall also guarantees that it will terminate, which is pretty useful if you’re embedding it in something else. In Python you could pretty easily accidentally or purposefully get into a while True loop.



They already were the first? I still remember when I upgraded Firefox on my phone and all of the extensions were gone. It’s nice that they’re finally bringing them back after all these years, but it’s just a return to the way things used to be.

EDIT: Headline here was changed from the original article, which doesn’t claim “first”, just “only”.


Here’s a better link that doesn’t even need an archive.is link:

https://pluralistic.net/2023/07/24/rent-to-pwn/

Kind of ironic to talk about enshittification on medium, that’s currently going through the process of enshittification. At least he’s got his writing on his regular site as well.


I’m a little unclear on why the license change would help monetization. Wouldn’t an MIT license allow big companies to just use it and not pay?