What’s even funnier is that meta literally spent millions on each one of them.

It’s a paper about an open source model discussing a new algorithm which essentially builds privacy into the model as part of training. Attempts to add privacy during the final tuning stage generally fail because the model has already memorized sensitive information during its initial learning phase. This approach mathematically limits how much any single document can influence the final model, and prevents the model from reciting verbatim snippets of private data while still allowing it to learn general patterns and knowledge.

It’s really spurring Chinese companies to make LLMs that don’t need a lake of water to tell you how many r’s there are in strawberry. 🤣

I think slop should really be defined by the purpose of the art rather than the medium. Any piece of advertisement is inherently far more slop than a piece of genAI art somebody made because they just had an idea in their had they wanted to express.

would just be a dumb terminal that’s locked down

Right, somehow he thought that fighting a trade war with the whole world at once would work. I guess his whole admin drank the kool-aid about the US being an indispensable country.

oh for sure, I think that a small model that’s optimized towards parsing human language and inferring what the user wants coupled with a logic engine could be an extremely powerful tool. Trying to make LLMs do stuff like math or formal reasoning is trying to ram a square peg into a round hole. It doesn’t make any sense to do this because we already have tools that are really good for that sort of stuff. What we don’t have are tools that can easily infer the intent from natural language, and that’s the gap LLMs can fill.

yeah, mcp is really a giant hack and it’s the most inefficient approach you can think of

They’re absolutely not useless. The trick is to figure out how to use them effectively. For a concrete example, here’s a project I made to implement the idea of using a REPL as a context that I read in a paper recently. The premise there is that even small models are fairly competent at writing individual functions, and dealing with a small context of a few lines. So, instead of feeding large documents into these models which break them, you can instead provide them with an API to interrogate the document by writing code. And sure enough, the idea works. I managed to get qwen2.5-coder:7b, which is a tiny model to reliably search through a large document that it would no hope of figuring out on its own. Here’s what a run of it looks like:

npx tsx src/index.ts \
  "use javascript to write code to find the total sales amount across all regions?" \
  ./test-fixtures/scattered-data.txt \
  --verbose

  Configuration:
  Provider: ollama
  Model: qwen2.5-coder:7b
  Max turns: 10
  Timeout: 30000ms

[RLM] Loaded document: 4,706 characters
[RLM] Sandbox created (maxSubCalls: 10, timeout: 30000ms)

──────────────────────────────────────────────────
[Turn 1/10] Querying LLM...
[Turn 1] Executing code:
```javascript
// Step 1: Search for data related to sales
const hits = grep("sales");  // Returns array of {match, line, lineNum}
console.log(JSON.stringify(hits, null, 2));

// Step 2: Process results - extract sales amounts from each matching line
let totalSales = 0;
for (const hit of hits) {
    const saleAmount = parseFloat(hit.line.match(/\d+(\.\d{1,2})?/)[0]);
    if (!isNaN(saleAmount)) {
        totalSales += saleAmount;
    }
}

// Step 3: Store the result in memory
memory.push({ key: "total_sales", value: totalSales });

// Final output
console.log("Total sales across all regions:", totalSales);
```
[Turn 1] Console output:
  [
  ...
  {
    "match": "Sales",
    "line": "- Sales Data: Revenue in USD before taxes",
    "lineNum": 122,
    "index": 4357,
    "groups": []
  }
]
[Turn 1] Error: TypeError: Cannot read properties of null (reading '0')

──────────────────────────────────────────────────
[Turn 2/10] Querying LLM...
[Turn 2] Executing code:
```javascript
// Extract sales data from the logs
const salesData = [
    { region: "NORTH", amount: 2340000 },
    { region: "SOUTH", amount: 3120000 },
    { region: "EAST", amount: 2890000 },
    { region: "WEST", amount: 2670000 },
    { region: "CENTRAL", amount: 1980000 }
];

// Calculate total sales
const totalSales = salesData.reduce((acc, curr) => acc + curr.amount, 0);

// Output the total sales
console.log("Total Sales:", totalSales);
```
[Turn 2] Console output:
  Total Sales: 13000000

──────────────────────────────────────────────────
[Turn 3/10] Querying LLM...
[Turn 3] Executing code:
```javascript
// Output the total sales
console.log("Total Sales:", totalSales);
```
[Turn 3] Console output:
  Total Sales: 13000000
[Turn 3] Final answer found after code execution

[RLM] Sandbox disposed
The total sales are 13000000.

so in just 3 calls with very small contexts, it managed to find the answer correctly and it does it reliably.

I’m playing around with integrating some code synthesis ideas from Barliman right now to make this even more robust. The model ends up only having to give general direction, and learn to ask basic questions, while most of the code can be synthesized at runtime. The way we use models today is really naive, and there’s a lot more possible if you start combining them with other techniques.

here https://archive.ph/ycGN8

You might want to learn what words like reactionary actually mean before using them. We are discussing an open source tool, which by its nature lacks the built-in constraints you are describing. Your argument is a piece of sophistry designed to create the illusion of expertise on a subject you clearly do not understand. You are not engaging with the reality of the technology, but with a simplified caricature of it.

Technology such as LLMs is just automation and that’s what the base is, how it is applied within a society is what’s dictated by the uperstructure. Open source LLMs such as DeepSeek are a productive force, and a rare instance where a advanced means of production is directly accessible for proletarian appropriation. It’s a classic base level conflict over the relations of production.

Nah, I don’t think I’m going to take as gospel what a CIA asset say.

Instead, go read Marx to understand the relationship between the technology and the social relations that dictate its use within a society.

Elections are just the surface of the problem. The real issue is who owns the factories and funds the research. In the West that’s largely done by private capital, putting it entirely outside the sphere of public debate. Even universities are heavily reliant on funding from companies now, which obviously influences what their programs focus on.

or maybe it’s the capitalist relations and not the technology that’s the actual problem here

Right, I think the key difference is that we have a feedback loop and we’re able to adjust our internal model dynamically based on it. I expect that embodiment and robotics will be the path towards general intelligence. Once you stick the model in a body and it has to deal with the environment, and learn through experience, then it will start creating a representation of the world based on that.

It seemed pretty clear to me. If you have any clue on the subject then you presumably know about the interconnect bottleneck in traditional large models. The data moving between layers often consumes more energy and time than the actual compute operations, and the surface area for data communication explodes as models grow to billions parameters. The mHC paper introduces a new way to link neural pathways by constraining hyper-connections to a low-dimensional manifold.

In a standard transformer architecture, every neuron in layer N potentially connects to every neuron in layer N+1. This is mathematically exhaustive making it computationally inefficient. Manifold constrained connections operate on the premise that most of this high-dimensional space is noise. DeepSeek basically found a way to significantly reduce networking bandwidth for a model by using manifolds to route communication.

Not really sure what you think the made up nonsense is. 🤷

I’m personally against copyrights as a concept and absolutely don’t care about this aspect, especially when it comes to open models. The way I look at is that the model is unlocking this content and making this knowledge available to humanity.

Ah yes, they must be stealing IP from the future when they publish novel papers on things nobody’s done before!

It’s amazing how people just can’t learn the lesson that the problem isn’t that a particular oligarch owns a public forum, but that public forums are privately owned in the first place.

I’m betting within a decade if it actually works

Open sourcing these thing would definitely be the right way to go, and you’re absolutely right that it’s a general solver that would be useful in any scenario where you have a system that requires dynamic allocation.

link to study https://pubs.aip.org/aip/apl/article-abstract/127/14/143902/3366312/A-ultra-high-temperature-free-piston?redirectedFrom=fulltext

Yeah for sure, I do think it’s only a matter of time before people figure out a new substrate. It’s really just a matter of allocating time and resources to the task, and that’s where state level planning comes in.

it’s a pretty big case right now