In modern LLM applications like RAG and Agents, the model is constantly fed new context. For example, in RAG, we retrieve relevant documents and stuff them into the prompt. The issue is that this dynamically retrieved context doesn't always appear at the beginning of the input sequence. Traditional KV caching only reuses a "common prefix," so if the new information isn't at the very start, the cache hit rate plummets, and your GPU ends up recomputing the same things over and over. CacheBlend changes the game by allowing for the reuse of pre-computed KV caches regardless of their position in the input sequence. This makes it possible to achieve a 100% KV Cache hit rate in applications like RAG. The performance gains are significant: * Faster Time-To-First-Token (TTFT): Get your initial response much quicker. * More Throughput: Serve significantly more users with the same hardware. * Almost lossless Output Quality: All of this is achieved with little degradation in the model's generation quality. CacheBlend works by intelligently handling the two main challenges of reusing non-prefix caches: * Positional Encoding Update: It efficiently updates positional encodings to ensure the model always knows the correct position of each token, even when we're stitching together cached and new data. * Selective Attention Recalculation: Instead of recomputing everything, it strategically recalculates only the minimal cross-attention needed between the new and cached chunks to maintain perfect generation quality. An interactive CacheBlend demo is available at: https://github.com/LMCache/LMCache-Examples/tree/main/demo-rag-blending

https://archive.ph/lC5R0

> We are constantly fed a version of AI that looks, sounds and acts suspiciously like us. It speaks in polished sentences, mimics emotions, expresses curiosity, claims to feel compassion, even dabbles in what it calls creativity. > > But what we call AI today is nothing more than a statistical machine: a digital parrot regurgitating patterns mined from oceans of human data (the situation hasn’t changed much since it was discussed here five years ago). When it writes an answer to a question, it literally just guesses which letter and word will come next in a sequence – based on the data it’s been trained on. > > This means AI has no understanding. No consciousness. No knowledge in any real, human sense. Just pure probability-driven, engineered brilliance — nothing more, and nothing less. > > So why is a real “thinking” AI likely impossible? Because it’s bodiless. It has no senses, no flesh, no nerves, no pain, no pleasure. It doesn’t hunger, desire or fear. And because there is no cognition — not a shred — there’s a fundamental gap between the data it consumes (data born out of human feelings and experience) and what it can do with them. > > Philosopher David Chalmers calls the mysterious mechanism underlying the relationship between our physical body and consciousness the “hard problem of consciousness”. Eminent scientists have recently hypothesised that consciousness actually emerges from the integration of internal, mental states with sensory representations (such as changes in heart rate, sweating and much more). > > Given the paramount importance of the human senses and emotion for consciousness to “happen”, there is a profound and probably irreconcilable disconnect between general AI, the machine, and consciousness, a human phenomenon. > > [https://archive.ph/Fapar](https://archive.ph/Fapar)

Google’s carbon emissions have soared by 51% since 2019 as artificial intelligence hampers the tech company’s efforts to go green. While the corporation has invested in renewable energy and carbon removal technology, it has failed to curb its scope 3 emissions, which are those further down the supply chain, and are in large part influenced by a growth in datacentre capacity required to power artificial intelligence. The company reported a 27% increase in year-on-year electricity consumption as it struggles to decarbonise as quickly as its energy needs increase. Datacentres play a crucial role in training and operating the models that underpin AI models such as Google’s Gemini and OpenAI’s GPT-4, which powers the ChatGPT chatbot. The International Energy Agency estimates that datacentres’ total electricity consumption could double from 2022 levels to 1,000TWh (terawatt hours) in 2026, approximately Japan’s level of electricity demand. AI will result in datacentres using 4.5% of global energy generation by 2030, according to calculations by the research firm SemiAnalysis.

Spotify, the world’s leading music streaming platform, is facing intense criticism and boycott calls following CEO Daniel Ek’s announcement of a €600m ($702m) investment in Helsing, a German defence startup specialising in AI-powered combat drones and military software. The move, announced on 17 June, has sparked widespread outrage from musicians, activists and social media users who accuse Ek of funnelling profits from music streaming into the military industry. Many have started calling on users to cancel their subscriptions to the service. “Finally cancelling my Spotify subscription – why am I paying for a fuckass app that works worse than it did 10 years ago, while their CEO spends all my money on technofascist military fantasies?” said one user on X.

https://archive.ph/xbx3M

Mark Zuckerberg’s Meta has won the backing of a judge in a copyright lawsuit brought by a group of authors, in the second legal victory for the US artificial intelligence industry this week. The writers, who included Sarah Silverman and Ta-Nehisi Coates, had argued that the Facebook owner had breached copyright law by using their books without permission to train its AI system. The ruling follows a decision on Monday that Anthropic, another major player in the AI field, had not infringed authors’ copyright. The US district judge Vince Chhabria, in San Francisco, said in his decision on the Meta case that the authors had not presented enough evidence that the technology company’s AI would dilute the market for their work to show that its conduct was illegal under US copyright law.

https://github.com/VectorSpaceLab/OmniGen2

cross-posted from: https://kbin.earth/m/[email protected]/t/1528736 > After 20 years, PNG is back with renewed vigor! A new PNG spec was just released.

Allows setting time limits on sites. To stop yourself from wasting your life on dumb stuff.

https://archive.ph/Q9FEn

https://archive.ph/MnjB5

https://archive.ph/MrmJt