Whilst I’ve heard lots of talk that lunduke is getting increasingly politica, and I disagree quite strongly with his politics, I’ll have to agree with him here. IA did something unnecessarily risky (redistributing unauthorised copies of print books), which has more jeopardised their mission of archiving the internet.
I also agree with everyone here saying that current copyright laws are ridiculous (and not just because they are “outdated”, the Victorians had better copyright laws than we do). However, I think only the most radical overhaul of copyright law would condone what IA did, and that isn’t coming any time soon (If ever).
Does this eBook downloading thing affect the rest of the Archive? Like, will the entire archive be affected or just the OpenLibrary part of the Archive?
I’ll give the parent poster the benefit of the doubt and say that it’s a satirical user account. It’s called spez_, named after the reddit CEO, I’m guessing.
The problem is an obsolet copyright law, made by big companies and elderly politicians confusing a remote control with a smartphone. Money is killing the culture and the knowledge.
145+ Petabytes for a single copy of the archive and they currently have two copies of everything for a total of around 290 Petaybtes.
The largest hard drive I’m aware of is 32TB so you’d “only” need over 9,000 (lol) of the largest drives ever made. I can’t even tell you what that would cost since Seagate doesn’t have a publicly available price for the damn things!
And it had to be replicated, so 3 copies somewhere (granted proper backups are compressed).
Let’s say they have a proper backup compressed at (a random) 60%. That one backup is 87 petabytes. With daily incrementals, so another what, 14 PB to get through 2 weeks of incrementals? Something in the range of 600 PB total with replicas?
(I’m completely pulling numbers out of my ass, I’m not familiar with how such large datasets are managed from a DR perspective).
And, you’d want/need redundancy. One on-site back up for quick restoration and one off-site for surviving physical disaster. So, you’d need at least 3 times that. In HDD prices, that is roughly 2.5 million per set-up, or 7.5 million total for all three. And in SSD prices, well it’s about 3x that. 7.5 million per set up and 22.5million for all three.
An alternate option is a distributed back-up. They could have people volunteer to store and host like 10 gigs each, and just hand out each 10 gig chunk to 10 different people. That would take alot of work to set up, but it would be alot safer. And there are already programs/systems like that to model after. 10 gigs is just an example, might be more successful or even more possible in chunks of 1-2 terabytes. Basically one full hard drive per volunteer.
Lol, had to add that after doing the math for 10 gigs to ten people and realising that was 1000 people per terabyte, so would take 150 million volunteers. Even at 2 petabytes each, assuming we still wanted 10x redundancy in that model, it would be like 750 thousand volunteers or something like that. Maybe there is no sustainable volunteer driven model, lol.
You’d need more than 9,000 of the largest hard drives made (32TB) to store the nearly 300 Petabytes of data they have. Still within the reach of an obscenely rich tech bro but not exactly cheap.
Copyright is no longer functioning properly, if the Internet Archive cannot survive under it.
IP holders are pushing their luck, lately.
Edit: Lately it feels like everyone is underestimating the power of librarians. Anyone working against librarians is on the wrong side of whatever they’re on about.
“Fuck you, I’m doing library stuff.” should be a valid legal defense.
Reading the article and I wonder what Internet Archive thought when they turned themselves into a Pirate Bay but for ebooks. They had this lawsuit coming and they have obviously no idea how to defend themselves in this case.
They should stick to archiving the web instead of shooting themselves in the foot.
I’ve been saying this for years, this was an incredibly boneheaded move by the Internet Archive and they just keep on doubling down on it. They shouldn’t have done it in the first place. When they got sued, they should have immediately admitted they screwed up and settled - the publishers would probably have been fine with a token punishment and a promise to shut down their ebook library, it’s not like IA cost them anything significant. But they just keep on fighting, and it’s only making things worse.
This isn’t even IA’s purpose in the first place! They archive the Internet. They’re like a guy who’s caring for a precious baby who decides he should go poke a bear with a stick, and when the bear didn’t respond at first he whacked it over the nose with the stick instead. Now the bear’s got his leg and he’s screaming “oh no, protect my baby!” And it’s entirely his fault the baby’s in danger.
They really fucked up and it’s so heartwrenching watching it all happen. I was following the story since it started and I just can’t believe they allowed anyone to download copyrighted books without a limit in 2020, without asking anyone for permission or whether it’s legally viable. Everyone knew they were losing this, and they gave publishers a convincing reason to sue them. by crossing the “legally grey” area to literal piracy.
FWIW, OpenLibrary is a good source of book metadata at least, even if it fails its goal of letting people read books on it.
Full access to currently published copyrighted books was way too much. Even Google just showed snippets or showed abandonware books. They really should have settled.
Unlimited free distribution of copyrighted media is something I’m all for, but that’s a really tall order in terms of political capital for getting the law changed, a few protests aren’t going to do it, basically every elected official would strongly oppose it, you’d have to replace them all first.
How is licking the boots of the copyright office helping the people? I don’t give the first fuck about individuals trying to hoard up IP anymore; IP hoarding-- and the very concept of IP in the first place-- is fuckin bullshit, fuck 'em if they got a problem with that take. We already have to pay through the nose for too much else in society, so why are you licking copyright-uplifting boots?
Authors have the right to be compensated for their work. There are many things that are wrong with current copyright, but denying artists compensation for their work is not the way.
Wouldn’t you also be mad if your salary was denied?
I live every day of my life under a regime that steals my wages, calls it ‘profit’, and then in my face, gets away with pocketing it, with STEALING IT; that shit is ALREADY happening. At this point, it’d just even the playing field. If you want me to have solidarity with my fellow worker, you cannot have it both ways where you expect me to uplift the tools of capitalist theft too. Fuck out of my face if all you have is platitudes to capital.
For me it’s not boot licking but recognizing that IA made a huge unforced error that may cost us all not just that digital lending program but stuff like the Wayback Machine and all the other good projects the IA runs.
Exactly. Less bootlickering and more “Leopards Ate My Face” material. This was heavily forseable. If they did this as a protest against copyright and announced it from the start, it would be one thing. But this was just incompetence and ignorance at a level that will likely ruin them.
I wrote that then deleted it cause I wasn’t sure how you address it internationally. Where PBS is broadcast in the USA, the internet is open to the world.
But, you’re right! It should be publicly funded. I’d have no problem with my tax dollars going towards that.
True, I don’t know how this would work for international stuff, but this is human knowledge and history, it’s something we should be archiving and not tossing to the wind.
To do that they need to make sure they have adequate funding and make sure they don’t incur some huge financial liabilities somehow. The Internet Archive failed at that last part when they decided to lend out ebooks that are under copyright without many limits (and potentially with their Great 78 Project regarding music as well).
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]
This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.
Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.
Rules:
1: All Lemmy rules apply
2: Do not post low effort posts
3: NEVER post naziped*gore stuff
4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.
5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)
6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist
7: crypto related posts, unless essential, are disallowed
Whilst I’ve heard lots of talk that lunduke is getting increasingly politica, and I disagree quite strongly with his politics, I’ll have to agree with him here. IA did something unnecessarily risky (redistributing unauthorised copies of print books), which has more jeopardised their mission of archiving the internet.
I also agree with everyone here saying that current copyright laws are ridiculous (and not just because they are “outdated”, the Victorians had better copyright laws than we do). However, I think only the most radical overhaul of copyright law would condone what IA did, and that isn’t coming any time soon (If ever).
Does this eBook downloading thing affect the rest of the Archive? Like, will the entire archive be affected or just the OpenLibrary part of the Archive?
removed by mod
No
Give me one, just one, good reason for this statement, please. If not, I’ll just block your account so I won’t see your trolling.
Who are you?
I’ll give the parent poster the benefit of the doubt and say that it’s a satirical user account. It’s called spez_, named after the reddit CEO, I’m guessing.
That’s probably true, but if the satire is annoying in its own right, I’m not going to indulge it either lol
The problem is an obsolet copyright law, made by big companies and elderly politicians confusing a remote control with a smartphone. Money is killing the culture and the knowledge.
Let see how long would be exist these
If not, the alternative
Operation archive the archive?
The internet archive probably contains sooo much data its probably 4 petabytes and thats hard to store
It’s “only” a couple hundred hard drives.
145+ Petabytes for a single copy of the archive and they currently have two copies of everything for a total of around 290 Petaybtes.
The largest hard drive I’m aware of is 32TB so you’d “only” need over 9,000 (lol) of the largest drives ever made. I can’t even tell you what that would cost since Seagate doesn’t have a publicly available price for the damn things!
And it had to be replicated, so 3 copies somewhere (granted proper backups are compressed).
Let’s say they have a proper backup compressed at (a random) 60%. That one backup is 87 petabytes. With daily incrementals, so another what, 14 PB to get through 2 weeks of incrementals? Something in the range of 600 PB total with replicas?
(I’m completely pulling numbers out of my ass, I’m not familiar with how such large datasets are managed from a DR perspective).
It was 30 Petabytes back in 2016.
Today a single copy of the Internet Archive is over 145 Petabytes!
And, you’d want/need redundancy. One on-site back up for quick restoration and one off-site for surviving physical disaster. So, you’d need at least 3 times that. In HDD prices, that is roughly 2.5 million per set-up, or 7.5 million total for all three. And in SSD prices, well it’s about 3x that. 7.5 million per set up and 22.5million for all three.
An alternate option is a distributed back-up. They could have people volunteer to store and host like 10 gigs each, and just hand out each 10 gig chunk to 10 different people. That would take alot of work to set up, but it would be alot safer. And there are already programs/systems like that to model after. 10 gigs is just an example, might be more successful or even more possible in chunks of 1-2 terabytes. Basically one full hard drive per volunteer.
Lol, had to add that after doing the math for 10 gigs to ten people and realising that was 1000 people per terabyte, so would take 150 million volunteers. Even at 2 petabytes each, assuming we still wanted 10x redundancy in that model, it would be like 750 thousand volunteers or something like that. Maybe there is no sustainable volunteer driven model, lol.
Too bad there are no obscenely rich techbros around for whom this would be nothing.
That’s chump yacht money.
You’d need more than 9,000 of the largest hard drives made (32TB) to store the nearly 300 Petabytes of data they have. Still within the reach of an obscenely rich tech bro but not exactly cheap.
even then you’d still need networking, caching, the rest of the servers, and someone to deploy all of this
We cannot lose the Internet archive
Copyright is no longer functioning properly, if the Internet Archive cannot survive under it.
IP holders are pushing their luck, lately.
Edit: Lately it feels like everyone is underestimating the power of librarians. Anyone working against librarians is on the wrong side of whatever they’re on about.
“Fuck you, I’m doing library stuff.” should be a valid legal defense.
Reading the article and I wonder what Internet Archive thought when they turned themselves into a Pirate Bay but for ebooks. They had this lawsuit coming and they have obviously no idea how to defend themselves in this case.
They should stick to archiving the web instead of shooting themselves in the foot.
I’ve been saying this for years, this was an incredibly boneheaded move by the Internet Archive and they just keep on doubling down on it. They shouldn’t have done it in the first place. When they got sued, they should have immediately admitted they screwed up and settled - the publishers would probably have been fine with a token punishment and a promise to shut down their ebook library, it’s not like IA cost them anything significant. But they just keep on fighting, and it’s only making things worse.
This isn’t even IA’s purpose in the first place! They archive the Internet. They’re like a guy who’s caring for a precious baby who decides he should go poke a bear with a stick, and when the bear didn’t respond at first he whacked it over the nose with the stick instead. Now the bear’s got his leg and he’s screaming “oh no, protect my baby!” And it’s entirely his fault the baby’s in danger.
They really fucked up and it’s so heartwrenching watching it all happen. I was following the story since it started and I just can’t believe they allowed anyone to download copyrighted books without a limit in 2020, without asking anyone for permission or whether it’s legally viable. Everyone knew they were losing this, and they gave publishers a convincing reason to sue them. by crossing the “legally grey” area to literal piracy.
FWIW, OpenLibrary is a good source of book metadata at least, even if it fails its goal of letting people read books on it.
Full access to currently published copyrighted books was way too much. Even Google just showed snippets or showed abandonware books. They really should have settled.
If people were acting in mass civil disobedience in defense of IA doing the right thing, we could change the law.
Unlimited free distribution of copyrighted media is something I’m all for, but that’s a really tall order in terms of political capital for getting the law changed, a few protests aren’t going to do it, basically every elected official would strongly oppose it, you’d have to replace them all first.
Wow, lot of boot lickers in this thread.
Unjust laws need pressure to be changed. I salute IA for their attempts to push progress.
“Anyone I disagree with is a bootlicker!”
No, but anyone who defends copyrights of major corporations that harm artists is a boot licker
How is distributing artist’s work freely without compensation or their permission helping them?
How is licking the boots of the copyright office helping the people? I don’t give the first fuck about individuals trying to hoard up IP anymore; IP hoarding-- and the very concept of IP in the first place-- is fuckin bullshit, fuck 'em if they got a problem with that take. We already have to pay through the nose for too much else in society, so why are you licking copyright-uplifting boots?
Authors have the right to be compensated for their work. There are many things that are wrong with current copyright, but denying artists compensation for their work is not the way.
Wouldn’t you also be mad if your salary was denied?
I live every day of my life under a regime that steals my wages, calls it ‘profit’, and then in my face, gets away with pocketing it, with STEALING IT; that shit is ALREADY happening. At this point, it’d just even the playing field. If you want me to have solidarity with my fellow worker, you cannot have it both ways where you expect me to uplift the tools of capitalist theft too. Fuck out of my face if all you have is platitudes to capital.
Great, then you agree that stealing is bad.
And now they nuked the entire IA.
Well done.
Intentions were probably correct but this result is kinda ass.
The only change that will come out of this is that IA will pay a huge bill. They’re too small to even make a nudge to the copyright laws.
I just hope this pointless move won’t bring down the wayback machine.
What was the pointless move you’re referring to?
Stir up the hornets nest by freely distributing copyrighted physical media. The only outcome is that they will get stung.
For me it’s not boot licking but recognizing that IA made a huge unforced error that may cost us all not just that digital lending program but stuff like the Wayback Machine and all the other good projects the IA runs.
Exactly. Less bootlickering and more “Leopards Ate My Face” material. This was heavily forseable. If they did this as a protest against copyright and announced it from the start, it would be one thing. But this was just incompetence and ignorance at a level that will likely ruin them.
Internet Archive and Wikipedia are two websites that need to exist in perpetuity.
They need to be publicly funded like we do with PBS. They’re to great of a resource to have corporations trying to destroy them.
I wrote that then deleted it cause I wasn’t sure how you address it internationally. Where PBS is broadcast in the USA, the internet is open to the world.
But, you’re right! It should be publicly funded. I’d have no problem with my tax dollars going towards that.
True, I don’t know how this would work for international stuff, but this is human knowledge and history, it’s something we should be archiving and not tossing to the wind.
FYI PBS gets very little from public funding.
Well that’s shitty. We need Mr. Rogers to rise like jesus and fix it.
Indeed, which is why I’m furious at the Internet Archive’s leadership for merrily dancing out into a minefield completely unbidden.
To do that they need to make sure they have adequate funding and make sure they don’t incur some huge financial liabilities somehow. The Internet Archive failed at that last part when they decided to lend out ebooks that are under copyright without many limits (and potentially with their Great 78 Project regarding music as well).
Wikipedia? The free encyclopedia that only the State Department can edit? 🤣🙄
This just in, corporate apologist media thinks Internet Archive should go away.
Why is this trash here?
Contact your representatives if you have good ones that are useful and effective. For the rest of us, pour one out.