Publishers are blocking the Internet Archive in a bid to outsmart AI scrapers, according to various major publications including The Guardian and The New York Times. These companies are concerned that their content is being used by AI companies' bots to indirectly scrape articles from the internet archive's collections.
In essence, the internet archive serves as a valuable resource for publishers, with its vast collections of records, academic texts, and other materials. However, with the advent of AI technology, these companies have started blocking access to their content, citing concerns that AI businesses are using the Internet Archive's API to scrape their articles.
The New York Times has taken steps to block a bot from accessing its content via the Wayback Machine, which provides unfettered access to the newspaper's archives without authorization. Similarly, The Financial Times and social forum Reddit have also made moves to selectively block how the Internet Archive catalogs their material.
These publishers have attempted to sue AI businesses for using their content in large language models, including OpenAI and Microsoft. While some media outlets have sought financial deals with AI companies, these arrangements seem to provide compensation to publishing companies rather than writers.
The issue extends beyond journalism, as creative fields such as fiction writers, visual artists, and musicians are also fighting against AI tools for copyright and piracy issues.
In essence, the internet archive serves as a valuable resource for publishers, with its vast collections of records, academic texts, and other materials. However, with the advent of AI technology, these companies have started blocking access to their content, citing concerns that AI businesses are using the Internet Archive's API to scrape their articles.
The New York Times has taken steps to block a bot from accessing its content via the Wayback Machine, which provides unfettered access to the newspaper's archives without authorization. Similarly, The Financial Times and social forum Reddit have also made moves to selectively block how the Internet Archive catalogs their material.
These publishers have attempted to sue AI businesses for using their content in large language models, including OpenAI and Microsoft. While some media outlets have sought financial deals with AI companies, these arrangements seem to provide compensation to publishing companies rather than writers.
The issue extends beyond journalism, as creative fields such as fiction writers, visual artists, and musicians are also fighting against AI tools for copyright and piracy issues.
. I mean, it's like they think the archive is just a free resource for them to dump their old content into... but the thing is, authors, artists, and creatives can't even access their own work if AI companies start scraping from it? 
It's all about who gets to control the narrative and profit from shared knowledge. I'm not surprised that some media outlets are trying to make deals with AI companies tho 
. Like, just because you're paying them doesn't mean they won't still use your work for their own gain. And what about fair compensation for writers? Shouldn't they be getting paid for their work, regardless of whether it's being used by a large language model or not? 
. And what about all the writers and creators who actually use the internet archive for research? Are they just gonna get left out of the loop because some publishers are too scared to adapt to change?
. Can't we just find a way to make tech work for everyone instead of against each other?
It seems like these companies are trying to control the narrative and limit access to information, rather than finding a solution that benefits everyone.
Can't we find a middle ground here? 
Maybe some kind of collaboration or licensing model that benefits everyone? 
. The problem is way bigger than just journalists; it's about creators everywhere fighting for their rights against these new tech giants . I'm worried we're going to see a whole new era of copyright laws and regulations come into play... not sure how that's gonna shake out 
... publishers blocking internet archive? sounds like a bad move to me. they're basically saying that if you can use our old content without paying us, then you should be blocked 
. but what about people who want to read/learn from their archives for research? doesn't that even out the scales?