Log in
Several people who received the CrowdStrike offer found that the gift card didn't work, while others got an error saying the voucher had been canceled.
© 2024 TechCrunch. All rights reserved. For personal use only.
Los Angeles, CA
Recent discussions on Reddit are no longer showing up in non-Google search engine results. The absence is the result of updates to Reddit’s Content Policy that ban crawling its site without agreeing to Reddit’s rules, which bar using Reddit content for AI training without Reddit’s explicit consent.
As reported by 404 Media, using "site:reddit.com" on non-Google search engines, including Bing, DuckDuckGo, and Mojeek, brings up minimal or no Reddit results from the past week. Ars Technica made searches on these and other search engines and can confirm the findings. Brave, for example, brings up a few Reddit results sometimes (examples here and here) but not nearly as many as what appears on Google when using identical queries. A standout is Kagi, which is a paid-for engine that pays Google for some of its search index and still shows recent Reddit results.
As 404 Media noted, Reddit's Robots Exclusion Protocol (robots.txt file) blocks bots from scraping the site. The protocol also states, "Reddit believes in an open Internet, but not the misuse of public content." Reddit has approved scrapers from the Internet Archive and some research-focused entities.
Los Angeles, CA
Enlarge (credit: Vithun Khamsong)
With the plunging price of photovoltaics, the construction of solar plants has boomed in the US. Last year, for example, the US's Energy Information Agency expected that over half of the new generating capacity would be solar, with a lot of it coming online at the very end of the year for tax reasons. Yesterday, the EIA released electricity generation numbers for the first five months of 2024, and that construction boom has seemingly made itself felt: generation by solar power has shot up by 25 percent compared to just one year earlier.
The EIA breaks down solar production according to the size of the plant. Large grid-scale facilities have their production tracked, giving the EIA hard numbers. For smaller installations, like rooftop solar on residential and commercial buildings, the agency has to estimate the amount produced, since the hardware often resides behind the metering equipment, so only shows up via lower-than-expected consumption.
In terms of utility-scale production, the first five months of 2024 saw it rise by 29 percent compared to the same period in the year prior. Small-scale solar was "only" up by 18 percent, with the combined number rising by 25.3 percent.
Los Angeles, CA
The web scraper bot for Anthropic’s AI chatbot Claude hit iFixit’s website nearly a million times in a single day, despite the repair database having terms of service provisions that state “reproducing, copying or distributing any Content, materials or design elements on the Site for any other purpose, including training a machine learning or AI model, is strictly prohibited without the express prior written permission of iFixit.”
iFixit CEO Kyle Wiens tweeted Wednesday “Hey @AnthropicAI: I get you're hungry for data. Claude is really smart! But do you really need to hit our servers a million times in 24 hours? You're not only taking our content without paying, you're tying up our devops resources. Not cool.”
Wiens sent me server logs that showed thousands of requests per minute for a several hour period. “We're just the largest database of repair information in the world, no big deal if they take it all without asking and swamp our servers in the process,” he told me, adding that iFixit’s website has millions of total pages. These include repair guides, revision histories for those guides, blogs, news posts, and research, forums, community-contributed repair guides and question-and-answer sections, etc.
This sort of scraping has become incredibly commonplace, and a recent study by the Data Provenance Institute shows that website owners are increasingly trying to signal to AI companies that they do not want their content scraped for the purpose of training commercial AI tools. Wiens said that iFixit modified its robots.txt file this week to specifically block Anthropic’s crawler bots.
This is particularly notable because, when I asked Anthropic about the fact that its bot hit iFixit a million times in a day, I was sent a blog post by the company that puts the onus on website owners to specifically block Anthropic’s crawler, called ClaudeBot.
“As per industry standard, Anthropic uses a variety of data sources for model development, such as publicly available data from the internet gathered via a web crawler,” the blog post reads. “Our crawling should not be intrusive or disruptive. We aim for minimal disruption by being thoughtful about how quickly we crawl the same domains and respecting Crawl-delay where appropriate.”
OpenAI Training Bot Crawls ‘World’s Lamest Content Farm’ 3 Million Times in One Day
“If you were wondering what they’re using to train GPT-5, well, now you know.”
Jason Koebler
The post adds that “opting out of being crawled by ClaudeBot requires modifying the robots.txt file” to block its crawler, meaning that instructing companies to not scrape content based on terms of service alone doesn’t actually do anything in practice unless a website wanted to sue the AI company.
Across the board, AI companies almost never respect terms of service, which is interesting because many of them have very long terms of service agreements themselves that sometimes restrict what users can do. In a paper published last week that we’ve already written about a few times, researchers at the Data Provenance Institute found that many websites have requested that their content not be scraped in their terms of service, but that often does not do anything.
This is a shame, lead author Shayne Longpre told me, because terms of service allows website owners to be more nuanced about the types of crawlers they want to allow or block than robots.txt does.
“The tragedy is that terms of service are specific and nuanced, but not machine readable and robots.txt is machine readable, but incredibly coarse and unspecific,” Longpre said. “With terms of service, I suspect the only ones that are being complied with are the very large companies that maybe have filed lawsuits, but they seem to be otherwise ignored.”
Los Angeles, CA
President Joe Biden called himself "the first president this century to report to the American people that the United States is not at war anywhere in the world" in a speech on Wednesday night. Less than an hour before Biden spoke those words, the U.S. military had announced that it was bombing Yemen again.
"In the past 24 hours, U.S. Central Command (USCENTCOM) forces successfully destroyed two Iranian-backed Houthi missiles on launchers in a Houthi-controlled area of Yemen," the U.S. military command responsible for the Middle East and Central Asia declared shortly before Biden's speech began. "It was determined these weapons presented an imminent threat to U.S., coalition forces, and merchant vessels in the region."
After completing the U.S. withdrawal from Afghanistan in 2021, Biden took credit for ending "the longest war in American history," what he called a "forever war." Yet the Biden administration has extended other U.S. forever wars—and introduced U.S. forces into new ones.
When Biden first ordered airstrikes on Yemen's Houthi movement in January, it was the first direct U.S. attack on the Houthis ever, and the first airstrike on Yemen by any force in two years. The U.S. had previously brokered a truce between the Houthi government in Sanaa and the rival Saudi-backed government in Aden.
But as war broke out in Gaza last year, the Houthis announced that they would attack Israeli-linked shipping in solidarity with the Palestinian cause. Foreign ships began avoiding the Red Sea off the coast of Yemen, a key international waterway, and Biden tried to reopen it by bombing the Houthis.
Asked by reporters whether the airstrikes were working, Biden gave a formula for endless war. "Well, when you say 'working,' are they stopping the Houthis? No. Are they going to continue? Yes," he said.
U.S. forces have also taken fire elsewhere in the Middle East due to the war in Gaza. Two weeks after U.S. airstrikes on Yemen began, Iraqi guerrillas killed three U.S. soldiers on the Jordanian-Syrian border with a drone strike. (It was part of a wave of dozens of attacks motivated by U.S. military support for Israel.) Biden bombed Iraqi militia sites in retaliation.
And older U.S. wars in the region continue to rage. Although both Biden and his predecessor Donald Trump have done victory laps, the Islamic State group is still around. In June 2024, the U.S. military assassinated a senior Islamic State group leader in Syria. The month before, it did the same in Somalia.
The Islamic State group isn't the only force that the U.S. is bombing in Somalia, a place where most Americans don't even know their country is at war. U.S. forces are fighting to prop up the Somali government against al-Shabab, another Islamist rebel group. U.S. Africa Command announced 15 airstrikes against al-Shabab in 2023 and six airstrikes this year.
That's not even counting U.S. proxy wars. Washington is providing a steady flow of weapons and intelligence to Ukraine and Israel, most of it at the American taxpayer's expense. While American fingers are not on the triggers and American personnel are not taking fire in Ukraine and Gaza—at least not yet—the Biden administration also owns those wars.
At the NATO summit in Washington earlier this month, U.S. officials co-signed a statement that Ukraine was on an "irreversible" path to joining the alliance. In other words, American troops may be pledged to defend Ukraine in a future war.
"We'll keep rallying a coalition of proud nations to stop [Russian President Vladimir] Putin from taking over Ukraine and doing more damage. We'll keep NATO strong," Biden said in his Wednesday night address. "I'll make it more powerful and more united than any time in all our history. Keep doing the same for allies in the Pacific."
Biden, like Trump, wants to have his cake and eat it too. He wants to take advantage of anti-war sentiment in the American public without being seen as compromising with rival powers or retreating from foreign conflicts. Biden's pitch is pretty similar to Trump's: "peace through strength." And when he can't deliver on that promise, he confuses and deceives the public.
The post Biden Brags That 'the United States Is Not at War' As He Bombs Yemen appeared first on Reason.com.
Los Angeles, CA
Next Page of Stories