I believe Yarrow is referencing this series of articles from David Thorstadt, which quotes primary sources extensively. 

What happens if this is true and AI improvements will primarily be inference driven? 

It seems like this would be very bad news for AI companies, because customers would have to pay for accurate AI results directly, on a per-run basis. Furthermore, they would have to pay exponentional costs for a linear increase in accuracy. 

As a crude example, would you expect a translation agency to pay four times as much for translations with half as many errors? In either case, you'd still need a human to come along and correct the errors. 

This analysis seems to have an oddly static view of the future, as if the values of the current day CCP will be locked-in forever. But the worldview of chinese leadership has changed massively, multiple times over the last century.

It's easily seen in history that the economic organisation of a company can have huge effects on it's culture and how it's governed: Would not the advent of powerful AI do the same? For example, perhaps the chinese would have more time to discuss moral philosophy if AI allowed them to not have to work as hard. 

Presumably a lot of these are all optimised for the current gen-AI paradigm, though. But we're talking about what happens if the current paradigm fails. I'm sure some of it would carry over to a different AI paradigm, but also it's pretty likely there would be other bottleneck we would have to tune to get things working. 

I feel like what you're saying is the equivalent of pointing out in 2020 that we have had so many optimisations and computing resources that went into, say, google searches, and then using that as evidence that surely the big data that goes into LLM's should be instantaneous as well. 

I don't think you should be treating points 1 to 3 as independent things. Even if AI is sandbagging, what we care about is whether it is sandbagging enough for it to be suddenly capable of world domination. I don't know what evidence you have for sandbagging, but I'm betting it's not finding that LLM's are capable of major scientific discoveries but just deciding not to do them for some reason. 

The two things that matter are how capable the next gen will be compared to the capability bar for conquering all of humanity against our will. I think the difficulty of world domination is truly ridiculously high compared to current-day LLMs. Even if I am overestimating the bar by a lot, and underestimating LLMs by a lot, it's still not enough for the next-gen to be world domination capable. 

Your bot seems pretty prone to making things up. Here is an extract of a conversation I had:

Every single link in the screenshot lead to dead end links like this one. You can look at the actual paper it's talking about here: there is no 69% figure, and the passage quoted by your bot does not exist. 

You have basically created a lying propaganda bot. 

I'll continue to defend the 'standard reading'. I think the story can be critiquing our lack of imagination of utopia and also be against standard utilitarianism and third world exploitation and so on. I don't think the two are opposed, I think they actually link up. 

I think what's missing from your interpretation is the climax of the story, which is also the title: the ones who walk away:

They leave Omelas, they walk ahead into the darkness, and theydo not come back. The place they go towards is a place even less imaginable to most of us that the city of happiness. I cannot describe it at all. It is possible that it does not exist. But they seem to know where they are going, the ones who walk away from Omelas

I don't think this finale works, if the story is only about Omelas being absurd. I think LeGuin is arguing that the type of society described in Omelas is wrong, and is arguing that it should be rejected. 

LeGuin was a left-wing anarchist and anti-capitalist. I'm guessing she was probably gesturing to western society as analogous to Omelas, as a place of plenty that benefits from the exploitation of others in the developing world. She was saying that people justify an unjust status quo with utilitarian arguments that exploitation is justified on greater good grounds. I see the story as an attack on peoples inability to think that a world without exploitation and inequality is possible: an attack on fatalistic acceptance of pain and suffering as inevitable. 

I'm guessing that you are not a left-wing anarchist anti-capitalist, which is fine. I think your interpretation is a valid one, but I would guess that it is not the one that is intended by the author, which is also fine. LeGuin was an excellent writer but nobody is obligated to agree with her politically. 

I think it's kinda weird to call the meaning that the vast majority of people, including LeGuin herself, ascribed to her work as a "misreading". Isn't it more likely that you have found another interpretation of the work that the author didn't intend? 

My interpetation is that Leguin did indeed believe in utopias, and in the passage you cited was indeed critiquing peoples inability to concieve of them. Her excellent book "the dispossessed" has the subtitle "a flawed utopia", and describes an anarchist society that is not sustained by torture or inequality. 

However, "Omelas" is a critique of "Utopias" which are not truly utopias, because the provide good lives to the majority at the expense of bad lives for a few. Under many forms of utilitarianism, a society like the one described in Omelas would be described as a very good one: LeGuin disagrees, and so do I. Omelas is about rejecting fake utopias, and pushing towards real ones. 

Figuring out whether money spent on philanthropy is effective is one of the central tenets of effective altruism. Everybody here critiques each other all the time. Why should private billionaires get a free pass? 

And the author very much does criticise billionaires spending money on "politicians and votes and social media platforms". The author is focusing on the philanthropy side because, well, that's the subject of the article. 

Great post! I think a ban on "brain farming" is extremely tractable, as I expect it to have a severe "ick" factor among the general population (as it did for me). 

There's a case for trying to get a ban as early as possible, before there is any entrenched opposition in place, or any economic penalty for existing industries.