Quick takes

If the people arguing that there is an AI bubble turn out to be correct and the bubble pops, to what extent would that change people's minds about near-term AGI?  I strongly suspect there is an AI bubble because the financial expectations around AI seem to be based on AI significantly enhancing productivity and the evidence seems to show it doesn't do that yet. This could change — and I think that's what a lot of people in the business world are thinking and hoping. But my view is a) LLMs have fundamental weaknesses that make this unlikely and b) scaling is running out of steam. Scaling running out of steam actually means three things: 1) Each new 10x increase in compute is less practically or qualitatively valuable than previous 10x increases in compute. 2) Each new 10x increase in compute is getting harder to pull off because the amount of money involved is getting unwieldy. 3) There is an absolute ceiling to the amount of data LLMs can train on that they are probably approaching. So, AI investment is dependent on financial expectations that are depending on LLMs enhancing productivity, which isn't happening and probably won't happen due to fundamental problems with LLMs and due to scaling becoming less valuable and less feasible. This implies an AI bubble, which implies the bubble will eventually pop.  So, if the bubble pops, will that lead people who currently have a much higher estimation than I do of LLMs' current capabilities and near-term prospects to lower that estimation? If AI investment turns out to be a bubble, and it pops, would you change your mind about near-term AGI? Would you think it's much less likely? Would you think AGI is probably much farther away?

I have a question I would like some thoughts on: As a utilitarianist I personally believe alignment to be the most important course area - though weirdly enough, even though I believe x-risk reduction to be positive in expectation, I believe the future is most likely to be net negative. I personally believe, without a high level of certainty, that the current utilities on earth are net negative due to wild animal suffering. If we therefore give the current world a utility value of -1, I would describe my believe for future scenarios like this: 1. ~5% likelihood: ~101000  (Very good future e.g. hedonium shockwave) 2. ~90% likelihood: -1 (the future is likely good for humans, but this is still negligible compared to wild animal suffering, which will remain) 3. ~5% likelihood: ~-10100 (s-risk like scenarios)   My reasoning for thinking “scenario 2” is more likely than “scenario 1” is based on what seems to be the values of the general public currently. Most people seem to care about nature conservation, but no one seems interested in mass producing (artificial) happiness. While the earth is only expected to remain habitable for about two billion years (and humans, assuming we avoid any x-risks, are likely to remain for much longer), I think, when it comes to it, we’ll find a way to keep the earth habitable, and thus wild animal suffering.  Based on these 3 scenarios, you don’t have to be a great mathematician to realize that the future is most likely to be net negative, yet positive in expectation. While I still, based on this, find alignment to be the most important course area, I find it quite demotivating that I spend so much of my time (donations) to preserve a future that I find unlikely to be positive. But the thing is, most long-termist (and EAs in general) don't seem to have similar beliefs to me. Even someone like Brian Tomasik has said that if you’re a classical utilitarian, the future is likely to be good.   So now I’m asking, what am I getting wro

The current US administration is attempting an authoritarian takeover. This takes years and might not be successful. My manifold question puts an attempt to seize power if they lose legitimate elections at 30% (n=37). I put it much higher.[1] Not only is this concerning by itself, this also incentivizes them to achieve a strategic decisive advantage via superintelligence over pro-democracy factions. As a consequence, they may be willing to rush and cut corners on safety.  Crucially, this relies on them believing superintelligence can be achieved before a transfer of power. I don't know how much the belief in superintelligence has spread into the administration. I don't think Trump is 'AGI-pilled' yet, but maybe JD Vance is? He made an accelerationist speech. Making them more AGI-pilled and advocating for nationalization (like Ashenbrenner did last year) could be very dangerous. 1. ^ So far, my pessimism about US Democracy has put me in #2 on the Manifold topic, with a big lead over other traders. I'm not a Superforecaster though.

Has anyone considered the implications of a Reform UK government? It would be greatly appreciated if someone with the relevant experience or knowledge could share their thoughts on this topic.  I know this hypothetical issue might not warrant much attention when compared to today's most pressing problems, but with poll after poll suggesting Reform UK will win the next election, it seems as if their potential impact should be analysed. I cannot see any mention of Reform UK on this forum. Some concerns from their manifesto: * Cutting foreign aid by 50% * Scrapping net zero and renewable energy subsidies * Freezing non-essential migration * Leaving the European convention on human rights Many thanks

Current takeaways from the 2024 US election <> forecasting community. First section in Forecasting newsletter: US elections, posting here because it has some overlap with EA. 1. Polymarket beat legacy institutions at processing information, in real time and in general. It was just much faster at calling states, and more confident earlier on the correct outcome. 2. The OG prediction markets community, the community which has been betting on politics and increasing their bankroll since PredictIt, was on the wrong side of 50%—1, 2, 3, 4, 5. It was the democratic, open-to-all nature of it, the Frenchman who was convinced that mainstream polls were pretty tortured and bet ~$45M, what moved Polymarket to the right side of 50/50. 3. Polls seem like a garbage in garbage out kind of situation these days. How do you get a representative sample? The answer is maybe that you don't. 4. Polymarket will live. They were useful to the Trump campaign, which has a much warmer perspective on crypto. The federal government isn't going to prosecute them, nor bettors. Regulatory agencies, like the CFTC and the SEC, which have taken such a prominent role in recent editions of this newsletter, don't really matter now, as they will be aligned with financial innovation rather than opposed to it. 5. NYT/Siena really fucked up with their last poll and the coverage of it. So did Ann Selzer. Some prediction market bettors might have thought that you could do the bounded distrust, but in hindsight it turns out that you can't. Looking back, to the extent you trust these institutions, they can ratchet their deceptiveness (from misleading headlines, incomplete stories, incomplete quotes out of context, not reporting on important stories, etc.) for clicks and hopium, to shape the information landscape for a managerial class that... will no longer be in power in America. 6. Elon Musk and Peter Thiel look like geniuses. In contrast Dustin Moskovitz couldn't get SB 1047 passed despite being the s

Not that we can do much about it, but I find the idea of Trump being president in a time that we're getting closer and closer to AGI pretty terrifying. A second Trump term is going to have a lot more craziness and far fewer checks on his power, and I expect it would have significant effects on the global trajectory of AI.

As someone predisposed to like modeling, the key takeaway I got from Justin Sandefur's Asterisk essay PEPFAR and the Costs of Cost-Benefit Analysis was this corrective reminder – emphasis mine, focusing on what changed my mind: More detail: Tangentially, I suspect this sort of attitude (Iraq invasion notwithstanding) would naturally arise out of a definite optimism mindset (that essay by Dan Wang is incidentally a great read; his follow-up is more comprehensive and clearly argued, but I prefer the original for inspiration). It seems to me that Justin has this mindset as well, cf. his analogy to climate change in comparing economists' carbon taxes and cap-and-trade schemes vs progressive activists pushing for green tech investment to bend the cost curve. He concludes:  Aside from his climate change example above, I'd be curious to know what other domains economists are making analytical mistakes in w.r.t. cost-benefit modeling, since I'm probably predisposed to making the same kinds of mistakes. 

For a long time I found this surprisingly nonintuitive, so I made a spreadsheet that did it, which then expanded into some other things. * Spreadsheet here, which has four tabs based on different views on how best to pick the fair place to bet where you and someone else disagree. (The fourth tab I didn't make at all, it was added by someone (Luke Sabor) who was passionate about the standard deviation method!)  * People have different beliefs / intuitions about what's fair! * An alternative to the mean probability would be to use the product of the odds ratios. Then if one person thinks .9 and the other .99, the "fair bet" will have implied probability more than .945. *  The problem with using Geometric mean can be highlighted if player 1 estimates 0.99 and player 2 estimates 0.01. This would actually lead player 2 to contribute ~90% of the bet for an EV of 0.09, while player 1 contributes ~10% for an EV of 0.89. I don't like that bet. In this case, mean prob and Z-score mean both agree at 50% contribution and equal EVs. * "The tradeoff here is that using Mean Prob gives equal expected values (see underlined bit), but I don't feel it accurately reflects "put your money where your mouth is". If you're 100 times more confident than the other player, you should be willing to put up 100 times more money. In the Mean prob case, me being 100 times more confident only leads me to put up 20 times the amount of money, even though expected values are more equal." * Then I ended up making an explainer video because I was excited about it   Other spreadsheets I've seen in the space: * Brier score betting (a fifth way to figure out the correct bet ratio!) * Posterior Forecast Calculator * Inferring Probabilities from PredictIt Prices These three all by William Kiely. Does anyone else know of any? Or want to argue for one method over another?