Jonas Hallgren 🔸

516 karmaJoined Uppsala, Sweden

Bio

Participation
5

Co-Director of Equilibria Network: https://eq-network.org/

I try to write as if I were having a conversation with you in person. 

I would like to claim that my current safety beliefs are a mix between Paul Christiano's, Andrew Critch's and Def/Acc.
 

Comments
83

Topic contributions
3

Firstly, great post thanks for writing it!

Secondly, with regards to the quantification section:

Putting numbers on the qualities people have feels pretty gross, which is probably why using quantification in hiring is rather polarising. On the one hand, there’s some line of thinking that the different ways in which people are well and ill suited to particular roles isn’t quantifiable and if you try to quantify it you’ll just be introducing bias. On the other hand, people in favour of quantification tend to strongly recommend that you stick exactly to the ranking your weightings produced. 

I just wanted to mention something that I've been experimenting a bit with lately that I think has worked reasonably well when it comes to this? One of the problems here is the overindexing on the numbers that you assign to people and taking the numbers too seriously. A way to go around taking things to seriously is play and we did an experiment where we took this seriously.

When we took mentees into our latest research program we divided people up into different D&D Classes such as "wizard", "paladin" and "engineer" based on their profiles. You're not going to be able to make a decision fully based on the experience level someone has as a "paladin", yet you're not going to feel bad using the information.

I imagine it can be a bit hard to implement in an existing organisation but I do think this degree of playfulness opens up a safety in talking about hiring decisions that wasn't there before. So I'll likely continue to use this system.

I'll post the list of classes below as well as how to evaluate their level from 1-10 if anyone is interested (you can also multi-class and experience is within a class):

Tank -  Can take a bunch of work and get things done
Healer -  Helps keep the team on track with excellent people management
Paladin - A leader that can heal but also take on a bunch of the operational work - generalist
Sorcerer - Communicator & creative that can magic things out into the real world intuitively
Bard - A communicator that has experience with talking with external stakeholders & writing beautiful prose about the work 
Engineer - Technical person who can make all the technical stuff happen
Wizard - Organised researcher with deep knowledge in fields that can create foundational work
Diplomat - Understanding institutional design and governance structures and crafting policies and frameworks that enable coordination

Levels:
1 - Hasn’t slain rats yet - no experience
3 - Finished the sewer level - Finished undergrad + initial project in AI Safety
5 - Can fight wolves relatively well - Done with PhD + initial knowledge in AI Safety
7 - When you’re slaying an epic monster you want this person in your team - Experience with taking responsibility in difficult domains
9 - Could probably slay a dragon if they try - Wooow, this person is like so cool, god damn.
10 - Legendary expert - possible one of the best people in their field

Very very well put. 

I became quite emotional when reading this because I resonated with it quite strongly. I've been in some longer retreats practicing the teachings in Seeing That Frees and I've noticed the connections between EA and Rob Burbea's way of seeing things but I haven't been able to express it well. 

I think that there's a very beauitful deepening of a seeing of non-self when acting impartialy. One of the things that I really like about applying this to EA is that you often don't see the outcomes of your actions. This is often seen as a bad thing but from a vipassyana perspective this also somehow gets rid of the near enemy of loving kindness in purpose of getting something back. So it is almost like loving kindness based on EA principles is somehow less clinging than existing loving kindness practices? 

I love the focus on the cultiavation of positive mental states as a foundation for doing effective work as well. Beautifully put, maybe one of my favourite forum posts of all time, thank you for writing this. 

The question that is on every single EAs mind is, of course, what about huel or meal replacements? I've been doing huel+supplements for a while now instead of meat and I want to know if you believe this to be suboptimal and if so to what extent? Nutrition is annoyingly complex and so all I know for sure is like protein=good, cal in=cal out and minimize sugar (as well as some other things) and huel seems to tick all the boxes? I'm probably missing something but I don't know what so if you have an answer, please enlighten me!

This one hit close to home (pun not intended). 

I've been thinking about this choice for a while now. There's the obvious network and work benefits in living in an EA Hub yet in my experience there's also the benefit of a slower pace leading to more time to think and reflect and develop my own writing and opinions on things which is easier to get when not in a hub.

Yet in AI safety (where I work) all of the stuff is happening in the Bay and London and mostly the Bay. For the last 3 years people have constantly been telling me "Come to the Bay, bro. It will be worth it, everything is happening here". So there's a lot of FOMO and also literal missing out involved in this decision. 

I had been thinking that I would delay this decision until later but like 6 of your 9 criteria are fulfilled for me and I find that it feels more value aligned and that it might also be smart to plan with this in mind from an earlier age. (I'm 23 from Sweden)

So I'm leaning on Sweden as a home base and to visit the other places for conferences and work, maybe some longer work stances but generally living in Sweden and having it as a base. 

It feels a bit drastic (and we'll see if this holds) but it kind of feels like you helped me resolve one of my larger questions in life so thanks? :D

Uncertain risk. AI infrastructure seems really expensive. I need to actually do the math here (and I haven’t! hence this is uncertain) but do we really expect growth on trend given the cost of this buildout in both chips and energy? Can someone really careful please look at this?

 

https://www.lesswrong.com/users/vladimir_nesov <- Got a bunch of stuff on energy calculations and similar required for AI companies, especially the 2028 post, some very good analysis of these things imo.

I'm curious about the link that goes to AI-enabled coups and it isn't working, could you perhaps relink it?

Besides the point that "shoddy toy models" might be emotionally charged, I just want to point out that accelerating progress majorly increases variance and unknown unknowns? The higher energy a system is and the more variables you have the more chaotic it becomes. So maybe an answer is that a agile short-range model is the best? Outside view it in moderation and plan with the next few years being quite difficult to predict?

You don't really need another model to disprove an existing one, you might as well point out that we don't know and that is okay too.

Yeah, I think you're right and I also believe that it can be a both and? 

You can have a general non-profit board and at the same time have a form of representative democracy going on which seems the best we can currently do for this?

I think it is fundamentally about a more timeless trade-off between hierarchical organisations that generally are able to act with more "commander's intent" versus democratic models that are more of a flat voting model. The democratic models suffer when there is a lot of single person linear thinking involved but do well at providing direct information for what people care about whilst the inverse is true for the hierarchical one and the project of good governance is to some extent somewhere in between.

Yeah for sure, I think the devil might be in the details here around how things are run and what the purpose of the national organisation is. Since Sweden and Norway have 8x less of a population than germany I think the effect of a "nation-wide group" might be different?

In my experience, I've found that EA Sweden focuses on and provides a lot of the things that you listed so I would be very curious to hear what the difference between a local and national organisation would be? Is there a difference in the dynamics of them being motivated to sustain themselves because of the scale? 

You probably have a lot more experience than me in this so it would be very interesting to hear!

Load more