Options

Decisions, curation and trust

Decisions

As we go forward, the group will get larger and the issues will start to become clearer.

At some point very soon, it will become obvious that there are certain things that we ought to be doing, the "no brainers," because they will have clear advantages and be obviously better than alternative ideas. Other ideas will be more controversial. Some things will be obviously stupid: "like subsidizing ethanol production in the U.S.," for example.

To create a real plan, or even parts of one, we're going to need to make decisions. These decisions can't be all made by John as he won't be able to keep up with all the different aspects of the plan as it grows. No one human being could.

So it seems like we ought to talk about how these decisions will get made in the future.

Curation

Another aspect of decision-making is the act of curation. Deciding which plans from others to highlight, critiquing them, pointing to science that makes sense and is relevant, not pointing to science or technology that is only a distraction. Deciding who gets to post on the blog and what subjects get presented.

There are many places on the web, where one can get a steady stream of news with varying degrees of importance and relevance. There are few places you can go that curate well. Places where the signal-to-noise ration is very high. This is a role that editors for the best newspapers fill.

Building a Network of Trust

It is my personal belief that the decisions for the plans and curation should be made on the basis of trust. That we cannot and will not be successful in this endeavor unless there is complete trust among the people who are in charge.

I'm new, so I don't expect anyone to trust me yet. But there are probably some people that John knows from their interactions here and elsewhere that he already trusts. It seems to me that those people are the most important people to be making decisions.

It also seems to me that trust is domain-specific. You might trust someone's intentions but not their judgement. You might trust their statements and observations but not their ability to evaluate a particular area of science. Each of us has strengths and weaknesses; and different experience and abilities we bring to bear on any given problem.

Openness and Trust

So perhaps in the spirit of openness, we ought to be explicit about this. Make it be known to the project and others who is trusted to do what and over which domain and even more importantly why they have that trust. I also think that an explicit trust linkage ought to flow out from John, i.e. for anyone in the organization that is trusted to make decisions or curate, you ought to be able to see a direct linkage from John to that person, one that shows the various intermediary people, and what they have done to earn the trust. And for people who are trusted enough to confer that trust on others, we should show what they have done to demonstrate that they too are are capable of evaluating and conferring trust.

I also think that it makes sense to articulate and make clear how the group works as a social construct, how it works now, how we expect that to change over time, and how new people can join in and help.

In my particular case, I think it makes sense for my work and ideas to be judged on their own merits since I don't have a preexisting relationship with John or anyone else here. But for new people we actively recruit, I think the question of: "Do I trust this individual?" ought to be the primary criterion, not: "How much do they know?," or "How well-respected are they?" And even more to the core of the scalability of this approach: "Do I trust this person to bring into the project only those people that I will end up trusting as well as soon as I get to know them?"

After all, the question you are really asking is: "When calamity comes and chaos reigns, is this person someone you trust to be helping decide the fate of the world?"


What do you all think?

How should we make decisions now?

How does that change in the future?

Do you think trust is this important?

Comments

  • 1.

    I need to get some other work done so I can't comment on this thoroughly right now.

    I also think that an explicit trust linkage ought to flow out from John, i.e. for anyone in the organization that is trusted to make decisions or curate, you ought to be able to see a direct linkage from John to that person...

    I have always believed this is the right way to go , but until I become dictator of the universe I don't think everyone will agree.

    Seriously, I don't think this the way to go. If we really want to formalize this business of trust, I urge a self-organizing reputation system as practiced quite successfully by Mathoverflow and Less Wrong. I urge everyone who cares about this issue to look at these websites and see how these reputation systems work in practice. One can tweak the algorithms in many ways.

    I believe that I could maintain a high reputation if the Azimuth Project adopted such a system, but a great advantage of this over a system where I'm automatically considered central is that it would keep me on my toes, and reduce the chance that I start acting like a jerk.

    Comment Source:I need to get some other work done so I can't comment on this thoroughly right now. > I also think that an explicit trust linkage ought to flow out from John, i.e. for anyone in the organization that is trusted to make decisions or curate, you ought to be able to see a direct linkage from John to that person... I have always believed this is the right way to go <img src = "http://math.ucr.edu/home/baez/emoticons/biggrin.gif" alt = ""/>, but until I become dictator of the universe I don't think everyone will agree. Seriously, I don't think this the way to go. If we really want to formalize this business of trust, I urge a self-organizing reputation system as practiced quite successfully by [Mathoverflow](http://mathoverflow.net/faq#reputation) and [Less Wrong](http://lesswrong.com/). I urge everyone who cares about this issue to look at these websites and see how these reputation systems work in practice. One can tweak the algorithms in many ways. I believe that I could maintain a high reputation if the Azimuth Project adopted such a system, but a great advantage of this over a system where I'm _automatically_ considered central is that it would keep me on my toes, and reduce the chance that I start acting like a jerk.
  • 2.

    I have always believed this is the right way to go , but until I become dictator of the universe I don't think everyone will agree.

    Seriously, I don't think this the way to go. If we really want to formalize this business of trust, I urge a self-organizing reputation system as practiced quite successfully by Mathoverflow and Less Wrong. I urge everyone who cares about this issue to look at these websites and see how these reputation systems work in practice. One can tweak the algorithms in many ways.

    I'm pleased that you don't want to run a dictatorship. I didn't think you would.

    The point I was trying to make about you being the source of trust in the trust network really applies to the beginning. As soon as you feel comfortable, it would be better to have a mechanism that was more like the Mathoverflow and Less Wrong sites.

    I'll add one important caveat. I think it is important to have hierarchies for making decisions. Reputation is only an approximation for trust. I don't think you can do this mechanically as well as you can do it using softer metrics. Some people on these sites get reputation through hard work but they don't know as much as others and their decisions are not as good. There needs to be some mechanisms for incorporating more subjective measures like the respect of peers.

    Full democracy doesn't work very well for large projects because it is too messy, decisions take too long, and politicians win when less vocal people might be better.

    I should say that I don't believe in power hierarchies. They tend to breed corruption and they don't work as well with bright independent people.

    Over the years, my wife has convinced me that hiearchies are important because most people don't have enough knowledge to make decisions that impact areas outside their specialty and decisions need to be made. So I do believe in leadership and responsibility hiearchies. It makes it much easier to make sure people are working together effectively an not duplicating work. And many people don't want to make decisions. They just want a list of things they can do to help. They don't want to lead.

    I haven't seen this done anywhere—except informally in the best high-tech companies—but I've come to believe that the best approach is one where the hierarchy itself is self-assembling and organic; one that maximizes the autonomy and free will of the participants. For instance, someone who works on the project is free to work with anyone else who similarly agrees. So Jim can work on a project Mary runs if Mary agrees. So no one has to work for someone they don't respect and no one has to lead someone they don't have time for or don't trust.

    Some people will attract more followers because they are better leaders or because they have more interesting projects. At some point it becomes unwieldy for the leader to help everyone, at this point the leader proposes a change in the hierarchy and the followers get to agree with the change, propose something different, or decide to work under the leadership of someone else.

    The main idea here is what you acknowledged elsewhere: "You can't tell people what to do." So you don't. The project leaders set the ground rules for letting each individual decide what to do and how they can work within the organization. New people follow those rules and join in.

    In the case of the Azimuth Project right now, you can probably manage all the interactions—with the exception of software development which is not your bailiwick. As the organization grows, you'll want to interact with a subset of the growing project team out of practical necessity.

    Comment Source:>I have always believed this is the right way to go <img src = "http://math.ucr.edu/home/baez/emoticons/biggrin.gif" alt = ""/>, but until I become dictator of the universe I don't think everyone will agree. >Seriously, I don't think this the way to go. If we really want to formalize this business of trust, I urge a self-organizing reputation system as practiced quite successfully by [Mathoverflow](http://mathoverflow.net/faq#reputation) and [Less Wrong](http://lesswrong.com/). I urge everyone who cares about this issue to look at these websites and see how these reputation systems work in practice. One can tweak the algorithms in many ways. I'm pleased that you don't want to run a dictatorship. I didn't think you would. The point I was trying to make about you being the source of trust in the trust network really applies to the beginning. As soon as you feel comfortable, it would be better to have a mechanism that was more like the [Mathoverflow](http://mathoverflow.net/faq#reputation) and [Less Wrong](http://lesswrong.com/) sites. I'll add one important caveat. I think it is important to have hierarchies for making decisions. Reputation is only an approximation for trust. I don't think you can do this mechanically as well as you can do it using softer metrics. Some people on these sites get reputation through hard work but they don't know as much as others and their decisions are not as good. There needs to be some mechanisms for incorporating more subjective measures like the respect of peers. Full democracy doesn't work very well for large projects because it is too messy, decisions take too long, and politicians win when less vocal people might be better. I should say that I don't believe in power hierarchies. They tend to breed corruption and they don't work as well with bright independent people. Over the years, my wife has convinced me that hiearchies are important because most people don't have enough knowledge to make decisions that impact areas outside their specialty and decisions need to be made. So I do believe in leadership and responsibility hiearchies. It makes it much easier to make sure people are working together effectively an not duplicating work. And many people don't want to make decisions. They just want a list of things they can do to help. They don't want to lead. I haven't seen this done anywhere—except informally in the best high-tech companies—but I've come to believe that the best approach is one where the hierarchy itself is self-assembling and organic; one that maximizes the autonomy and free will of the participants. For instance, someone who works on the project is free to work with anyone else who similarly agrees. So Jim can work on a project Mary runs if Mary agrees. So no one has to work for someone they don't respect and no one has to lead someone they don't have time for or don't trust. Some people will attract more followers because they are better leaders or because they have more interesting projects. At some point it becomes unwieldy for the leader to help everyone, at this point the leader proposes a change in the hierarchy and the followers get to agree with the change, propose something different, or decide to work under the leadership of someone else. The main idea here is what you acknowledged elsewhere: "You can't tell people what to do." So you don't. The project leaders set the ground rules for letting each individual decide what to do and how they can work within the organization. New people follow those rules and join in. In the case of the Azimuth Project right now, you can probably manage all the interactions—with the exception of software development which is not your bailiwick. As the organization grows, you'll want to interact with a subset of the growing project team out of practical necessity.
  • 3.
    edited January 2011

    The point I was trying to make about you being the source of trust in the trust network really applies to the beginning. As soon as you feel comfortable, it would be better to have a mechanism that was more like the Mathoverflow and Less Wrong sites.

    I think it's not quite time for this yet, but I think Andrew Stacey may have suggested that it's technically possible to do something like this on the Azimuth Forum, so maybe in a while.

    (I don't quite remember what he said.)

    In the case of the Azimuth Project right now, you can probably manage all the interactions—with the exception of software development which is not your bailiwick. As the organization grows, you'll want to interact with a subset of the growing project team out of practical necessity.

    I really look forward to this transition, though it will be rather unnerving.

    Comment Source:> The point I was trying to make about you being the source of trust in the trust network really applies to the beginning. As soon as you feel comfortable, it would be better to have a mechanism that was more like the [Mathoverflow](http://mathoverflow.net/faq#reputation) and [Less Wrong](http://lesswrong.com/) sites. I think it's not quite time for this yet, but I think Andrew Stacey may have suggested that it's technically possible to do something like this on the Azimuth Forum, so maybe in a while. (I don't quite remember what he said.) > In the case of the Azimuth Project right now, you can probably manage all the interactions—with the exception of software development which is not your bailiwick. As the organization grows, you'll want to interact with a subset of the growing project team out of practical necessity. I really look forward to this transition, though it will be rather unnerving.
  • 4.

    I think it's not quite time for this yet, but I think Andrew Stacey may have suggested that it's technically possible to do something like this on the Azimuth Forum, so maybe in a while.

    (I don't quite remember what he said.)

    It's technically possible to do most anything we want. At some point, I suspect we're going to want to merge technologies and integrate the technology behind the Wiki and forum so that we can more tightly integrate the functionality itself. This will only make sense down the road when we have more volunteers and a clearer vision for what we want that isn't possible with the combined Ruby + Rails based Instiki Wiki and the PHP-based Vanilla forums.

    The nice thing about the choices that have been made is that both the Instiki Wiki software and the Vanilla forum software are simple systems without a lot extra features we don't need, so transitioning will be fairly straightforward if we need to in the future.

    But what I was thinking about when I say like Mathoverflow or Less Wrong or even the relatively new Physics Stack Exchange is the same basic idea but with two important differences:

    1) There needs to be some sort of hierarchy for decisions that must be made quickly and for specific domains. If this project takes off there will be many different domains of expertise and someone with 2,000 points because of their programming expertise is not necessarily the best person to be making decisions about the relevance of a particular climate science report they are not qualified to judge.

    2) Likewise, you may have someone join in and you want them to participate at a higher level from the start because of their prior expertise and because of trust developed in the real-world. So there ought to be some way of conferring trust provisionally for these sorts of people.

    Finally, I think it is best to put human faces on the decisions whenever possible. The other software makes this all invisible, so you don't know who voted down questions, for example, unless the voters say in a comment.

    If you want to avoid the group writing problems of Wikipedia there also needs to be some editors/curators making sure that the most visible pages are up to snuff. Having everyone have equal access works better for some things than others. Again, I think we are going to want a hybrid of human and machine generated access controls.

    Comment Source:>I think it's not quite time for this yet, but I think Andrew Stacey may have suggested that it's technically possible to do something like this on the Azimuth Forum, so maybe in a while. >(I don't quite remember what he said.) It's technically possible to do most anything we want. At some point, I suspect we're going to want to merge technologies and integrate the technology behind the Wiki and forum so that we can more tightly integrate the functionality itself. This will only make sense down the road when we have more volunteers and a clearer vision for what we want that isn't possible with the combined Ruby + Rails based Instiki Wiki and the PHP-based Vanilla forums. The nice thing about the choices that have been made is that both the Instiki Wiki software and the Vanilla forum software are simple systems without a lot extra features we don't need, so transitioning will be fairly straightforward if we need to in the future. But what I was thinking about when I say like [Mathoverflow](http://mathoverflow.net/faq#reputation) or [Less Wrong](http://lesswrong.com/) or even the relatively new [Physics Stack Exchange](http://physics.stackexchange.com/) is the same basic idea but with two important differences: 1) There needs to be some sort of hierarchy for decisions that must be made quickly and for specific domains. If this project takes off there will be many different domains of expertise and someone with 2,000 points because of their programming expertise is not necessarily the best person to be making decisions about the relevance of a particular climate science report they are not qualified to judge. 2) Likewise, you may have someone join in and you want them to participate at a higher level from the start because of their prior expertise and because of trust developed in the real-world. So there ought to be some way of conferring trust provisionally for these sorts of people. Finally, I think it is best to put human faces on the decisions whenever possible. The other software makes this all invisible, so you don't know who voted down questions, for example, unless the voters say in a comment. If you want to avoid the group writing problems of Wikipedia there also needs to be some editors/curators making sure that the most visible pages are up to snuff. Having everyone have equal access works better for some things than others. Again, I think we are going to want a hybrid of human and machine generated access controls.
  • 5.
    edited January 2011

    John said:

    I think it's not quite time for this yet, but I think Andrew Stacey may have suggested that it's technically possible to do something like this on the Azimuth Forum, so maybe in a while.

    Curtis responded:

    It's technically possible to do most anything we want.

    What Curtis means is that Instiki is based on a flexible and extentable framework, so if you have enough manpower you can extend Instiki to do anything that any other webapplication can do, too. When Andrew says something is technically possible, I think he means that it would be possible to be done by himself, given his very limited amount of time :-)

    Comment Source:John said: <blockquote> <p> I think it's not quite time for this yet, but I think Andrew Stacey may have suggested that it's technically possible to do something like this on the Azimuth Forum, so maybe in a while. </p> </blockquote> Curtis responded: <blockquote> <p> It's technically possible to do most anything we want. </p> </blockquote> What Curtis means is that Instiki is based on a flexible and extentable framework, so if you have enough manpower you can extend Instiki to do anything that any other webapplication can do, too. When Andrew says something is technically possible, I think he means that it would be possible to be done by himself, given his very limited amount of time :-)
  • 6.

    What Tim said ;)

    Comment Source:What Tim said ;)
  • 7.

    Curtis said:

    But what I was thinking about when I say like Mathoverflow or Less Wrong or even the relatively new Physics Stack Exchange is the same basic idea but with two important differences:

    1) There needs to be some sort of hierarchy for decisions that must be made quickly and for specific domains. If this project takes off there will be many different domains of expertise and someone with 2,000 points because of their programming expertise is not necessarily the best person to be making decisions about the relevance of a particular climate science report they are not qualified to judge.

    Methinks that, too: Instead of one "trust number" a whole set would be better, perhaps organized in a graph, with connected nodes summing up to total trust numbers etc.

    Which reminds me of the nLab etc. being an old dream of mine come true: The Internet Academy! Well, almost. What's still a dream is an interactive graphical representation of the content - plus, a graphical representation of the academy members' expertise.

    That's where I'd love to contribute my programming skills (provided I find time and the hominids around me let me be...)

    Comment Source:Curtis said: >But what I was thinking about when I say like [Mathoverflow](http://mathoverflow.net/faq#reputation) or [Less Wrong](http://lesswrong.com/) or even the relatively new [Physics Stack Exchange](http://physics.stackexchange.com/) is the same basic idea but with two important differences: >1) There needs to be some sort of hierarchy for decisions that must be made quickly and for specific domains. If this project takes off there will be many different domains of expertise and someone with 2,000 points because of their programming expertise is not necessarily the best person to be making decisions about the relevance of a particular climate science report they are not qualified to judge. Methinks that, too: Instead of one "trust number" a whole set would be better, perhaps organized in a graph, with connected nodes summing up to total trust numbers etc. Which reminds me of the nLab etc. being an old dream of mine come true: The Internet Academy! Well, almost. What's still a dream is an interactive graphical representation of the content - plus, a graphical representation of the academy members' expertise. That's where I'd love to contribute my programming skills (provided I find time and the hominids around me let me be...)
  • 8.

    All this stuff sounds great - I hope you guys try some of it. I think for now I'll focus on things I'm better at. But if our project succeeds, the number of people involved will grow until we need better ways to keep track of who to trust for what purposes...

    Comment Source:All this stuff sounds great - I hope you guys try some of it. I think for now I'll focus on things I'm better at. But if our project succeeds, the number of people involved will grow until we need better ways to keep track of who to trust for what purposes...
  • 9.

    I agree whole-heartedly with most of the opening comments by Curtis. It is true we will grow. It is true that trust and trust relationships are required - to insure healthy growth. I'm glad John did not consent to be dictator, but I would not want to assume that role either. I did have occasion, recently, to raise related questions and concerns in another thread, which I should mention here.

    In a Forum thread on Spam for "Cash Registers", I tell the tale of finding a link on the Azimuth Home Page one day - for Cash Registers, and promptly deleting it. Of course; finding such a link right on the front page made me wonder if perhaps there should be locked pages, or assigned privileges for trusted editors, and so on. Perhaps there should be pages AnonymousCoward can't edit, or that require a Member ID.

    As more and more of the Main Pages get built - which link to many sub-topics - it may be wise to mark some of them 'official,' at least in terms of their editability. The templates page, for example, has seen some recent tweaking - but could be locked in a fixed or stable condition, as it should never need to be changed once it is correct-as-is.

    That said; it would be nice to have some discussion on the forum, as to the content on pages that are being locked, on which pages should be locked, or what level of privilege should be required to edit various pages. In my opinion, the majority of pages could or should remain pages anyone can edit. But it might be good to qualify that somewhat.

    Perhaps we need three levels - anyone can edit, any member can edit, only administrators can edit. It would be nice to have known trusted, known thoughtful, or known expert individuals in charge of sensitive areas of the site. I would have thought the Home Page would already be one that only authorized parties could change, but I found out otherwise. The main thing, in my mind, is to somehow assure that people editing our Wiki are acting in good faith.

    Finding ways to help assure that should not be impossible.

    Regards,
    Jonathan

    Comment Source:I agree whole-heartedly with most of the opening comments by Curtis. It is true we will grow. It is true that trust and trust relationships are required - to insure healthy growth. I'm glad John did not consent to be dictator, but I would not want to assume that role either. I did have occasion, recently, to raise related questions and concerns in another thread, which I should mention here. In a Forum thread on [Spam for "Cash Registers"](http://www.math.ntnu.no/~stacey/Mathforge/Azimuth/comments.php?DiscussionID=523), I tell the tale of finding a link on the Azimuth Home Page one day - for Cash Registers, and promptly deleting it. Of course; finding such a link right on the front page made me wonder if perhaps there should be locked pages, or assigned privileges for trusted editors, and so on. Perhaps there should be pages AnonymousCoward can't edit, or that require a Member ID. As more and more of the Main Pages get built - which link to many sub-topics - it may be wise to mark some of them 'official,' at least in terms of their editability. The templates page, for example, has seen some recent tweaking - but could be locked in a fixed or stable condition, as it should never need to be changed once it is correct-as-is. That said; it would be nice to have some discussion on the forum, as to the content on pages that are being locked, on which pages should be locked, or what level of privilege should be required to edit various pages. In my opinion, the majority of pages could or should remain pages anyone can edit. But it might be good to qualify that somewhat. Perhaps we need three levels - anyone can edit, any member can edit, only administrators can edit. It would be nice to have known trusted, known thoughtful, or known expert individuals in charge of sensitive areas of the site. I would have thought the Home Page would already be one that only authorized parties could change, but I found out otherwise. The main thing, in my mind, is to somehow assure that people editing our Wiki are acting in good faith. Finding ways to help assure that should not be impossible. Regards, Jonathan
  • 10.

    Jonathan

    That said; it would be nice to have some discussion on the forum, as to the content on pages that are being locked, on which pages should be locked, or what level of privilege should be required to edit various pages. In my opinion, the majority of pages could or should remain pages anyone can edit. But it might be good to qualify that somewhat.

    This issue will become important in time if our project is successful. If we succeed, ultimately some of our pages will become important enough and receive enough attacks and/or well-meaning but incorrect "improvements" that we'll want to lock them.

    But I don't want to spend much time thinking about this yet! At present, almost our pages need to be vastly improved, and Andrew Stacey's existing protective measures, combined with vigilance on our part, seem sufficient.

    If you want to discuss the technical options available to us, please do that in the category "Technical" so Andrew sees it. (I guess you already are).

    If you want to become a lab elf, you can do us a great favor by clicking on Recently Revised every day or two, and seeing if anything suspicious has occurred.

    Comment Source:Jonathan > That said; it would be nice to have some discussion on the forum, as to the content on pages that are being locked, on which pages should be locked, or what level of privilege should be required to edit various pages. In my opinion, the majority of pages could or should remain pages anyone can edit. But it might be good to qualify that somewhat. This issue will become important in time if our project is successful. If we succeed, ultimately some of our pages will become important enough and receive enough attacks and/or well-meaning but incorrect "improvements" that we'll want to lock them. But I don't want to spend much time thinking about this yet! At present, almost our pages need to be vastly improved, and Andrew Stacey's existing protective measures, combined with vigilance on our part, seem sufficient. If you want to discuss the technical options available to us, please do that in the category "Technical" so Andrew sees it. (I guess you already are). If you want to become a [[lab elf]], you can do us a great favor by clicking on [Recently Revised](http://www.azimuthproject.org/azimuth/recently_revised) every day or two, and seeing if anything suspicious has occurred.
  • 11.

    Yes. For now, so long as one of us checks "Recently Revised" every so often, that is probably adequate watchkeeping.

    John is right. At this point we have so many unwritten or unfinished pages that we need to make editing as easy as possible. Roadblocks to that are to be avoided. But as Curtis suggests, it is better to start thinking about some of the pending issues before they become problems.

    I agree with the comments above. No action is called for, to limit access yet, but locking down certain pages once they are perfected may yet be a good thing. We are better to shelve this issue for now, but wise to consider it.

    Comment Source:Yes. For now, so long as one of us checks "Recently Revised" every so often, that is probably adequate watchkeeping. John is right. At this point we have so many unwritten or unfinished pages that we need to make editing as easy as possible. Roadblocks to that are to be avoided. But as Curtis suggests, it is better to start thinking about some of the pending issues before they become problems. I agree with the comments above. No action is called for, to limit access yet, but locking down certain pages once they are perfected may yet be a good thing. We are better to shelve this issue for now, but wise to consider it.
Sign In or Register to comment.