Options

Download a copy of the Azimuth Wiki, please!

As Fukushima has shown us, even technology with multiple backup safety systems can break.

So, please download a copy of the entire Azimuth Wiki today! Go to any page and click on "Export" near the top right. If you're like me, you'll download it in both HTML and Markup format.

I think the Markup format is more useful if we ever need to reconstruct the whole Wiki in the event of a disaster. Right?

Comments

  • 1.

    Already done that, but for different reasons...the provider should have a backup strategy that is reliable enough for us, anyway, so that this Library of Alexandria thing won't happen to us :-)

    Comment Source:Already done that, but for different reasons...the provider should have a backup strategy that is reliable enough for us, anyway, so that this Library of Alexandria thing won't happen to us :-)
  • 2.

    Having backup copies brings up the question of how to efficiently keep up-to-update. I think Andrew has mentioned that the history is done using bzr: is it a better idea to make regular "pulls" from that repository?

    Comment Source:Having backup copies brings up the question of how to efficiently keep up-to-update. I think Andrew has mentioned that the history is done using bzr: is it a better idea to make regular "pulls" from that repository?
  • 3.

    I did that yesterday but just the MML.

    Comment Source:I did that yesterday but just the MML.
  • 4.

    No, no, no, please don't use the inbuilt export features! Both tie up an instiki process and use plenty of resources while the export is in progress. The HTML is the worst as instiki has to do the conversion as well. In addition, these only give you a snapshot and don't include the revision history.

    Actually, at the current size then this probably isn't a huge resource hog on Azimuth, but it doesn't scale well and we've disabled it on the nLab.

    There are two ways to keep regular backups. The first is to take copies of the database. This is certainly the easiest to recover from, but as it contains everything, it can't be distributed. Included in "everything" are the main password and if we ever have personal webs all of those will be included, including the private ones.

    The other way is via the BZR repository. Each day, the server runs a script that exports the pages to a BZR repository and each day I pull that from the server to my work machine here in Norway. The export script exports each page change as a separate commit, so everything that would be needed to reconstruct the wiki is included, barring a few stylesheet tweaks and the uploaded files (I could easily add those to a backup scheme). This system works on a web-by-web basis, and doesn't include sensitive information, so is completely safe for public distribution.

    The current size of the repository is 13Mb, which isn't too big (for comparison, the nLab is nearing 200Mb), but is a bit big to download in one go from the server so if anyone wants to start doing this, I'd prefer to send them a "seed file" to get them started. After that, doing a daily update doesn't involve much bandwidth, and remembering to do the update is easy with automated scripts.

    (Incidentally, this scheme is just something I dreamt up when thinking about how to solve this issue on the nLab; if anyone has any suggestions on how to improve it then please do say so.)

    Comment Source:No, no, no, please **don't** use the inbuilt export features! Both tie up an instiki process and use plenty of resources while the export is in progress. The HTML is the worst as instiki has to do the conversion as well. In addition, these only give you a snapshot and don't include the revision history. Actually, at the current size then this probably isn't a huge resource hog on Azimuth, but it doesn't scale well and we've disabled it on the nLab. There are two ways to keep regular backups. The first is to take copies of the database. This is certainly the easiest to recover from, but as it contains _everything_, it can't be distributed. Included in "everything" are the main password and if we ever have personal webs all of those will be included, including the private ones. The other way is via the BZR repository. Each day, the server runs a script that exports the pages to a BZR repository and each day I pull that from the server to my work machine here in Norway. The export script exports each page change as a separate commit, so _everything_ that would be needed to reconstruct the wiki is included, barring a few stylesheet tweaks and the uploaded files (I could easily add those to a backup scheme). This system works on a web-by-web basis, and doesn't include sensitive information, so is completely safe for public distribution. The current size of the repository is 13Mb, which isn't too big (for comparison, the nLab is nearing 200Mb), but is a bit big to download in one go from the server so if anyone wants to start doing this, I'd prefer to send them a "seed file" to get them started. After that, doing a daily update doesn't involve much bandwidth, and remembering to do the update is easy with automated scripts. (Incidentally, this scheme is just something I dreamt up when thinking about how to solve this issue on the nLab; if anyone has any suggestions on how to improve it then please do say so.)
  • 5.

    Hi Andrew,

    The current size of the repository is 13Mb, which isn't too big (for comparison, the nLab is nearing 200Mb), but is a bit big to download in one go from the server so if anyone wants to start doing this, I'd prefer to send them a "seed file" to get them started

    I'll try and set p bzr mirroring. However, I'd prefer not to have a huge file in my email, so would it be possible to stick the seed file on some file sharing site like dropbox? (Since we're trying to get everyone interested in Azimuth's content it's not as if potential access by others is an issue.)

    Comment Source:Hi Andrew, > The current size of the repository is 13Mb, which isn't too big (for comparison, the nLab is nearing 200Mb), but is a bit big to download in one go from the server so if anyone wants to start doing this, I'd prefer to send them a "seed file" to get them started I'll try and set p bzr mirroring. However, I'd prefer not to have a huge file in my email, so would it be possible to stick the seed file on some file sharing site like dropbox? (Since we're trying to get everyone interested in Azimuth's content it's not as if potential access by others is an issue.)
  • 6.

    David, Yes, I wouldn't send it by email! (though with compression it would probably come in a fair bit under 13Mb) There are various places that I could put it, from an Ubuntu One account to my work webspace. Let me know when you're ready and I'll set it up - also say what archive format you're most happy with.

    Comment Source:David, Yes, I wouldn't send it by email! (though with compression it would probably come in a fair bit under 13Mb) There are various places that I could put it, from an Ubuntu One account to my work webspace. Let me know when you're ready and I'll set it up - also say what archive format you're most happy with.
  • 7.
    edited March 2011

    Ah, I think I need to read up on bzr. (I'm used to git and mercurial where, except for some nasty hacks, you can't ever have a copy of anything short of the full repo history.) I'm working on Linux and I can access just about any archive format, I just need to figure out how to convert a "seed" to a bzr repo I can pull incremental updates into.

    Comment Source:Ah, I think I need to read up on bzr. (I'm used to git and mercurial where, except for some nasty hacks, you can't ever have a copy of anything short of the full repo history.) I'm working on Linux and I can access just about any archive format, I just need to figure out how to convert a "seed" to a bzr repo I can pull incremental updates into.
  • 8.

    Sorry for attempting to do it in a technically suboptimal way, Andrew. Thanks for trying to figure out a better way, David!

    Needless to say, I want a copy...

    Comment Source:Sorry for attempting to do it in a technically suboptimal way, Andrew. Thanks for trying to figure out a better way, David! <img src = "http://math.ucr.edu/home/baez/emoticons/thumbsup.gif" alt = ""/> Needless to say, I want a copy...
  • 9.

    David, the "seed" would be a full BZR repository, so you could just unpack it into a directory, do bzr update and be done. No conversion needed. My understanding is that as far as the basics are concerned, bzr is similar to git and mercurial. You can have "lightweight checkouts" but that isn't what I'm suggesting here.

    John, I think it would be very good for you to have a copy, and it would be very good if we can work out instructions that you can follow which we can put on the wiki. The starting point is to ask what operating system you're using.

    David: it is, I'm pretty sure, easy to convert between different DVCS's, even on the fly. So the fact that I use BZR as the host repository doesn't mean that others have to use BZR to download it to. For someone who is ... er ... "non-technical", such as John, is there a DVCS that is easiest to use?

    Comment Source:David, the "seed" would be a full BZR repository, so you could just unpack it into a directory, do `bzr update` and be done. No conversion needed. My understanding is that as far as the _basics_ are concerned, bzr is similar to git and mercurial. You can have "lightweight checkouts" but that isn't what I'm suggesting here. John, I think it would be very good for you to have a copy, and it would be very good if we can work out instructions that you can follow which we can put on the wiki. The starting point is to ask what operating system you're using. David: it is, I'm pretty sure, easy to convert between different DVCS's, even on the fly. So the fact that I use BZR as the host repository doesn't mean that others have to use BZR to download it to. For someone who is ... er ... "non-technical", such as John, is there a DVCS that is easiest to use?
  • 10.
    edited March 2011

    Ah, if the seed is an actual repostiory I may as well just use bzr directly. (It's not as if I'm doing anything other than pulling into it.) I don't think there's any DVCS that will be noticeably easier for general users that any of the others.

    I suspect John may be using Windows. My only real experience on windows (using git) is that with their alternate line-ending convention and many windows programs autoconverting, it's easy to end up with with what looks to the version control system like an altered file if you look at a file in an editor which then decides to autosave. That confuses the bejezus out of me, so I hate to think what general people will think.

    Comment Source:Ah, if the seed is an actual repostiory I may as well just use bzr directly. (It's not as if I'm doing anything other than pulling into it.) I don't think there's any DVCS that will be noticeably easier for general users that any of the others. I suspect John may be using Windows. My only real experience on windows (using git) is that with their alternate line-ending convention and many windows programs autoconverting, it's easy to end up with with what looks to the version control system like an altered file if you look at a file in an editor which then decides to autosave. That confuses the bejezus out of me, so I hate to think what general people will think.
Sign In or Register to comment.