Options

Azimuth Backup Project

The Azimuth Backup Project is very active now. Most of the work is happening not here but on email, the blog and this new site:

I'm about to start a Kickstarter campaign, and the main advertisement may be at this location on the Wiki:

Comments

  • 1.

    This project would interest me if the data was stored in a format better suited for modern analysis. Use a triple-store for example and attach it to an ontology as linked open data. But only people that have used this approach in the past and seen the potential would be on board.

    Comment Source:This project would interest me if the data was stored in a format better suited for modern analysis. Use a triple-store for example and attach it to an ontology as [linked open data](https://en.wikipedia.org/wiki/Linked_data). But only people that have used this approach in the past and seen the potential would be on board.
  • 2.
    edited December 2016

    I wonder if Google could be encouraged to make and keep a backup of all of the available US data. They are perhaps the group that can do this most easily. They may already have a copy of much of it, whatever is on the web, given that they cache so much data. Keeping the data would be in the spirit of its vision "to organize the world's information and make it universally accessible and useful." And it could be an excellent business investment should the US government ever delete its data.

    Comment Source:I wonder if Google could be encouraged to make and keep a backup of all of the available US data. They are perhaps the group that can do this most easily. They may already have a copy of much of it, whatever is on the web, given that they cache so much data. Keeping the data would be in the spirit of its vision "to organize the world's information and make it universally accessible and useful." And it could be an excellent business investment should the US government ever delete its data.
  • 3.

    The Azimuth Backup Project has a member that works for Google: Scott Maxwell. He's considered talking to the big bosses of Google and seeing if they'd be interested in backing up climate data... but apparently they are not so easy to talk to, for fairly obvious reasons.

    Luckily I seem to have persuaded the head of Computing and Communications at my own university, the University of California at Riverside, to permanently store the data we are backing up.

    I believe the usual Google caches of websites are not sufficient to save the data we're talking about.

    I'd be happy for someone to study this. If you go to this important National Oceanic and Atmospheric Administration database:

    https://ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html

    click on the variable WaterTemp and "Get Binary", you'll get a file of data. The Internet Archive has backed up this webpage:

    http://web.archive.org/web/20161218071345/https://ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html

    but when you click on the variable WaterTemp and "Get Binary", you get an error message. All the data is missing! And when I search for this website on the End of Term Web Archive:

    http://eotarchive.cdlib.org/search?browse-all=yes

    I get nothing at all. If I go to the Google cache:

    http://webcache.googleusercontent.com/search?q=cache:-yNErynXsqgJ:ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html+&cd=1&hl=en&ct=clnk&gl=us&client=ubuntu

    I do get the data. But:

    1) is the data coming from the cache or the original site?

    2) if the original site is deleted or changed, how long will the cache last?

    Of course Google could take action to save the caches. But for that we would need someone to persuade them to do so.

    Comment Source:The Azimuth Backup Project has a member that works for Google: Scott Maxwell. He's considered talking to the big bosses of Google and seeing if they'd be interested in backing up climate data... but apparently they are not so easy to talk to, for fairly obvious reasons. Luckily I seem to have persuaded the head of Computing and Communications at my own university, the University of California at Riverside, to permanently store the data we are backing up. I believe the usual Google caches of websites are not sufficient to save the data we're talking about. I'd be happy for someone to study this. If you go to this important National Oceanic and Atmospheric Administration database: [https://ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html](https://ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html) click on the variable WaterTemp and "Get Binary", you'll get a file of data. The Internet Archive has backed up this webpage: [http://web.archive.org/web/20161218071345/https://ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html](http://web.archive.org/web/20161218071345/https://ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html) but when you click on the variable WaterTemp and "Get Binary", you get an error message. All the data is missing! And when I search for this website on the End of Term Web Archive: [http://eotarchive.cdlib.org/search?browse-all=yes](http://eotarchive.cdlib.org/search?browse-all=yes) I get nothing at all. If I go to the Google cache: [http://webcache.googleusercontent.com/search?q=cache:-yNErynXsqgJ:ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html+&cd=1&hl=en&ct=clnk&gl=us&client=ubuntu](http://webcache.googleusercontent.com/search?q=cache:-yNErynXsqgJ:ecowatch.ncddc.noaa.gov/thredds/dodsC/hycom/hycom_reg1_agg/HYCOM_Region_1_Aggregation_best.ncd.html+&cd=1&hl=en&ct=clnk&gl=us&client=ubuntu) I _do_ get the data. But: 1) is the data coming from the cache or the original site? 2) if the original site is deleted or changed, how long will the cache last? Of course Google could take action to save the caches. But for that we would need someone to persuade them to do so.
  • 4.

    Probably missing CGI parameters in the archive request. I almost feel like shaking my head about all the details needed to get right in this effort.

    Comment Source:Probably missing CGI parameters in the archive request. I almost feel like shaking my head about all the details needed to get right in this effort.
  • 5.

    John, Congratulations on meeting and surpassing (doubling!) your Kickstarter goal by https://www.kickstarter.com/projects/592742410/azimuth-climate-data-backup-project and you have 17 days left. Great to hear that UC Riverside is supporting your vision. Congratulations also on you and your team's advances in network theory.

    I think that your endeavor is easy for Google to support, relevant to its mission, AND a business opportunity if the US government should ever delete its data.

    I sent a note through LinkedIn to Erikas Napjus, who once was Director of Global Infrastructure Acquisition at Google. I don't know him, just his parents. I could try some other acquaintances in Silicon Valley.

    Comment Source:John, Congratulations on meeting and surpassing (doubling!) your Kickstarter goal by https://www.kickstarter.com/projects/592742410/azimuth-climate-data-backup-project and you have 17 days left. Great to hear that UC Riverside is supporting your vision. Congratulations also on you and your team's advances in network theory. I think that your endeavor is easy for Google to support, relevant to its mission, AND a business opportunity if the US government should ever delete its data. I sent a note through LinkedIn to Erikas Napjus, who once was Director of Global Infrastructure Acquisition at Google. I don't know him, just his parents. I could try some other acquaintances in Silicon Valley.
  • 6.

    Thanks! We've now gone above 3 times our original funding goal, and the big question is: what's the best way to spend this money? It's not really a lot of money, so we have to be careful with it. If anyone has good ideas, please post them here:

    Comment Source:Thanks! We've now gone above 3 times our original funding goal, and the big question is: what's the best way to spend this money? It's not really a lot of money, so we have to be careful with it. If anyone has good ideas, please post them here: * John Baez, [Azimuth backup project (part 3)](https://johncarlosbaez.wordpress.com/2017/01/22/azimuth-backup-project-part-3/), 22 January 2017.
Sign In or Register to comment.