It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 2.4K
- Chat 505
- Study Groups 21
- Petri Nets 9
- Epidemiology 4
- Leaf Modeling 2
- Review Sections 9
- MIT 2020: Programming with Categories 51
- MIT 2020: Lectures 20
- MIT 2020: Exercises 25
- Baez ACT 2019: Online Course 339
- Baez ACT 2019: Lectures 79
- Baez ACT 2019: Exercises 149
- Baez ACT 2019: Chat 50
- UCR ACT Seminar 4
- General 75
- Azimuth Code Project 111
- Statistical methods 4
- Drafts 10
- Math Syntax Demos 15
- Wiki - Latest Changes 3
- Strategy 113
- Azimuth Project 1.1K
- - Spam 1
- News and Information 148
- Azimuth Blog 149
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 719

Options

Hello John and Hello to everyone

What if the human intuition and vision is too specific to cognize patterns in data, what if we used machine vision to identify 'keypoints' as features in temperature data (per day) and then use machine learning to cluster these keypoints (derivatives taken on the splined original data no normalization):

Cluster Vision: Laplacian Surface Temperatures 2010

Paul check out the Himalayas.

Cluster Vision: dtdx Surface Temperatures 2010

Mahler patterns are visible, so they are not artifacts of normalization.

Cluster Vision: dtdx Surface Temperatures 2010

In particular SURF keypoints, does not have to be SURF or SIFT we could choose any or make our own:

## Comments

In CDF form these videos could be stepped though day by day, you could also do that in Youtube. As you can see the images SURF keypoints are few about 20 or so, therefore we could collect them and their clusters, we might be able to use these points for extended periods of time to conduct classifications and weather patterns, as opposed to using the entire data sets.

Unfortunately we could examine these sorts of ideas only after building prototypes and playing around with them.

Dara

`In CDF form these videos could be stepped though day by day, you could also do that in Youtube. As you can see the images SURF keypoints are few about 20 or so, therefore we could collect them and their clusters, we might be able to use these points for extended periods of time to conduct classifications and weather patterns, as opposed to using the entire data sets. Unfortunately we could examine these sorts of ideas only after building prototypes and playing around with them. Dara`

So SIFT/SURF features are designed to find gradient extremum patterns which are highly robust to (ie, almost invariant to) affine transformations (which are a big problem when trying to match points between real-world images). Is it obvious that this is a relevant criterion for use in data mining from statically co-ordinatized datasets?

`So SIFT/SURF features are designed to find gradient extremum patterns which are highly robust to (ie, almost invariant to) affine transformations (which are a big problem when trying to match points between real-world images). Is it obvious that this is a relevant criterion for use in data mining from statically co-ordinatized datasets?`

YES! So FIRST you run the computer vision algorithm get these points (or regions) and SECOND you run on the latter the machine learning algorithms e.g. clustering classification and FINALLY forecast. The advantages are that the points selected are fewer and most relevant.

Do not know what you mean 100% but I see either this method or brute force use all the pixels in the data

`>So SIFT/SURF features are designed to find gradient extremum patterns which are highly robust to (ie, almost invariant to) affine transformations (which are a big problem when trying to match points between real-world images). YES! So FIRST you run the computer vision algorithm get these points (or regions) and SECOND you run on the latter the machine learning algorithms e.g. clustering classification and FINALLY forecast. The advantages are that the points selected are fewer and most relevant. >Is it obvious that this is a relevant criterion for use in data mining from statically co-ordinatized datasets? Do not know what you mean 100% but I see either this method or brute force use all the pixels in the data`

These are the computer vision algorithms which runs fairly fast in Mathematica:

Computer Vision in Mathematica

Obviously we could add our own if needed.

SO you could get an idea

D

`These are the computer vision algorithms which runs fairly fast in Mathematica: [Computer Vision in Mathematica](http://reference.wolfram.com/language/guide/ComputerVision.html) Obviously we could add our own if needed. SO you could get an idea D`

So the point I was making is that SIFT/SURF are designed to be "robust to affine changes" rather than "interesting" -- SIFT/SURF will throw out patterns that are more interesting in favour of ones that should be more robust. It's not clear that this robustness is useful. It might be that going back to older computer vision interest points which aren't as useful for actual computer vision -- because they're not robust to camera motion -- for the purpose of doing interest point detection in statically gridded data. (If there was some reason to believe affine transformations were going on in the data SIFT/SURF might be good.)

`So the point I was making is that SIFT/SURF are designed to be "robust to affine changes" rather than "interesting" -- SIFT/SURF will throw out patterns that are more interesting in favour of ones that should be more robust. It's not clear that this robustness is useful. It might be that going back to older computer vision interest points which aren't as useful for actual computer vision -- because they're not robust to camera motion -- for the purpose of doing interest point detection in statically gridded data. (If there was some reason to believe affine transformations were going on in the data SIFT/SURF might be good.)`

I could not say that. After all what is INTERESTING? Someone looking at 150MB of volumetric data and finds a point in the middle and feel something interesting about it?

Again I have no idea what ROBUST means.

You see we have downloaded chuck full of GPM satellite data on rain and wind and temperature and so on, one orbital of data is HUGE! and all of it look to me interesting and robust.

The classic mistake of all machine learning developers is that they have a priori concept of something of interest or how things should come out in forecast! The idea is to let the data and code show us the way, than preconcieved notions.

I won't push you to think this way or that way, that is your prerogative. But I am faced with terabytes of data per hour in a few days from one group of satellites.

Dara

D

`>So the point I was making is that SIFT/SURF are designed to be “robust to affine changes” rather than “interesting” – SIFT/SURF will throw out patterns that are more interesting in favour of ones that should be more robust. I could not say that. After all what is INTERESTING? Someone looking at 150MB of volumetric data and finds a point in the middle and feel something interesting about it? Again I have no idea what ROBUST means. You see we have downloaded chuck full of GPM satellite data on rain and wind and temperature and so on, one orbital of data is HUGE! and all of it look to me interesting and robust. The classic mistake of all machine learning developers is that they have a priori concept of something of interest or how things should come out in forecast! The idea is to let the data and code show us the way, than preconcieved notions. I won't push you to think this way or that way, that is your prerogative. But I am faced with terabytes of data per hour in a few days from one group of satellites. Dara D`

By "robust to affine changes" I mean the goal of the SURF interest point operator is to find a patch P in an image such that

SURF(P) approximately equals SURF(A(P))

(same for SIFT) where A is an affine mapping. This is a good design goal for image processing because local patches in the 3-D world undergo local affine transformations in the camera images as the camera moves. There's a lot of "interesting" structures in images that SURF decides to throw away because it can't make it fit the equation above. In other words, the points SURF gives you may well be interesting, but there are also going to be lots of interesting points it won't give you.

I was just suggesting that it might be useful to try older "interest point operators" that don't include affine invariance in their requirements.

`By "robust to affine changes" I mean the goal of the SURF interest point operator is to find a patch P in an image such that SURF(P) approximately equals SURF(A(P)) (same for SIFT) where A is an affine mapping. This is a good design goal for image processing because local patches in the 3-D world undergo local affine transformations in the camera images as the camera moves. There's a lot of "interesting" structures in images that SURF decides to throw away because it can't make it fit the equation above. In other words, the points SURF gives you may well be interesting, but there are also going to be lots of interesting points it won't give you. I was just suggesting that it might be useful to try older "interest point operators" that don't include affine invariance in their requirements.`

Could you give some examples? please.

Paul's sloshing concepts or wind/water streams are robust, for that matter anything else discussed in this group, since they are affine transformation independent. Sloshing is sloshing no matter how you rotate or scale the data set.

`>There’s a lot of “interesting” structures in images that SURF decides to throw away because it can’t make it fit the equation above. Could you give some examples? please. Paul's sloshing concepts or wind/water streams are robust, for that matter anything else discussed in this group, since they are affine transformation independent. Sloshing is sloshing no matter how you rotate or scale the data set.`

I'll try to post a detailed respond soon. (I'm working on something else at the moment.)

`I'll try to post a detailed respond soon. (I'm working on something else at the moment.)`

Take your time, this takes a few years of back breaking work to bring to fruition, not a quick thingy

`Take your time, this takes a few years of back breaking work to bring to fruition, not a quick thingy`