Options

Scala

Created stub for Scala. Scala seems to be much better suited for numerical algorithms than Java or languages from the .NET framework (although F# would be a candidate, too, that I don't know much about).

In addition, there is already a project underway that is about the implementation of MATLAB like objects for numerical linear algebra called Scalala. There does not seem to be a similar project in other modern languages. Maybe I need to revise my first impression that Scala is a bit immature compared to Java, but I'll need to take a closer look at both the language and its evolution from the current release 2.8 to the next release.

The implementation of the FFT in Scala should be much more fun than in Java, anyway.

Comments

  • 1.

    People trying to integrate programming with mathematics find Scala is ideal. See for example the scalaz library [look at the Type Classes section].

    Comment Source:People trying to integrate programming with mathematics find Scala is ideal. See for example [the scalaz library](http://code.google.com/p/scalaz/) [look at the Type Classes section].
  • 2.
    edited February 2011

    Thanks! I already asked the community via a broadcast to one of the google groups, but wasn't pointed to scalaz.

    Hey, do these people implement things from category theory like monads and functors ?!

    Comment Source:Thanks! I already asked the community via a broadcast to one of the google groups, but wasn't pointed to scalaz. Hey, do these people implement things from category theory like monads and functors ?!
  • 3.

    Monads are the basis of the do statement in Haskell and the for statement in Scala. [Dom Verity said in a recent talk in Sydney that programming monads correspond to "monads with strength" in Category theory]. Scalaz implements a lot of Haskell stuff and reportedly improves on some of it. However Scala is not a lazy language by default, so it doesn't need monads to interface to the real world as Haskell does.

    It's a fun area, but possibly not relevant in itself. However it does illustrate the strength of the language. The scalaz people pushed Scala beyond anything imagined by the language designers and showed the tremendous power and flexibility of that design.

    Comment Source:Monads are the basis of the *do* statement in Haskell and the *for* statement in Scala. [Dom Verity said in a recent talk in Sydney that programming monads correspond to "monads with strength" in Category theory]. Scalaz implements a lot of Haskell stuff and reportedly improves on some of it. However Scala is not a lazy language by default, so it doesn't need monads to interface to the real world as Haskell does. It's a fun area, but possibly not relevant in itself. However it does illustrate the strength of the language. The scalaz people pushed Scala beyond anything imagined by the language designers and showed the tremendous power and flexibility of that design.
  • 4.

    There is nothing stranger than learning category theory as a mathematician and then hearing computer scientists say what monads are for. It sounds sort of like "an abelian group is a data type with floating-points as a subclass, which permits one to undo the ADD operation", except 10 times more complicated.

    Luckily, since my student Mike Stay works at Google, he has explained this stuff to me.

    Comment Source:There is nothing stranger than learning category theory as a mathematician and then hearing computer scientists say what monads are for. It sounds sort of like "an abelian group is a data type with floating-points as a subclass, which permits one to undo the ADD operation", except 10 times more complicated. Luckily, since my student Mike Stay works at Google, he has explained this stuff to me.
  • 5.
    edited March 2011

    Hi Tim -- I would definitely count Scala 2.8 as a mature language, indeed I use it in my day job (at Credit Suisse in New York).

    Recently we even started using Scalala.. our experience is that it is quite difficult to learn: the documentation has not caught up with the code. And if you do decide to "use the source, Luke" you will quickly find that the higher language features mentioned above can make it more difficult to follow the (very general) implementation of some methods. The src/test/scala folder in github is probably a good place to start, as well as this thread on the "reboot" branch of the code

    Comment Source:Hi Tim -- I would definitely count Scala 2.8 as a mature language, indeed I use it in my day job (at Credit Suisse in New York). Recently we even started using Scalala.. our experience is that it is quite difficult to learn: the documentation has not caught up with the code. And if you do decide to "use the source, Luke" you will quickly find that the higher language features mentioned above can make it more difficult to follow the (very general) implementation of some methods. The src/test/scala folder in github is probably a good place to start, as well as [this thread on the "reboot" branch of the code](http://groups.google.com/group/scalala/browse_thread/thread/2414774016aea6b5)
  • 6.

    If you are considering Scala for any of the Azimuth Code Project, let me know! I can definitely help get you up and running. I'd also like to recommend the 'sbt' build tool.

    It is very easy to work with, has clear documentation, plugins to get you up and running with either Netbeans or IntelliJ IDEA (another mature IDE for Scala), and plays nicely with Maven/Ivy.

    Comment Source:If you are considering Scala for any of the Azimuth Code Project, let me know! I can definitely help get you up and running. I'd also like to recommend the ['sbt' build tool](http://code.google.com/p/simple-build-tool/). It is very easy to work with, has clear documentation, plugins to get you up and running with either Netbeans or IntelliJ IDEA (another mature IDE for Scala), and plays nicely with Maven/Ivy.
  • 7.

    rks: Haskell Monads - was an after the fact construction by Simon Peyton Jones and Philip Wadler - to make input/output be more functional. But I think the language was better when they just used lazy lists (like Oz and Alice today). Also the Actor concept was originally from MIT (Carl Hewitt) but came to fame when some colleagues introduced it into Erlang as processes with messages

    all: Has anyone used or have any opinion on having XML as built into the language?

    Tim: As I said in the other thread I think this is a good decision. Have a look at akka as well , so you get a feeling for the kind of scalability it can enable.

    John B: :-) so o you think you can find a commutative group of order 4 in Scala ?

    Comment Source:rks: Haskell Monads - was an after the fact construction by Simon Peyton Jones and Philip Wadler - to make input/output be more functional. But I think the language was better when they just used lazy lists (like Oz and Alice today). Also the Actor concept was originally from MIT (Carl Hewitt) but came to fame when some colleagues introduced it into Erlang as processes with messages all: Has anyone used or have any opinion on having XML as built into the language? Tim: As I said in the other thread I think this is a good decision. Have a look at [akka as well](http://akka.io) , so you get a feeling for the kind of scalability it can enable. John B: :-) so o you think you can find a commutative group of order 4 in Scala ?
  • 8.
    edited March 2011

    Unless I'm very behind on changes to Haskell, you can still use lazy-list IO in Haskell today (providing you hook them up to the runtime via monads), it's just that the preferred style is monadic (particularly since lazy lists are problematic if you need to deal with a stateful external, eg, writing out a temporary file before launching an external command on it). I'm unsure of my attitude to more other monads used in FP -- eg things like limited computational step monads -- since they're doing information hiding which is both good and bad. (Despite the fact that I talk a lot about low-level computing issues, I'm actually a great fan of using languages like Haskell for tasks where the problem is "program complexity" rather than program performance.)

    Regarding XML, I think that it's as good as anything as a standardised low-volume data representation. (Please don't do things like encode images or dense (hyper-)volumentric data in XML.) The one thing that I absolutely rant about is that XML should not ever, ever, ever be used for high-frequency data transfer within a "concurrent system" (eg, something where processes or threads have been split for modularity reasons like separate development, robustness, etc). This is because, particularly with bad experience on one project, I've never found the XML-isation code to not require ridiculous numbers of heap allocations for nodes and strings, followed by a slow parsing step on the other side. This is fine occasionally, but doing it at even 25 times a second I frequently found that almost all of the CPU time budget was spent doing XML creation and parsing rather than useful work. I haven't looked at how Scala uses XML, but providing it's not used for high-frequency "data transfer" internally I don't care either way.

    Comment Source:Unless I'm very behind on changes to Haskell, you can still use lazy-list IO in Haskell today (providing you hook them up to the runtime via monads), it's just that the preferred style is monadic (particularly since lazy lists are problematic if you need to deal with a stateful external, eg, writing out a temporary file before launching an external command on it). I'm unsure of my attitude to more other monads used in FP -- eg things like limited computational step monads -- since they're doing information hiding which is both good and bad. (Despite the fact that I talk a lot about low-level computing issues, I'm actually a great fan of using languages like Haskell for tasks where the problem is "program complexity" rather than program performance.) Regarding XML, I think that it's as good as anything as a standardised low-volume data representation. (Please don't do things like encode images or dense (hyper-)volumentric data in XML.) The one thing that I absolutely rant about is that XML should **not ever, ever, ever** be used for high-frequency data transfer within a "concurrent system" (eg, something where processes or threads have been split for modularity reasons like separate development, robustness, etc). This is because, particularly with bad experience on one project, I've never found the XML-isation code to not require ridiculous numbers of heap allocations for nodes and strings, followed by a slow parsing step on the other side. This is fine occasionally, but doing it at even 25 times a second I frequently found that almost all of the CPU time budget was spent doing XML creation and parsing rather than useful work. I haven't looked at how Scala uses XML, but providing it's not used for high-frequency "data transfer" internally I don't care either way.
  • 9.

    Allen said:

    If you are considering Scala for any of the Azimuth Code Project, let me know! I can definitely help get you up and running. I'd also like to recommend the 'sbt' build tool.

    Thanks, I'll have a look. I kind of hoped that Maven could be used with Scala, because I very much like how Maven handles dependencies, and I thought that beyond that, I wouldn't need any other tools, but we'll see.

    If you are considering Scala for any of the Azimuth Code Project, let me know! I can definitely help get you up and running.

    I'm considering Scala as the primary language for a GCM, that would need numerical solvers for incompressible and compressible fluids - which is in itself a rather big topic. Right now I'm concurrently learning the language itself and looking around for explanations of the necessary mathematics and physics and libraries, I'm not into programming anything yet.

    I think the first step should be a simulation of an incompressible flow in a 3D cube with periodic boundary conditions. It is easy enough for a first step (ignoring boundary wall problems etc) but still complicated enough to learn something.

    If you'd like to help, you could try to implement some FFT algorithms. These will be needed for spectral methods.

    Comment Source:Allen said: <blockquote> <p> If you are considering Scala for any of the Azimuth Code Project, let me know! I can definitely help get you up and running. I'd also like to recommend the 'sbt' build tool. </p> </blockquote> Thanks, I'll have a look. I kind of hoped that Maven could be used with Scala, because I very much like how Maven handles dependencies, and I thought that beyond that, I wouldn't need any other tools, but we'll see. <blockquote> <p> If you are considering Scala for any of the Azimuth Code Project, let me know! I can definitely help get you up and running. </p> </blockquote> I'm considering Scala as the primary language for a [[GCM]], that would need numerical solvers for incompressible and compressible fluids - which is in itself a rather big topic. Right now I'm concurrently learning the language itself and looking around for explanations of the necessary mathematics and physics and libraries, I'm not into programming anything yet. I think the first step should be a simulation of an incompressible flow in a 3D cube with periodic boundary conditions. It is easy enough for a first step (ignoring boundary wall problems etc) but still complicated enough to learn something. If you'd like to help, you could try to implement some [[FFT]] algorithms. These will be needed for [[spectral methods]].
  • 10.

    Staffan said:

    Have a look at akka as well , so you get a feeling for the kind of scalability it can enable.

    I've heard of this project, but I'm still into learning the basics of the language and barely able to parse the syntax. But I have some experience with multithreaded systems and can only agree: It is too hard, even with the support introduced with Java 5.

    Has anyone used or have any opinion on having XML as built into the language?

    Handling XML cannot be much easier than it is with Java's JAXB, the problem is, as David said, that it eats memory and performance. The best use is with initialization and configuration data that is read at startup. As you probably know, that's where it is used all over the Java landscape.

    I think it should be used for topographical data for GCMs, for example. I'm dreaming of loading topographical data (coast lines, mountain heights, albedo) from a webservice and initializing a GCM with it.

    Comment Source:Staffan said: <blockquote> <p> Have a look at akka as well , so you get a feeling for the kind of scalability it can enable. </p> </blockquote> I've heard of this project, but I'm still into learning the basics of the language and barely able to parse the syntax. But I have some experience with multithreaded systems and can only agree: It is too hard, even with the support introduced with Java 5. <blockquote> <p> Has anyone used or have any opinion on having XML as built into the language? </p> </blockquote> Handling XML cannot be much easier than it is with Java's JAXB, the problem is, as David said, that it eats memory and performance. The best use is with initialization and configuration data that is read at startup. As you probably know, that's where it is used all over the Java landscape. I think it should be used for topographical data for [[GCM]]s, for example. I'm dreaming of loading topographical data (coast lines, mountain heights, albedo) from a webservice and initializing a [[GCM]] with it.
  • 11.

    Sorry for the confusion on XML I was asking about Scalas builtin type XML, nothing else. But I agree that XML in general is a capacity hog.

    Tim: Lift uses Maven to facilitate setup so the answer is yes

    David: You are probably right about Haskell, since its been some years since I looked at Haskell. There is a very active Haskell research community in Gothenburg. And some commercial spin offs using Haskell like quvik.com

    Comment Source:Sorry for the confusion on XML I was asking about Scalas builtin type XML, nothing else. But I agree that XML in general is a capacity hog. Tim: Lift uses Maven to facilitate setup so the answer is yes David: You are probably right about Haskell, since its been some years since I looked at Haskell. There is a very active Haskell research community in Gothenburg. And some commercial spin offs using Haskell like quvik.com
  • 12.

    Tim/Staffan: although you can use Maven directly with Scala (as in Lift), sbt wraps both Maven and Ivy functionality, so you can manage dependencies in exactly the same way, but with concise Scala build files rather than XML project descriptions. Akka is also a very cool project -- we are using this at work too! Akka uses 'sbt' to build btw -- it is definitely one of the "reference projects" I turn to for Scala coding style, best practices, testing techniques etc

    Tim: let me get back to you soon re FFT in Scala :-)

    Comment Source:Tim/Staffan: although you can use Maven directly with Scala (as in Lift), sbt wraps both Maven and Ivy functionality, so you can manage dependencies in exactly the same way, but with concise Scala build files rather than XML project descriptions. Akka is also a very cool project -- we are using this at work too! Akka uses 'sbt' to build btw -- it is definitely one of the &quot;reference projects&quot; I turn to for Scala coding style, best practices, testing techniques etc Tim: let me get back to you soon re FFT in Scala :-)
  • 13.

    Allen wrote:

    ...although you can use Maven directly with Scala (as in Lift), sbt wraps both Maven and Ivy functionality, so you can manage dependencies in exactly the same way, but with concise Scala build files rather than XML project descriptions.

    Hm, I never had any problems with maven's POM files...Is there a plugin for NetBeans?

    Comment Source:Allen wrote: <blockquote> <p> ...although you can use Maven directly with Scala (as in Lift), sbt wraps both Maven and Ivy functionality, so you can manage dependencies in exactly the same way, but with concise Scala build files rather than XML project descriptions. </p> </blockquote> Hm, I never had any problems with maven's POM files...Is there a plugin for NetBeans?
  • 14.

    Scala 2.9 has been released, so I'll take another look if the IDE and other tools are a little bit more stable now, as promised by the developer teams.

    After playing around with Python I think that there are several features that make me prefer Scala (or Java or .NET):

    • large systems need a strongly typed language,

    • performant systems need a hot spot compiling mechanism, a language that can only run as a dynamically interpreted system as Python cannot be efficient.

    Comment Source:Scala 2.9 has been released, so I'll take another look if the IDE and other tools are a little bit more stable now, as promised by the developer teams. After playing around with Python I think that there are several features that make me prefer Scala (or Java or .NET): - large systems need a strongly typed language, - performant systems need a hot spot compiling mechanism, a language that can only run as a dynamically interpreted system as Python cannot be efficient.
  • 15.
    edited May 2011

    Regarding your second point, I think there are two separate issues:

    1. Python is developing a tracing JIT system (called PyPy). They're trying to do lots of things automatically that are normally put into the code generator specially by hand, so it's not that far along.

    2. However, a JIT is only really useful for resolving abstractions that weren't removed at compile time, which can be effective for things like web-server apps or GUI code. It doesn't, as far as studies I've seen, do the kind of analysis of large sections of code to fully optimise them. That tends to require a language where the semantics have been formulated to make certain classes of important optimisations apparent to the compiler.

    Comment Source:Regarding your second point, I think there are two separate issues: 1. Python is developing a tracing JIT system (called [PyPy](http://morepypy.blogspot.com/)). They're trying to do lots of things automatically that are normally put into the code generator specially by hand, so it's not that far along. 2. However, a JIT is only really useful for resolving abstractions that weren't removed at compile time, which can be effective for things like web-server apps or GUI code. It doesn't, as far as studies I've seen, do the kind of analysis of large sections of code to fully optimise them. That tends to require a language where the semantics have been formulated to make certain classes of important optimisations apparent to the compiler.
  • 16.

    I'd really like to play around with numerics code, applications like web stores usually have a couple of performance bottlenecks that eat up 99% of the time like network latency or bad SQL. The saying "all performance optimizing is evil" is even more true in this context than it was when it was first formulated (I forgot who said that first. Donald Knuth?).

    For a three-tier web application you cannot say anything about performance bottlenecks without a profiler, and with a profiler you'll usually see two or three hot spots. Which means that one cannot extrapolate any experience to the performance of say, standalone climate models. And as far as I know no one has written a large number crunching app in Java or Scala. So, well, I'd say the result is very much open :-)

    I wonder if the "performance optimizations" that the Scala team did for release 2.9 refer to the runtime performance or to the tools, like the compiler. The 2.8 compiler was really slow, I think because it has to do a lot on top of the Java compiler.

    BTW: Even if an implementation in Java or Scala is 10 times slower than an equivalent implementation in another language, I think it would still be worthwhile if it is considerably shorter, easier to understand and to test, and runs everywhere. For a PhD candidate it is of course very relevant if he gets his result every other hour, or instead every other week, but it does not matter to me :-)

    Comment Source:I'd really like to play around with numerics code, applications like web stores usually have a couple of performance bottlenecks that eat up 99% of the time like network latency or bad SQL. The saying "all performance optimizing is evil" is even more true in this context than it was when it was first formulated (I forgot who said that first. Donald Knuth?). For a three-tier web application you cannot say anything about performance bottlenecks without a profiler, and with a profiler you'll usually see two or three hot spots. Which means that one cannot extrapolate any experience to the performance of say, standalone climate models. And as far as I know no one has written a large number crunching app in Java or Scala. So, well, I'd say the result is very much open :-) I wonder if the "performance optimizations" that the Scala team did for release 2.9 refer to the runtime performance or to the tools, like the compiler. The 2.8 compiler was really slow, I think because it has to do a lot on top of the Java compiler. BTW: Even if an implementation in Java or Scala is 10 times slower than an equivalent implementation in another language, I think it would still be worthwhile if it is considerably shorter, easier to understand and to test, and runs everywhere. For a PhD candidate it is of course very relevant if he gets his result every other hour, or instead every other week, but it does not matter to me :-)
  • 17.

    Bruce Eckel has some prominence in the Java world as an author, recently, he has written an interesting blog post, comparing Scala and Python. Well, interesting for programmers, at least.

    Scala: The Static Language that Feels Dynamic

    Comment Source:Bruce Eckel has some prominence in the Java world as an author, recently, he has written an interesting blog post, comparing Scala and Python. Well, interesting for programmers, at least. <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=328540">Scala: The Static Language that Feels Dynamic</a>
  • 18.

    that is exactlu what i liked when I tried it, and read some books, the static is there but its almost transparent.Martin Ordersky will do a keynote speech at OSCON btw, see here

    Comment Source:that is exactlu what i liked when I tried it, and read some books, the static is there but its almost transparent.Martin Ordersky will do a keynote speech at OSCON btw, [see here](http://www.oscon.com/oscon2011/public/schedule/detail/21055)
Sign In or Register to comment.