I've listed some math tools and libraries before. A lot of math tools have their own DSL (domain-specific language) for quick & easy access to their various math functions with some primitive control flows. DSL is better in the sense that it's relatively convenient and it gets hooked more tightly to the underlying base (i.e. shorter code, more inner-involving). A general programming language has its own strength: it's powerful, elegant, much more mature and more easy to access the 'outer' world. DSL for the math tools (like the 3 big Ms) sometimes looks kinda awkward from a 'programming' point of view. Besides, people tend to like doing work in their own favorite languages.
Python is rather popular nowadays. It's been used in various fields -- sys admin, net utility, web, prototyping, unit testing, etc. -- and of course, in mathematics, science, and engineering. One of the well-known tool chains is: scipy + numpy + matplotlib + ipython aka the PyLab. scipy/numpy is tremendously good for numeric computation and matplotlib can generate high-quality 2D plots -- in your favorite language.
Another tool chain i'd like to share is for symbolic manipulation: gmpy + mpmath + sympy. gmpy provides the python binding for gmp, mpmath provides some high-level arbitrary-precision computing functions while sympy is really good at symbolic processing with great 2D/3D plotting support and a convenient interactive 'isympy'. Your distro may have these pkgs but please check if they are outdated. I recommend you use the latest version built from their own repos. (sympy has included mpmath, btw). Newer versions are way faster -- Try calculating 1M digits of Pi.
And here's a big all-in-one math tool called sage, which is using python as the primary programming language and can be regarded as a free alternative to the 3 big Ms.
Also, two tools not specifically for math: psyco and parallel python (via bones). psyco is kind of a python JIT. If you do a lot of cpu-bound instructions in python, it is your lovely friend. pp is for parallel computing (multi-core or even network cluster). It can split computation at the function level.
Finally, for speed-lovers, there's a language called wirbel. It has very similar syntax to python (almost equal). The key difference is that it can compile src to native bin, thus it's very fast. But limited: not all python code can be easily changed and compiled by it of course, at least not now. Still an interesting alternative anyway. You might want to have a look.