In previous post (here) I described some approach how I overcame some limitation of Matplotlib when plotting large data sets. I also mentioned that using both cores of processor could further speed up calculation. Now we’ll see some short example how to use multicore and threading. According the python’s documentation, multiprocessing APIs was designed to mimic those of threading. In the next we will see simple examples and some comparison.

Continue reading →

Python and Matplotlib is very good combination for processing data and their presentation. But this time, trying to do some “ray tracing”, I encountered some problem to plot large, or rather huge amounts of data. Task was to plot several thousands to millions of lines in a plot with opacity close to 0.005 and see how they blend together.

Comparing Linux computer with 756 MB ram and Mac computer with 2 GB ram it was obvious that the more ram the better, but even that amount was not enough. I tried several backends (agg, pdf, ps, cairo, macosx) and file formats (pdf, eps, png, svg) without any improvement. It still would take several hours to finish the run. In the case of svg file format it even took time to render the image as it is XML based format. For example Inkscape or Gimp could generate the output image in about 2~4 minutes. But there was The Idea.

Continue reading →