Generating A LOT of images. 1M+
dyb last edited by dyb
I was asked to generate a lot of images.
Definitely more than 1M... I'd say around 33 554 432 specifically ^^'.
I'm running drawBot as a module that i set up in a virtualenv for this.
I'm pretty happy with the performance of my script. It generates around 100 images/second and if i calculate correctly it will finish in about 90 hours...
So i opened the activity monitor and noticed that Python is maxed out at 100% since it only uses one core (thread?). I'm not familiar with the topic... What would be the best way to utilise all CPU power? My CPU (i7-6820HQ) has 4 cores and each has 2 threads i guess? So that would be potentially 8x faster?
I can split the script in parts and run 4 scripts at once.
Can i use the same virtualenv Python installation in 4 terminal windows?
Or do i have to set up 4 virtualenvs?
Use another machine
I only have one.
I'm also concerned about the increasing number of memory used by Python. Right now it's at 100MB and it's increasing proportionally the more files i generate. 33M files would take around 30GB in this case.
What would be the best way to approach this?