Generating A LOT of images. 1M+



  • I was asked to generate a lot of images.
    Definitely more than 1M... I'd say around 33 554 432 specifically ^^'.

    I'm running drawBot as a module that i set up in a virtualenv for this.

    I'm pretty happy with the performance of my script. It generates around 100 images/second and if i calculate correctly it will finish in about 90 hours...

    So i opened the activity monitor and noticed that Python is maxed out at 100% since it only uses one core (thread?). I'm not familiar with the topic... What would be the best way to utilise all CPU power? My CPU (i7-6820HQ) has 4 cores and each has 2 threads i guess? So that would be potentially 8x faster?

    My ideas:

    Idea #1:
    I can split the script in parts and run 4 scripts at once.

    Concern:
    Can i use the same virtualenv Python installation in 4 terminal windows?
    Or do i have to set up 4 virtualenvs?

    Idea #2:
    Use another machine

    Concern:
    I only have one.

    I'm also concerned about the increasing number of memory used by Python. Right now it's at 100MB and it's increasing proportionally the more files i generate. 33M files would take around 30GB in this case.

    What would be the best way to approach this?


Log in to reply
 

Looks like your connection to DrawBot Forum was lost, please wait while we try to reconnect.