multithreading - Ruby rake tasks thread optimization -
i have rake tasks need run every hour approximately, each 1 of them has contact web site, download file , import file db.
i wondering best approach in terms of efficiency , speed of execution. correct if think making thread each task save me time? doing should able minimize time spent waiting server response of slowest 1 among threads.
once downloaded files, thinking of using gem "parallel" importing data db large csv , xml files.
please let me know should have suggestions.
when number of such files low, not care order of execution , can afford memory - simpliest solution run them in different processes cron (for example - gem 'whenever').
if there're more - use http gems parallel downloading - typhoeus, curb, em-http-request etc
Comments
Post a Comment