multithreading - Ruby rake tasks thread optimization -
i have rake tasks need run every hour approximately, each 1 of them has contact web site, download file , import file db.
i wondering best approach in terms of efficiency , speed of execution. correct if think making thread each task save me time? doing should able minimize time spent waiting server response of slowest 1 among threads.
once downloaded files, thinking of using gem "parallel"
importing data db large csv , xml files.
please let me know should have suggestions.
when number of such files low, not care order of execution , can afford memory - simpliest solution run them in different processes cron (for example - gem 'whenever'
).
if there're more - use http gems parallel downloading - typhoeus
, curb
, em-http-request
etc
Comments
Post a Comment