asynchronous - Parallel HTTP requests in ruby -

i have array of urls , wan't open each 1 , fetch specific tag.
want in parallel.

here pseudocode want do:

 urls = [...] tags = [] urls.each |url|   fetch_tag_asynchronously(url) |tag|     tags << tag   end end wait_for_all_requests_to_finish() 

if done in nice , safe way awesome.
use thread doesn't arrays thread safe in ruby.

you can achieve thread-safety using mutex:

require 'thread'  # mutex  urls = %w(   http://test1.example.org/   http://test2.example.org/   ... )  threads = [] tags = [] tags_mutex = mutex.new  urls.each |url|   threads << thread.new(url, tags) |url, tags|     tag = fetch_tag(url)     tags_mutex.synchronize { tags << tag }   end end  threads.each(&:join) 

it counter-productive use new thread every url, limiting number of threads might more performant:

thread_count = 8  # tweak number maximum performance.  tags = [] mutex = mutex.new  thread_count.times.map {   thread.new(urls, tags) |urls, tags|     while url = mutex.synchronize { urls.pop }       tag = fetch_tag(url)       mutex.synchronize { tags << tag }     end   end }.each(&:join)