An asynchronous client and server implementation of HTTP/1.0, HTTP/1.1 and HTTP/2 including TLS. Support for streaming requests and responses. Built on top of async and async-io. falcon provides a rack-compatible server.
Add this line to your application's Gemfile:
gem'async-http'And then execute:
$ bundle Or install it yourself as:
$ gem install async-http Please see the project documentation or serve it locally using bake utopia:project:serve.
Here is an example showing how to post a data structure as JSON to a remote resource:
#!/usr/bin/env rubyrequire'json'require'async'require'async/http/internet'data={'life'=>42}Asyncdo# Make a new internet:internet=Async::HTTP::Internet.new# Prepare the request:headers=[['accept','application/json']]body=[JSON.dump(data)]# Issues a POST request:response=internet.post("https://httpbin.org/anything",headers,body)# Save the response body to a local file:ppJSON.parse(response.read)ensure# The internet is closed for business:internet.closeendConsider using async-rest instead.
To issue multiple requests concurrently, you should use a barrier, e.g.
#!/usr/bin/env rubyrequire'async'require'async/barrier'require'async/http/internet'TOPICS=["ruby","python","rust"]Asyncdointernet=Async::HTTP::Internet.newbarrier=Async::Barrier.new# Spawn an asynchronous task for each topic:TOPICS.eachdo |topic| barrier.asyncdoresponse=internet.get"https://www.google.com/search?q=#{topic}"puts"Found #{topic}: #{response.read.scan(topic).size} times."endend# Ensure we wait for all requests to complete before continuing:barrier.waitensureinternet&.closeendIf you need to limit the number of simultaneous requests, use a semaphore.
#!/usr/bin/env rubyrequire'async'require'async/barrier'require'async/semaphore'require'async/http/internet'TOPICS=["ruby","python","rust"]Asyncdointernet=Async::HTTP::Internet.newbarrier=Async::Barrier.newsemaphore=Async::Semaphore.new(2,parent: barrier)# Spawn an asynchronous task for each topic:TOPICS.eachdo |topic| semaphore.asyncdoresponse=internet.get"https://www.google.com/search?q=#{topic}"puts"Found #{topic}: #{response.read.scan(topic).size} times."endend# Ensure we wait for all requests to complete before continuing:barrier.waitensureinternet&.closeendTo keep connections alive, install the thread-local gem, require async/http/internet/instance, and use the instance, e.g.
#!/usr/bin/env rubyrequire'async'require'async/http/internet/instance'Asyncdointernet=Async::HTTP::Internet.instanceresponse=internet.get"https://www.google.com/search?q=test"puts"Found #{response.read.size} results."endHere is an example showing how to download a file and save it to a local path:
#!/usr/bin/env rubyrequire'async'require'async/http/internet'Asyncdo# Make a new internet:internet=Async::HTTP::Internet.new# Issues a GET request to Google:response=internet.get("https://www.google.com/search?q=kittens")# Save the response body to a local file:response.save("/tmp/search.html")ensure# The internet is closed for business:internet.closeendHere is a basic example of a client/server running in the same reactor:
#!/usr/bin/env rubyrequire'async'require'async/http/server'require'async/http/client'require'async/http/endpoint'require'async/http/protocol/response'endpoint=Async::HTTP::Endpoint.parse('http://127.0.0.1:9294')app=lambdado |request| Protocol::HTTP::Response[200,{},["Hello World"]]endserver=Async::HTTP::Server.new(app,endpoint)client=Async::HTTP::Client.new(endpoint)Asyncdo |task| server_task=task.asyncdoserver.runendresponse=client.get("/")putsresponse.statusputsresponse.readserver_task.stopendYou can hook into SSL certificate verification to improve server verification.
require'async'require'async/http'# These are generated from the certificate chain that the server presented.trusted_fingerprints={"dac9024f54d8f6df94935fb1732638ca6ad77c13"=>true,"e6a3b45b062d509b3382282d196efe97d5956ccb"=>true,"07d63f4c05a03f1c306f9941b8ebf57598719ea2"=>true,"e8d994f44ff20dc78dbff4e59d7da93900572bbf"=>true,}Asyncdoendpoint=Async::HTTP::Endpoint.parse("https://www.codeotaku.com/index")# This is a quick hack/POC:ssl_context=endpoint.ssl_contextssl_context.verify_callback=procdo |verified,store_context| certificate=store_context.current_certfingerprint=OpenSSL::Digest::SHA1.new(certificate.to_der).to_siftrusted_fingerprints.include?fingerprinttrueelseConsole.logger.warn("Untrusted Certificate Fingerprint"){fingerprint}falseendendendpoint=endpoint.with(ssl_context: ssl_context)client=Async::HTTP::Client.new(endpoint)response=client.get(endpoint.path)ppresponse.status,response.headers.fields,response.readendHere's a basic example with a timeout:
#!/usr/bin/env rubyrequire'async/http/internet'Asyncdo |task| internet=Async::HTTP::Internet.new# Request will timeout after 2 secondstask.with_timeout(2)doresponse=internet.get"https://httpbin.org/delay/10"endrescueAsync::TimeoutErrorputs"The request timed out"ensureinternet&.closeendOn a 4-core 8-thread i7, running ab which uses discrete (non-keep-alive) connections:
$ ab -c 8 -t 10 http://127.0.0.1:9294/ This is ApacheBench, Version 2.3 <$Revision: 1757674 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 127.0.0.1 (be patient) Completed 5000 requests Completed 10000 requests Completed 15000 requests Completed 20000 requests Completed 25000 requests Completed 30000 requests Completed 35000 requests Completed 40000 requests Completed 45000 requests Completed 50000 requests Finished 50000 requests Server Software: Server Hostname: 127.0.0.1 Server Port: 9294 Document Path: / Document Length: 13 bytes Concurrency Level: 8 Time taken for tests: 1.869 seconds Complete requests: 50000 Failed requests: 0 Total transferred: 2450000 bytes HTML transferred: 650000 bytes Requests per second: 26755.55 [#/sec] (mean) Time per request: 0.299 [ms] (mean) Time per request: 0.037 [ms] (mean, across all concurrent requests) Transfer rate: 1280.29 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 0 0 0.2 0 6 Waiting: 0 0 0.2 0 6 Total: 0 0 0.2 0 6 Percentage of the requests served within a certain time (ms) 50% 0 66% 0 75% 0 80% 0 90% 0 95% 1 98% 1 99% 1 100% 6 (longest request) On a 4-core 8-thread i7, running wrk, which uses 8 keep-alive connections:
$ wrk -c 8 -d 10 -t 8 http://127.0.0.1:9294/ Running 10s test @ http://127.0.0.1:9294/ 8 threads and 8 connections Thread Stats Avg Stdev Max +/- Stdev Latency 217.69us 0.99ms 23.21ms 97.39% Req/Sec 12.18k 1.58k 17.67k 83.21% 974480 requests in 10.10s, 60.41MB read Requests/sec: 96485.00 Transfer/sec: 5.98MB According to these results, the cost of handling connections is quite high, while general throughput seems pretty decent.
HTTP/1 has an implicit scheme determined by the kind of connection made to the server (either http or https), while HTTP/2 models this explicitly and the client indicates this in the request using the :scheme pseudo-header (typically https). To normalize this, Async::HTTP::Client and Async::HTTP::Server have a default scheme which is used if none is supplied.
HTTP/1 has an explicit version while HTTP/2 does not expose the version in any way.
HTTP/1 responses contain a reason field which is largely irrelevant. HTTP/2 does not support this field.
We welcome contributions to this project.
- Fork it.
- Create your feature branch (
git checkout -b my-new-feature). - Commit your changes (
git commit -am 'Add some feature'). - Push to the branch (
git push origin my-new-feature). - Create new Pull Request.
This project uses the Developer Certificate of Origin. All contributors to this project must agree to this document to have their contributions accepted.
This project is governed by Contributor Covenant. All contributors and participants agree to abide by its terms.
- benchmark-http — A benchmarking tool to report on web server concurrency.
- falcon — A rack compatible server built on top of
async-http. - async-websocket — Asynchronous client and server websockets.
- async-rest — A RESTful resource layer built on top of
async-http. - async-http-faraday — A faraday adapter to use
async-http.