spiffy-websockets/spiffy.websockets.wiki

382 lines
18 KiB
Plaintext

[[tags: egg]]
== websockets
[[toc:]]
=== Description
{{websockets}} is a fast, lightweight websockets implementation. It supports version 13 of the websocket protocol and, currently, no extensions. It passes all of the [[http://autobahn.ws/testsuite/|Autobahn websockets compliance tests]].
All errors triggered by the library are of condition type {{websocket}} and all are continuable (not that they should be continued in all cases).
{{websockets}} also works with TLS. Just use it within a spiffy TLS request and connect to it with {{wss://}} instead of {{ws://}}.
=== Author
[[http://thintz.com|Thomas Hintz]] with contributions from Seth Alves.
Please send an email to t@thintz.com or chicken-users@nongnu.org with questions, bug reports, or feature requests.
=== Repository
The git repository for the websockets source code is hosted by bitbucket:
[[https://bitbucket.org/thomashintz/websockets|https://bitbucket.org/thomashintz/websockets]].
=== Requirements
The following eggs are required:
* [[/egg/spiffy|spiffy]]
* [[/egg/intarweb|intarweb]]
* [[/egg/uri-common|uri-common]]
* [[/egg/base64|base64]]
* [[/egg/simple-sha1|simple-sha1]]
* [[/egg/mailbox|mailbox]]
* [[/egg/comparse|comparse]]
=== Quick start example
Put these two files in the same folder.
'''index.html'''
<enscript highlight="html">
<html>
<body>
<script type="text/javascript">
var ws = new WebSocket("ws://localhost:8080/web-socket");
ws.onmessage = function(evt) {
alert(evt.data);
};
ws.onopen = function() {
ws.send('Hello!');
}
</script>
</body>
</html>
</enscript>
'''echo.scm'''
<enscript highlight="scheme">
(import chicken scheme)
(use spiffy websockets)
(handle-not-found
(lambda (path)
(when (string= path "/web-socket")
(with-websocket
(lambda ()
(send-message (string-append "you said: " (receive-message))))))))
(root-path ".")
(start-server port: 8080)
</enscript>
Then, in the same directory, run:
<enscript>
csi -s echo.scm
</enscript>
Navigate to http://localhost:8080 and you should see an alert saying "you said: Hello!".
The websocket specific code is really simple. In the above example it looks like this:
<enscript highlight="scheme">
(with-websocket
(lambda ()
(send-message (string-append "you said: " (receive-message)))))
</enscript>
You can put that in any request handler it it will work the same.
=== Parameters
===== {{current-websocket}}
<parameter>(current-websocket [websocket])</parameter>
{{(current-websocket)}} will be bound to a websocket object. Many procedures provide an optional argument for a websocket object and it is bound to {{current-websocket}} by default.
===== {{ping-interval}}
<parameter>(ping-interval [number])</parameter>
How often to ping the client, in seconds, in the background. If {{0}} then automatic pinging will be disabled. This defaults to 15 seconds.
If this is set to a value greater than 0 then a thread will run in the background sending ping messages to the client at the specified interval. This is used to keep connections open that will otherwise be closed. Often connections without a transmission will be killed after awhile. Receiving pongs in response to a ping will also reset the connection close timer.
pong responses to ping messages are not passed through to the user but they are used to update the timestamp that the connection timeout thread uses to decide if it should kill a connection.
===== {{close-timeout}}
<parameter>(close-timeout [number])</parameter>
How long to wait, in seconds, for the client to respond to a connection close request before timing out. If {{0}} then the connection timeout will be disabled. The default value is 5 seconds.
===== {{connection-timeout}}
<parameter>(connection-timeout [number])</parameter>
The length of time in seconds without a response (of any kind) from the client before the connection to the client will be cut off. If {{0}} then the connection will never be timed out by the websocket (something else can, of course, timeout causing the websocket connection to fail anyways). The default is 60 seconds.
===== {{accept-connection}}
<parameter>(accept-connection [procedure])</parameter>
A one-argument procedure taking as argument the URL path of the current page which tells
whether to accept the websocket connection. If {{#t}} accept, {{#f}} reject. '''IT IS HIGHLY RECOMMENDED''' that this parameter be set otherwise it opens up your application to potential security vulnerabilities. You will probably want to verify it is coming from a specific source. The default is to accept all connections.
===== {{access-denied}}
<parameter>(access-denied [procedure])</parameter>
A procedure to be called when the {{origin}} header value is rejected by the {{accept-connection}} procedure.
The default is:
<enscript highlight="scheme">
(lambda () (send-status 'forbidden "<h1>Access denied</h1>"))
</enscript>
===== {{drop-incoming-pings}}
<parameter>(drop-incoming-pings [boolean])</parameter>
Clients should usually not initiate pings so the default is to drop all incoming pings without responding with a pong. This defaults to {{#t}}. Set this to {{#f}} to respond to incoming pings with a pong.
===== {{propagate-common-errors}}
<parameter>(propagate-common-errors [boolean])</parameter>
A lot of errors that occur in the lifecycle of a websocket connection are more-or-less expected and it often is not interesting to receive and deal with these errors. The default is to correctly close the connection when an error occurs with the required opcode but the error is then not signaled to the user. The default is {{#f}}. Set to {{#t}} to receive all errors.
All websocket specific errors are covered by this. The only error you may receive if this is {{#f}} is of type {{websocket}} and {{unexpected-error}} if something goes terribly wrong with the implementation and usually means there is a bug in the implementation. This does not affect errors that are caused by user code, they will always be passed on but the websocket will be properly with an error code of 1011 (unexpected-error).
Note that this parameter is only relevant when using {{with-websocket}} or {{with-concurrent-websocket}}. If you don't use one of those then all errors will be propagated.
===== {{max-frame-size}}
<parameter>(max-frame-size [number])</parameter>
The maximum allowed frame payload size. The default is {{1MiB}}. If a frame exceeds this size then its payload will not be read and the connection will be dropped immediately. This signals a {{message-too-large}} error.
The maximum frame size supported by the current {{websockets}} implementation is {{1GiB}}.
===== {{max-message-size}}
<parameter>(max-message-size [number])</parameter>
The maximum allowed total message size. The default is {{1MiB}}. If a frame will cause the {{max-message-size}} to be exceeded then its payload is not read and the connection is dropped immediately. This signals a {{message-too-large}} error.
The maximum message size supported by the current {{websockets}} implementation is {{1GiB}}.
=== Interface
The {{websockets}} interface is not a "traditional" asynchronous API as is often seen with other websocket libraries. Instead a blocking and synchronous interface is provided as well as an asynchronous ''backed'' interface that looks and behaves like a blocking interface. See the function definitions below for further details.
The main difference between {{with-websocket}} and {{with-concurrent-websocket}} is that messages are guaranteed to be delivered in order with {{with-websocket}}. This also means that, when using {{with-websocket}} but not {{with-concurrent-websocket}}, no messages will be read into memory and processed unless a call is made to {{receive-message}}. This includes pong messages that may be sent in response to any pings sent in the background ping thread, if enabled. See the definition of {{ping-interval}} for more details on how background pings and pongs work. One implication of this is that unless you are waiting to receive a message (with {{receive-message}}) then pongs will not be received and the connection timeout timer will not be reset.
When receiving a {{connection-close}} message you should not send a response and you should exit the body of {{with-websocket}} or {{with-concurrent-websocket}}. At that point the connection will be closed properly. If you aren't expecting the connection to be closed you can signal an error and the websocket will be closed with an error code.
==== {{with-websocket}}
<procedure>(with-websocket [procedure])</procedure>
{{with-websocket}} handles the process of setting up and closing off a websocket connection. {{procedure}} is a thunk to be executed after the websocket has been successfully setup. When you use {{with-websocket}} you can be assured that the connection will always be closed correctly even if errors occur in your application code and for all protocol violations.
{{with-websocket}} sends and receives all messages in a blocking fashion. Only one message will be sent or received at a time unless more threads are introduced in {{procedure}}. Even then the processing will block sending and receiving whereas it will not with {{with-concurrent-websocket}}.
Unless you expect a high volume of messages or messages of very large size on one websocket connection then this is the recommended procedure to use. It is easier to program when you know messages will arrive in order and it is more lightweight.
==== {{with-concurrent-websocket}}
<procedure>(with-concurrent-websocket [procedure])</procedure>
This will, like {{with-websocket}}, handle setting up and closing a websocket connection. {{procedure}} is a thunk to be executed after the websocket has been successfully setup.
{{with-concurrent-websocket}} adds to {{with-websocket}} by running a thread in the background to read in messages as they arrive and spins off a new thread to process each message. Messages can arrive out-of-order, especially if there are a mix of very large and very small messages. {{with-concurrent-websocket}} can be very useful if very large messages are expected since they can take a long time to unmask and UTF8 validate.
Sending messages are still done in a blocking fashion but sending is relatively fast and will likely not be a problem. This could change in the future though so don't rely on it.
==== {{receive-message}}
<procedure>(receive-message #!optional (ws (current-websocket)))</procedure>
Read in a message from the client. Returns two values. The first is the message and the second is the message type, either {{'text}} or {{'binary}}. Regardless of the message type the message itself will always be a string. Takes an optional {{websocket}} object that is bound to {{(current-websocket)}} by default. It will always block until a message is received but if used within {{with-concurrent-websocket}} it will not block other messages from being received or processed and will return the first message that is finished being processed.
{{receive-message}} is "concurrent" aware so if it is used within {{with-websocket}} it will behave in a purely blocking fashion and when used within {{with-concurrent-websocket}} it will utilize the provided concurrency mechanism internally.
Note on performance: the websocket protocol requires all messages to be valid UTF8. The current validation process is extremely fast and generates very little garbage for UTF8 messages that only contain ASCII code points (chars less than 128), but is quite slow and uses a lot of memory when it contains higher plane code points. If you are processing a high volume of messages or exceptionally large messages it may be worthwhile to try to keep all or most messages ASCII only.
This method is thread safe.
A list of the possible message types (symbols):
* {{'text}}
* {{'binary}}
* {{'connection-close}}
If a message is of type {{binary}} then converting it to something possibly more "binary" like, such as a u8vector, could be done with the following. It will incur one copy of the data though.
<enscript highlight="scheme">
(blob->u8vector/shared (string->blob msg))
</enscript>
You could also use a string port to read the data in different fashions.
<enscript highlight="scheme">
(with-input-from-string msg
(lambda ()
(read-byte)
(read-u8vector))) ; etc
</enscript>
==== {{send-message}}
<procedure>(send-message data #!optional (message-type 'text) (ws (current-websocket)))</procedure>
Send a message to the client. {{data}} is a string or u8vector to be sent to the client. {{message-type}} is one of the types listed under the {{message-types}} section and defaults to {{'text}}. Usually this will be {{'text}} or {{'binary}} but any valid type is allowed. Note that if the {{data}} argument is a u8vector it must be copied before being sent due to some CHICKEN internal limitations, strings will not be. {{send-message}} also takes an optional {{websocket}} object that is bound to {{(current-websocket)}} by default.
This method is thread safe.
A list of the possible message types (symbols):
* {{'text}}
* {{'binary}}
* {{'ping}}
=== Exceptions
Possible exceptions and a brief description. See individual function definitions for details on when specific ones will be triggered.
<table>
<tr><th>type</th><th>description</th></tr>
<tr><td>websocket</td><td>A part of every websocket error as a composite condition. You can use this as a catchall for all websocket related errors.</td></tr>
<tr><td>protocol-error</td><td>Anytime something happens that violates the websocket protocol. The {{msg}} error property will contain details on the protocol broken.</td></tr>
<tr><td>invalid-optype</td><td>If an invalid message type (optype) is passed into any method that takes it as an argument.</td></tr>
<tr><td>reserved-bits-not-supported</td><td>A frame had a reserved bit set.</td></tr>
<tr><td>message-too-large</td><td>A message exceeds the set {{(max-frame-size)}} or {{(max-message-size)}} parameters.</td></tr>
<tr><td>unhandled-optype</td><td>The implementation received a frame with an opcode it could not handle. The {{optype}} property contains the unhandleable optype (as produced by {{(opcode->optype)}}).</td></tr>
<tr><td>invalid-data</td><td>Currently only signaled when a text payload contains invalid UTF8 codepoints. The {{msg}} property will hold a description of the "invalid data".</td></tr>
<tr><td>missing-header-upgrade</td><td>The request does not contain an "upgrade" parameter or it is not set to "websocket".</td></tr>
<tr><td>connection-timeout</td><td>The connection has timed out.</td></tr>
<tr><td></td><td></td></tr>
</table>
=== Examples
==== echo server for the Autobahn test suite
To run the tests make sure to have the test suites installed.
[[http://autobahn.ws/testsuite/|Autobahn test suite]]
'''echo-server.scm'''
<enscript highlight="scheme">
(import chicken scheme)
(use spiffy websockets)
(ping-interval 0)
(drop-incoming-pings #f)
(handle-not-found
(lambda (path)
(with-websocket
(lambda ()
(let loop ()
(receive (data type) (receive-message)
(unless (eq? type 'connection-close)
(send-message data type)
(loop))))))))
(start-server port: 8080)
</enscript>
'''ws-test.spec'''
<enscript>
{
"servers": [
{"agent": "AutobahnServer",
"url": "ws://localhost:8080/web-socket",
"options": {"version": 13}}
],
"cases": ["1.*", "2.*", "3.*", "4.*", "5.*"],
"exclude-cases": [],
"exclude-agent-cases": {}
}
</enscript>
Run and compile it:
'''terminal'''
<enscript>
csc -O3 echo-server.scm
./echo-server
</enscript>
'''terminal 2'''
<enscript>
wstest -m fuzzingclient -s ws-test.spec
</enscript>
=== Contributors
Thanks to Seth Alves for developing the initial version. Thanks also to Andy Bennet and Peter Bex for helping solve implementation problems and providing advice as well as all the others on #chicken for their contributions.
=== Versions
==== 0.1.6
Bug fix: flush the correct output port when sending a frame. (thanks to Alex Charlton)
==== 0.1.5
Bug fix: Re-exposing (current-websocket) (thanks to Alex Charlton)
==== 0.1.4
Numerous bug fixes including:
* preventing some race conditions.
* preventing some buffer overflows.
* flushing the output port after each frame is sent (thanks to Mario Goulart).
* correcting the upgrade header check (thanks to Andras Pahi).
==== 0.1.3
Remove reliance on C99 feature.
==== 0.1.2
Require spiffy 5.3.1 or newer.
==== 0.1.1
Fix bug where the connection was not being failed when a close reason contained an invalid UTF8 character.
==== 0.1.0
First release.
==== 0.0.1
Initial version.
=== License
Copyright (c) 2014, Thomas Hintz, Seth Alves
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the authors may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.