Emacs Web Server User Manual
****************************

This file documents the Emacs Web Server (web-server)

Copyright (C) 2013 Eric Schulte <schulte.eric@gmail.com>

     Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with the Invariant Section being ``GNU GENERAL PUBLIC LICENSE,'' A copy of the license is included in the section entitled ``GNU Free Documentation License.''

Introduction
************

The Emacs Web Server is a Web server implemented entirely in Emacs Lisp.  HTTP requests are matched to handlers (see Handlers) which are Emacs Lisp functions.  Handlers receive as their only argument a request object (see Requests) which holds information about the request and a process holding the HTTP network connection.  Handlers write their responses directly to the network process.

A number of examples (see Usage Examples) demonstrate usage of the Emacs Web Server.  All public functions of the Emacs Web Server are listed (see Function Index).

Handlers
********

The function ws-start takes takes two arguments handlers and port.  It starts a server listening on port responding to requests with handlers.  Handlers may be either a single function or an association list composed of pairs of matchers and handler functions.  When handlers is a single function the given function is used to serve every request, when it is an association list, the function of the first matcher to match each request handles that request.

Matchers
========

Matchers may be a regular expression or a function.  Regular expression matchers consists of an HTTP header and a regular expression.  When the regular expression matches the content of the given header the matcher succeeds and the associated handler is called.  For example the following matches any GET request whose path starts with the substring ``foo''.

     (:GET . "^foo")

A function matcher is a function which takes the request object (see Requests) and succeeds when the function returns a non-nil value.  For example the following matcher matches every request,

     (lambda (_) t)

and the following matches only requests in which the supplied ``number'' parameter is odd.

     (lambda (request)
       (oddp (string-to-number (cdr (assoc "number" request)))))

Handler Function
================

Each handler is a function which takes a request object (see Requests) as its only argument.  The function may respond to the request by writing to the network process held in the process field of the request object.  For example, the process-send-string function may be used to write string data to a request as in the following.

       (process-send-string (process request) "hello world")

When the handler function exits the connection is terminated unless the handler function returns the keyword :keep-alive.

Requests
********

Each HTTP requests is represented using a ws-request object (see ws-request).  The request object serves two purposes, one internal and one external.  Internally, request objects are used to hold state while HTTP headers are parsed incrementally as the HTTP request text is received from the network.  Externally, request objects are used to decide which handler to call, and are then passed as the only argument to the called handler.

In addition to fields used internally, each ws-request object holds the network process in the process and holds all HTTP headers and request GET or POST parameters in the headers alist.  HTML Headers are keyed using uppercase keywords (e.g., :GET), and user supplied parameters are keyed using the string name of the parameter.

The process field may be used by handlers to send data to a client as in the following example.

     (process-send-string (process request) "hello world")

The headers field may be used to access request information such as the requested path,

     (cdr (assoc :GET (headers request)))

or named parameters as from a web form.

     (cdr (assoc "message" (headers request)))

Usage Examples
**************

These examples demonstrate usage.

Hello World
===========

The simplest possible ``hello world'' example.  The handler consists of a single (matcher . handler) pair.  The function matcher matches every incoming HTTP request.  The handler responds by setting the content type to text/plain, and then sending the string ``hello world''.  When the handler exits the network connection of the request is closed.

     [file 000-hello-world.el not included]

Hello World UTF8
================

This example only differs from the previous in that the ``Content-type'' indicates UTF8 encoded data, and the hello world sent is selected at random from a list of different languages.

     [file 001-hello-world-utf8.el not included]

Hello World HTML
================

     [file 002-hello-world-html.el not included]

This variation of the ``hello world'' example sends a text/html response instead of a simple text/plain response.

File Server
===========

The following example implements a file server which will serve files from the docroot document root set to the current working directory in this example.  Four helper functions are used; ws-in-directory-p is used to check if the requested path is within the document root.  If not then ws-send-404 is used to send a default ``File Not Found''.  If so then the file is served with ws-send-file (which appropriately sets the mime-type of the response based on the extension of the file) if it is a file or is served with ws-send-directory-list if it is a directory.

     [file 003-file-server.el not included]

URL Parameter Echo
==================

This example demonstrates access of URL-encoded parameters in a GET request.  For example the following URL http://localhost:9005/example?foo=bar&baz=qux will render as the following HTML table.

foo   bar
baz   qux

     [file 004-url-param-echo.el not included]

POST Echo
=========

The following example echos back the content of the ``message'' field in a POST request.

     [file 005-post-echo.el not included]

Basic Authentication
====================

The following example demonstrates BASIC HTTP authentication.  The handler prompts an unauthenticated client for authentication by sending a ``WWW-Authenticate'' header.

     (ws-response-header process 401
       '("WWW-Authenticate" . "Basic realm=\"example\"")
       '("Content-type" . "text/plain"))

The client replies by setting the ``Authorization'' HTTP header which is parsed into a list of the form (PROTOCOL USERNAME . PASSWORD).  Currently only BASIC HTTP authentication is supported.

Note: BASIC HTTP authentication passes user credentials in plain text between the client and the server and should generally only be used with HTTPS network encryption.  While the Emacs web server currently doesn't support HTTPS network encryption it may be run behind an HTTPS proxy server (e.g., Apache or Nginx) with HTTPS support.

     [file 006-basic-authentication.el not included]

Org-mode Export
===============

The following example exports a directory of Org-mode files as either text, HTML or LaTeX.  The Org-mode export engine is used to export files on-demand as they are requested.

     [file 007-org-mode-file-server.el not included]

File Upload
===========

The following example demonstrates accessing an uploaded file.  This simple server accesses the file named ``file'' and returns it's sha1sum and file name.

     [file 008-file-upload.el not included]

A file may be uploaded from an HTML form, or using the curl program as in the following example.

     $ curl -s -F file=usr/share/emacs/24.3/etc/COPYING localhost:9008
     8624bcdae55baeef00cd11d5dfcfa60f68710a02  COPYING
     $ sha1sum /usr/share/emacs/24.3/etc/COPYING
     8624bcdae55baeef00cd11d5dfcfa60f68710a02  /usr/share/emacs/24.3/etc/COPYING

Web Socket
==========

Example demonstrating the use of web sockets for full duplex communication between clients and the server.  Handlers may use the ws-web-socket-connect function (see ws-web-socket-connect) to check for and respond to a web socket upgrade request sent by the client (as demonstrated with the new WebSocket JavaScript code in the example).  Upon successfully initializing a web socket connection the call to ws-web-socket-connect will return the web socket network process.  This process may then be used by the server to communicate with the client over the web socket using the process-send-string and ws-web-socket-frame functions. All web socket communication must be wrapped in frames using the ws-web-socket-frame function.

The handler must pass a function as the second argument to ws-web-socket-connect.  This function will be called on every web socket message received from the client.

Note: in order to keep the web socket connection alive the request handler from which ws-web-socket-connect is called must return the :keep-alive keyword, as demonstrated in the example.

     [file 009-web-socket.el not included]

Gzipped Transfer Encoding
=========================

HTTP Responses may be compressed by setting the ``gzip'' (or ``compress'' or ``deflate'') content- or transfer-encoding HTTP headers in ws-response-header.  Any further data sent to the process using ws-send will automatically be appropriately compressed.

     [file 016-content-encoding-gzip.el not included]

Chunked Transfer Encoding
=========================

Similarly, HTTP Responses may be sent using the ``chunked'' transfer encoding by passing the appropriate HTTP header to ws-response-header.  Any further data sent to the process using ws-send will automatically be appropriately encoded for chunked transfer.

     [file 017-transfer-encoding-chunked.el not included]

Function Index
**************

The following functions implement the Emacs Web Server public API.

Objects
=======

The following objects represent web servers and requests.

 -- Class: ws-server handlers process port requests
     Every Emacs web server is an instance of the ws-server class. Each instance includes the handlers association list and port passed to ws-start, as well as the server network process and a list of all active requests.

 -- Class: ws-request process pending context boundary index active headers
     The ws-request class represents an active web request.  The process field holds the network process of the client and may be used by handlers to respond to requests.  The headers field holds an alist of information on the request for use by handlers.  The remaining pending, context, boundary, index and active fields are used to maintain header parsing information across calls to the ws-filter function.

Starting and Stopping Servers
=============================

The following functions start and stop Emacs web servers.  The ws-servers list holds all running servers.

 -- Function: ws-start handlers port &optional log-buffer &rest network-args
     ws-start starts a server listening on port using handlers (see Handlers) to match and respond to requests. An instance of the ws-server class is returned.

 -- Variable: ws-servers
     The ws-servers list holds all active Emacs web servers.

 -- Function: ws-stop server
     ws-stop stops server deletes all related processes, and frees the server's port.  Evaluate the following to stop all emacs web servers.

          (mapc #'ws-stop ws-servers)

 -- Function: ws-stop-all
     ws-stop-all stops all emacs web servers by mapping ws-stop over ws-servers.

Convenience Functions
=====================

The following convenience functions automate many common tasks associated with responding to HTTP requests.

 -- Function: ws-response-header process code &rest headers
     Send the headers required to start an HTTP response to proc. proc should be a ws-request proc of an active request.

     For example start a standard 200 ``OK'' HTML response with the following.

          (ws-response-header process 200 '("Content-type" . "text/html"))

     The encoding may optionally be set in the HTTP header.  Send a UTF8 encoded response with the following.

          (ws-response-header process 200
                      '("Content-type" . "text/plain; charset=utf-8"))

     Additionally, when ``Content-Encoding'' or ``Transfer-Encoding'' headers are supplied any subsequent data written to proc using ws-send will be encoded appropriately including sending the appropriate data upon the end of transmission for chunked transfer encoding.

     For example with the header ("Content-Encoding" . "gzip"), any data subsequently written to proc using ws-send will be compressed using the command specified in ws-gzip-cmd.  See Gzipped Transfer Encoding and Chunked Transfer Encoding for more complete examples.

 -- Function: ws-send proc string
     Send string to process proc.  If any Content or Transfer encodings are in use, apply them to string before sending.

 -- Function: ws-send-500 process &rest msg-and-args
     ws-send-500 sends a default 500 ``Internal Server Error'' response to process.

 -- Function: ws-send-404 process &rest msg-and-args
     ws-send-500 sends a default 404 ``File Not Found'' response to process.

 -- Function: ws-send-file process path &optional mime-type
     ws-send-file sends the file located at path to process.  If the optional mime-type is not set, then the mime-type is determined by calling mm-default-file-encoding on path or is set to ``application/octet-stream'' if no mime-type can be determined.

 -- Function: ws-send-directory-list process directory &optional match
     ws-send-directory-list sends the a listing of the files located in directory to process.  The list is sent as an HTML list of links to the files.  Optional argument match may be set to a regular expression, in which case only those files that match are listed.

 -- Function: ws-in-directory-p parent path
     Check if path is under the parent directory.

          (ws-in-directory-p "/tmp/" "pics")
              => "/tmp/pics"

          (ws-in-directory-p "/tmp/" "..")
              => nil

          (ws-in-directory-p "/tmp/" "~/pics")
              => nil

 -- Function: ws-with-authentication handler credentials &optional realm unauth invalid
     Return a version of handler which is protected by credentials.  Handler should be a normal handler function (see Handlers) and credentials should be an association list of usernames and passwords.

     For example, a server running the following handlers,

          (list (cons '(:GET  . ".*") 'view-handler)
                (cons '(:POST . ".*") 'edit-handler))

     could have authorization added by changing the handlers to the following.

          (list (cons '(:GET  . ".*") view-handler)
                (cons '(:POST . ".*") (ws-with-authentication
                                       'org-ehtml-edit-handler
                                       '(("admin" . "password")))))

 -- Function: ws-web-socket-connect request handler
     If request is a web socket upgrade request (indicated by the presence of the :SEC-WEBSOCKET-KEY header argument) establish a web socket connection to the client.  Call handler on web socket messages received from the client.

          (ws-web-socket-connect request
            (lambda (proc string)
              (process-send-string proc
                (ws-web-socket-frame (concat "you said: " string)))))
              => #<process ws-server <127.0.0.1:34921>>

Customization Variables
=======================

The following variables may be changed to control the behavior of the web server.  Specifically the ws-*-cmd variables specify the command lines used to compress data according to content and or transfer encoding HTTP headers passed to ws-response-header.

 -- Variable: ws-compress-cmd
     Command used for the ``compress'' Content or Transfer coding.

 -- Variable: ws-deflate-cmd
     Command used for the ``deflate'' Content or Transfer coding.

 -- Variable: ws-gzip-cmd
     Command used for the ``gzip'' Content or Transfer coding.

GNU GENERAL PUBLIC LICENSE
**************************

     [file gpl.texi not included]

GNU Free Documentation License
******************************

     [file doclicense.texi not included]